R Tip: Use match_order() to Align Data
R tip. Use wrapr::match_order() to align data. Suppose we have data in two data frames, and both of these data frames have common row-identifying columns called “idx“. library("wrapr") d1 <- build_frame( "idx", "x" | 3 , "a" | 1 , "b" | 2 , "c" ) d2 <- build_frame( "idx", "y" |... Read more
Sheddable Requests: The Intersection of Hackweeks, Book Clubs, and Site Reliability Engineering
One of the things I love about working at Civis is the opportunity we have for continuous learning. Not long ago I had the opportunity to be involved in a book club which read through Google’s Site Reliability Engineering book. One of the essays in this book addressed various methods for handling overload.... Read more
Laminar flow with ggplot2 and gganimate
Preface I’ve realized that all my previous posts were quite substantial in length and took quite a long time to create them. From this point forward I’ll be generating posts of shorter length (partially for my sanity and more for my impulsivity with ideas). A few of these posts won’t be... Read more
R Tip: Use let() to Re-Map Names
Another R tip. Need to replace a name in some R code or make R code re-usable? Use wrapr::let(). Here is an example involving dplyr. Let’s look at some example data: library("dplyr") library("wrapr") starwars %>% select(., name, homeworld, species) %>% head(.) # # A tibble: 6 x 3 # name homeworld species #... Read more
Using Excel for Data Entry
This article shows you how to enter data so that you can easily open in statistics packages such as R, SAS, SPSS, or jamovi (code or GUI steps below). Excel has some statistical analysis capabilities, but they often provide incorrect answers. For a comprehensive list of these limitations, see http://www.forecastingprinciples.com/paperpdf/McCullough.pdfand http://www.burns-stat.com/documents/tutorials/spreadsheet-addiction. Simple Data... Read more
Dask Release 0.17.2
This work is supported by Anaconda Inc. and the Data Driven Discovery Initiative from the Moore Foundation. I’m pleased to announce the release of Dask version 0.17.2. This is a minor release with new features and stability improvements. This blogpost outlines notable changes since the 0.17.0 release on February 12th. You can... Read more
When shuffling large arrays, how much time can be attributed to random number generation?
It is well known that contemporary computers don’t like to randomly access data in an unpredictible manner in memory. However, not all forms of random accesses are equally harmful. To randomly shuffle an array, the textbook algorithm, often attributed to Knuth, is simple enough: void swap(int arr, int i,... Read more
R Tip: Use let() to Re-Map Names
Another Rtip. Need to replace a name in some R code or make R code re-usable? Use wrapr::let(). Here is an example involving dplyr. Let’s look at some example data: library("dplyr") library("wrapr") starwars %>% select(., name, homeworld, species) %>% head(.) # # A tibble: 6 x 3 # name homeworld species #... Read more
Apache Cassandra and ALLOW FILTERING
Prologue Aspiring Cassandra engineer-apprentice was fiddling with Cassandra cluster trying to fetch the data he needed. For a while, he was receiving strange responses from the server. But after hacking his way through the CQL, he finally received the response he was looking for. He felt so proud… For a moment.... Read more
R Tip: Break up Function Nesting for Legibility
There are a number of easy ways to avoid illegible code nesting problems in R. In this R tip we will expand upon the above statement with a simple example. At some point it becomes illegible and undesirable to compose operations by nesting them, such as in the following code. head(mtcars[with(mtcars, cyl... Read more