fbpx
Statistics, Simians, the Scottish, and Sizing up Soothsayers
A predictive model can be a parametrized mathematical formula, or a complex deep learning network, but it can also be a talkative cab driver or a slides-wielding consultant. From a mathematical point of view, they are all trying to do the same thing, to predict what’s... Read more
Generalizing Abstract Arrays: opportunities and challenges
Introduction: generic algorithms with AbstractArrays Somewhat unusually, this blog post is future-looking: it mostly focuses on things that don’t yet exist. Its purpose is to lay out the background for community discussion about possible changes to the core API for AbstractArrays, and serves as background reading... Read more
More notebooks for Think Stats
More notebooks for Think Stats As I mentioned in the previous post, I am getting ready to teach Data Science in the spring, so I am going back through Think Stats and updating the Jupyter notebooks.  I am done with Chapters 1 through 6 now. If you are reading... Read more
Do Resampling Estimates Have Low Correlation to the Truth?
The Answer May Shock You. One criticism that is often leveled against using resampling methods (such as cross-validation) to measure model performance is that there is no correlation between the CV results and the true error rate. Let’s look at this with some simulated data. While... Read more
Thomas originally posted this article here at http://twiecki.github.io  Hierarchical models are underappreciated. Hierarchies exist in many data sets and modeling them appropriately adds a boat load of statistical power (the common metric of statistical power). I provided an introduction to hierarchical models in a previous blog post: Best... Read more
Exploring the Relationship between Religion and Demographics in R
Today’s guest post is by Julia Silge. Take a look at her work on (“Mapping US Religion Adherence by County in R“) where she demonstrated how to work with US religion adherence data in R. In this post she explores the relationship between that dataset and US... Read more
New notebooks for Think Stats
Getting ready to teach Data Science in the spring, I am going back through Think Stats and updating the Jupyter notebooks.  When I am done, each chapter will have a notebook that shows the examples from the book along with some small exercises, with more substantial... Read more
This is a two-part series about using machine learning to hack my taste in music. In this first piece, I applied unsupervised learning techniques and tools on Pandora data to analyze songs that I like. The second part, which will be published soon, is about using supervised on... Read more
I was reading yet another blog post titled “Why our team moved from to ” (I forgot which one) and I started wondering if you can generalize it a bit. Is it possible to generate a N * N contingency table of moving from language X... Read more
It wasn’t an overbooking problem. United Airlines was trying to move four flight crew members to the next airport. They forced passengers to get off the plane with the consequences we saw on the video from last Sunday, but don’t take our word for it. Let’s talk... Read more