What is knyfe?
Knyfe is a python utility for rapid exploration of datasets. Use it when you have some kind of dataset and you want to get a feel for how it is composed, run some simple tests on it, or prepare it for further processing. The great thing about knyfe is... Read more
Another batch of Think Stats notebooks
Getting ready to teach Data Science in the spring, I am going back through Think Stats and updating the Jupyter notebooks.  When I am done, each chapter will have a notebook that shows the examples from the book along with some small exercises, with more substantial exercises at the... Read more
General Tips for Web Scraping with Python
The great majority of the projects about machine learning or data analysis I write about here on Bigish-Data have an initial step of scraping data from websites. And since I get a bunch of contact emails asking me to give them either the data I’ve scraped myself, or help with getting... Read more
Regular Expression & Treemaps to Visualize Emergency Department Visits
It’s been a while since my last post on some TB WHO data. A lot has happened since then, including the opportunity to attend the Open Data Science Conference (ODSC) East held in Boston, MA. Over a two day period I had the opportunity to listen to a number of leaders... Read more
Python as a way of thinking
This article contains supporting material for this blog post at Scientific American.  The thesis of the post is that modern programming languages (like Python) are qualitatively different from the first generation (like FORTRAN and C), in ways that make them effective tools for teaching, learning, exploring, and thinking. I presented... Read more
More notebooks for Think Stats
More notebooks for Think Stats As I mentioned in the previous post, I am getting ready to teach Data Science in the spring, so I am going back through Think Stats and updating the Jupyter notebooks.  I am done with Chapters 1 through 6 now. If you are reading the book, you... Read more
Handwritten digits recognition using Tensorflow with Python
The progress in technology that has happened over the last 10 years is unbelievable. Every corner of the world is using the top most technologies to improve existing products while also conducting immense research into inventing products that make the world the best place to live. Some of these... Read more
RIDE – A New Data Science IDE for Python and R
The data science world is split into two parts: the (i)Python and the R community. Both groups offer a plethora of tools and libraries enriching our work-life as a data scientist. Interestingly, many of the offerings are complementary, such that professional data scientists should know both environments to pick... Read more
Hello all and welcome to the second of the series – NLP with NLTK. The first of the series can be found here, incase you have missed. In this article we will talk about basic NLP concepts and use NLTK to implement the concepts. Contents: Corpus Tokenization/Segmentation Frequency Distribution... Read more
Dealing with arrays which are bigger than memory – an introduction to biggus
I often deal with huge gridded datasets which either stretch or indeed are beyond the limits of my computer’s memory. In the past I’ve implemented a couple of workarounds to help me handle this data to extract meaningful analyses from them. One of the most intuitive ways of reducing... Read more