Data Wrangling Archives - Open Data Science - Your News Source for AI, Machine Learning & more

The Importance of Processing Data the Right Way

Data Wranglingposted by Caspar Wylie, ODSC October 16, 2018

There are so many different aspects of training a neural network that affect its performance. Many data scientists spend too much time thinking about learning rates, neuron structures, and epochs before actually using correctly optimized data. Without properly formatting data, your neural network will be useless, regardless of the... Read more

Top Data Wrangling Skills Required for Data Scientists

Data Wrangling Modeling data munging data wranglingposted by Daniel Gutierrez, ODSC September 19, 2018

Whatever you want to call it – data wrangling, data munging, or data transformation, the part of the Data Science Process sitting in between data acquisition and exploratory data analysis (EDA) is one of the core skills a data scientist must have. It includes a set of tasks you... Read more

No Need for Deciphering – Learn How to Make Your Own Dataset Instead

Conferences Data Wrangling Modeling odsc east 2018posted by Jacquelyn Elias, ODSC August 24, 2018

Key Takeaways: By creating, capturing, and curating data, one can practice “data creationism” and be creative with data to make your own dataset. While Iris and Titanic are well-known datasets available to experiment with machine learning and data science, challenge yourself to create your own dataset. Anything can be... Read more

Bayesian Estimation, Group Comparison, and Workflow

Data Wrangling Modelingposted by Eric Ma July 31, 2018

Over the past year, having learned about Bayesian inference methods, I finally see how estimation, group comparison, and model checking build upon each other into this really elegant framework for data analysis. Parameter Estimation The foundation of this is “estimating a parameter”. In a typical situation, we are most... Read more

How Well Did Data Scientists Predict the 2018 World Cup? (Hint: Not Very)

Data Wrangling Modeling Predictive Analytics Research Statistics popular culture world cupposted by Alex Amari July 26, 2018

This year’s World Cup in Russia was the most watched sporting event in history. GlobalWebIndex reports that up to 3.4 billion people – around half of the world’s population – watched some part of the tournament. As with past World Cups, a global prediction market emerged allowing spectators to... Read more

The Best Mario Kart Character According to Data Science

Data Wrangling Modelingposted by Civis Analytics Team July 20, 2018

Mario Kart was a staple of my childhood — my friends and I would spend hours after school as Mario, Luigi, and other characters from the Nintendo universe racing around cartoonish tracks and lobbing pixelated bananas at each other. One thing that always vexed our little group of would-be speedsters was... Read more

Starting a Data Science Project

Career Insights Data Wrangling Modelingposted by Kristen Kehrer July 18, 2018

I spoke in a Webinar this past Saturday about how to get into Data Science. One of the questions asked was “What does a typical day look like?” I think there is a big opportunity to explain what really happens before any machine learning takes place for a large... Read more

Predicting the Truncated xorshift32* Random Number Generator

Data Wrangling Modelingposted by Daniel Lemire July 17, 2018

Software programmers need random number generators. For this purpose, they often use functions with outputs that appear random. Gerstmann has a nice post about Better C++ Pseudo Random Number Generator. He investigates the following generator: uint32_t xorshift(uint64_t *m_seed) { uint64_t result = *m_seed * 0xd989bcacc137dcd5ull; *m_seed ^= *m_seed >> 11;... Read more

How Quickly Can You Compute the Dot Product Between Two Large Vectors?

Data Wrangling Modelingposted by Daniel Lemire July 10, 2018

A dot (or scalar) product is a fairly simple operation that simply sums the many products: float sum = 0; for (size_t i = 0; i < len; i++) { sum += x1 * x2; } return sum; It is nevertheless tremendously important. You know these fancy machine learning... Read more

Data Processing on Modern Hardware

Data Wrangling Modelingposted by Daniel Lemire July 2, 2018

If you had to design a new database system optimized for the hardware we have today, how would you do it? And what is the new hardware you should care about? This was the topic of a seminar I attended last week in Germany at Dagstuhl. Here are some thoughts:... Read more