Smoothing Data in SQL
A problem found throughout the world of data is how to distinguish signal from noise. When dealing with data that comes in a sequence, such as time-series data (the most familiar example but by no means the only example), a frequent method of dealing with the problem is to... Read more
An Introduction to AWS Networking – Virtual Private Cloud
Cloud computing is one of the major trends in computing today and has been for many years. Public cloud providers have transformed the start-up industry and what it means to launch a service from scratch. We no longer need to build our own infrastructure; we can pay public cloud... Read more
A Data Pattern with an R data.table Solution.
Summary: This blog examines a loading pattern seen often with government-generated, web-accessible data. The data comprise millions of records across multiple text or csv files, generally demarcated by time. The files may present different, but overlapping, attributes, while much of the data has a character representation, posing the challenge... Read more
Data Manipulation in R
Not all datasets are as clean and tidy as you would expect. Therefore, after importing your dataset into RStudio, most of the time you will need to prepare it before performing any statistical analyses. Data manipulation can even sometimes take longer than the actual analyses when the quality of the... Read more
Creating if/elseif/else Variables in Python/Pandas
Frequencies and Chaining in Python-Pandas
A few years ago, in a Q&A session following a presentation I gave on data analysis (DA) to a group of college recruits for my then consulting company, I was asked to name what I considered the most important analytic technique. Though a surprise to the audience, my answer,... Read more
The goal of a data analysis pipeline in Python is to allow you to transform data from one state to another through a set of repeatable, and ideally scalable, steps. Problems for which I have used data analysis pipelines in Python include: Processing financial / stock market data, including... Read more
Guide to R and Python in a Single Jupyter Notebook
Why pick one when you can use both at the same time? R is primarily used for statistical analysis, while Python provides a more general approach to data science. R and Python are object-oriented towards data science for programming language. Learning both is an ideal solution. Python is a... Read more
Complex Queries in SQL
Hopefully, your SQL queries aren’t tangled like this. (Photo: Author) Texts on SQL are good at providing the basic templates of SQL syntax, but sometimes the queries in those books are a little idealized compared to real life, with no more than two or three tables to join and... Read more
PI and Simulation Art in R
I spent the better part of an afternoon last week perusing a set of old flash drives I’d made years ago for my monthly notebook backups. One that especially caught my attention had a folder of R scripts, probably at least 15 years old — harking back to my... Read more