What Does a Data Engineering Job Involve in 2024?
Data engineering is a hot topic in the AI industry right now. And as data’s complexity and volume grow, its importance across industries will only become more noticeable. But what exactly do data engineers do? Well, there’s a lot that goes into the job. Not only... Read more
How to Shift from Data Science to Data Engineering
Data engineering is a rapidly growing field, and there is a high demand for skilled data engineers. If you are a data scientist, you may be wondering if you can transition into data engineering. The good news is that there are many skills that data scientists... Read more
10 Data Engineering Topics and Trends You Need to Know in 2024
Now that we’re in 2024, it’s important to remember that data engineering is a critical discipline for any organization that wants to make the most of its data. These data professionals are responsible for building and maintaining the infrastructure that allows organizations to collect, store, process,... Read more
8 Data Lake Vendors to Make Your Data Life Easier in 2023
Data has to be stored somewhere. Data warehouses are repositories for your cleaned, processed data, but what about all that unstructured data your organization is starting to notice? Where does it go? To make your data management processes easier, here’s a primer on data lakes, and... Read more
The 30 Most Useful Python Libraries for Data Engineering
For the upcoming Data Engineering Summit on January 18th, we’ve reached out to some of the top experts in the field to speak on the topic. We observed from our discussions and research that the most popular data engineering programming languages include Python, Java, Scala, R,... Read more
Ten Areas of Data Engineering Every Team Should Excel At
Data engineering is undoubtedly one of the fastest-growing fields of technology and many companies are expanding the responsibility of their data teams to better manage the data stack. Gone are the days when it was simply batch-ingested ETL to analytics dashboards. Companies are increasingly seeking to... Read more
Announcing the Free Virtual Data Engineering Summit January 18th
ODSC and Ai+ couldn’t be more excited to announce our first-ever Data Engineering Summit. This free virtual event will take place on January 18th, 2023, and will be a full day of expert-led instruction in data engineering and related topics. The Data Engineering Summit is designed... Read more
How to Start Your Next Data Engineering Project
Many programmers who are just starting out struggle with starting new data engineering projects. In our recent poll on YouTube, most viewers admitted that they have the most difficulty with even starting a data engineering project. The most common reasons noted in the poll were: Finding... Read more
What to Consider When Building Data Pipelines
In 2021 we watched Fivetran raise $565 million, Airbyte $150 Million, Matillion $100 million, Rivery raised $16 million and Informatica went public. All of these companies have some piece of their business connected to data pipelines. Also sometimes referenced as ETL, ELT, E(t)LT, and CDC. For... Read more
A Day in the Life of a Data Engineer
The first time I considered becoming a data professional was in my last year of college when I took a class in machine learning and big data. I saw the potential impact and use cases of machine learning that have simply not been seen before. Even... Read more