It’s been a while since I worked on a machine learning project from exploratory phase to model development. I bumped into a real newbie error and decided to write it down for my future self, and for you of course. Here’s the code I wrote and I hoped to get…
Pandas can be somewhat puzzling, sometimes. It has a ton load of functionalities, but that can make the syntax and methods obscure. Simply judging from the method name, the ‘join’ and ‘merge’ method could be the same thing. However, they aren’t. Here’s an error that I used to run into…
I’ve spent hours trying to find an elegant solution for this problem, and I’m ashamed about how easy it eventually was. I had a column with comma-separated values that I wanted to create features from. Each value should represent one feature. I got started with building loops over the values…
Surprisingly, the Pandas error “columns overlap but no suffix specified”, is one I ran into surprisingly late. Solving it is usually very straightforward. We’ll tackle it in this blog post. First, let’s take a closer look at the error: “ValueError: columns overlap but no suffix specified: Index([<list of columns>], dtype=’object’)”…
Creating a tracking solution has never been easier. There are numerous (free) tools available to collect and store event data. In this blog post I explain how to collect data using Fivetran webhooks and storing it in ElephantSQL, a managed PostgreSQL database. In this blog post, we will: Set up…
In this short blog post, I tackle an error related to a classic problem within machine learning: how to treat unseen categorical values and solve the “found unknown categories” error. Imagine you have a train and a test data set with the following values in a column: Train: [‘green’, ‘red’,…
In the past couple of weeks, I started to use sklearn pipelines more intensively. It seemed like a good project to find out more about them and share my experiences in a blog post. So here it is: a sklearn pipeline tutorial. Why pipelines? In most machine learning projects, you…
In a previous post, I explained how you can sample two Pandas DataFrame exactly the same way. In this blog post, I want to use that helper function to undersample your predictors and target variable. When you are working with an imbalanced data set, it’s often good practice to under-…
I’ve been trying to build a model using machine learning today and I bumped into an error when I wanted to dummify my categorical predictors. It seemed I didn’t really know how Scikit’s OneHotEncoder worked. But I do now. And I want to share it with you. I had a…
In this blog post, two quick solutions to sample two pandas data frames in the same way. Why? Because sometimes you load your predictors (X) and your target (y) from two different files and you don’t want to put them together in one table. Sampling two data frames with a…
