JSON or JavaScript Object Notation is a popular file format for storing semi-structured data. Transforming it to a table is not always easy and sometimes downright ridiculous. However, Pandas offers the possibility via the read_json function. If you are not familiar with the orient argument, you might have a hard…
Brief blog post on a silly Python error. I’m still working on my ML project started hypertuning my sklearn pipeline — manually. To set the parameters of my pipeline in every iteration, I had to use set_params. My estimator is called model_pipeline and I needed to set the parameters via…
It’s been a while since I worked on a machine learning project from exploratory phase to model development. I bumped into a real newbie error and decided to write it down for my future self, and for you of course. Here’s the code I wrote and I hoped to get…
Pandas can be somewhat puzzling, sometimes. It has a ton load of functionalities, but that can make the syntax and methods obscure. Simply judging from the method name, the ‘join’ and ‘merge’ method could be the same thing. However, they aren’t. Here’s an error that I used to run into…
I’ve spent hours trying to find an elegant solution for this problem, and I’m ashamed about how easy it eventually was. I had a column with comma-separated values that I wanted to create features from. Each value should represent one feature. I got started with building loops over the values…
Surprisingly, the Pandas error “columns overlap but no suffix specified”, is one I ran into surprisingly late. Solving it is usually very straightforward. We’ll tackle it in this blog post. First, let’s take a closer look at the error: “ValueError: columns overlap but no suffix specified: Index([<list of columns>], dtype=’object’)”…
Creating a tracking solution has never been easier. There are numerous (free) tools available to collect and store event data. In this blog post I explain how to collect data using Fivetran webhooks and storing it in ElephantSQL, a managed PostgreSQL database. In this blog post, we will: Set up…
In this short blog post, I tackle an error related to a classic problem within machine learning: how to treat unseen categorical values and solve the “found unknown categories” error. Imagine you have a train and a test data set with the following values in a column: Train: [‘green’, ‘red’,…
In the past couple of weeks, I started to use sklearn pipelines more intensively. It seemed like a good project to find out more about them and share my experiences in a blog post. So here it is: a sklearn pipeline tutorial. Why pipelines? In most machine learning projects, you…
In a previous post, I explained how you can sample two Pandas DataFrame exactly the same way. In this blog post, I want to use that helper function to undersample your predictors and target variable. When you are working with an imbalanced data set, it’s often good practice to under-…