I’ve been trying to build a model using machine learning today and I bumped into an error when I wanted to dummify my categorical predictors. It seemed I didn’t really know how Scikit’s OneHotEncoder worked. But I do now. And I want to share it with you. I had a…
In this blog post, two quick solutions to sample two pandas data frames in the same way. Why? Because sometimes you load your predictors (X) and your target (y) from two different files and you don’t want to put them together in one table. Sampling two data frames with a…
The SettingWithCopyWarning message is a confusing warning to many who are new to Pandas. If you’ve ever taken a computer science course, you might be aware of passing/copying by value or by reference. Well, it very much applies to pandas DataFrames too. Let’s go. Basically, when you are slicing a…
This is a cool one I used for a feature engineering task I did recently. I had multiple documents in a Pandas DataFrame, in long format. These documents belonged to people and it had an n:1 relation: people could have multiple documents. I was wondering how to concatenate each person’s…
Coming from R, I’m not a big fan of MultiIndex DataFrames in Pandas. I definitely see the merits, but it just doesn’t feel right within a machine learning and feature engineering context. However, sometimes you will end up with a MultiIndex DataFrame, after some ninja line of code. In this…
There are many ways to remove a column in a pandas DataFrame. However, some ways are better than others. In this blog post, I elaborate on multiple solutions and what the pros and cons are. First, let’s load the iris dataset from the Seaborn package on GitHub. Drop a pandas…
In some situations, especially when adding some basic error handling to your Python scripts, you want to check if a column exists before performing operations on it. In this blog post I tell you how. For starters, let’s load the iris dataset from the Seaborn package on GitHub. I’ll be…
Another blog post about an absolute starter subject. Calculating the numbers of rows of a pandas DataFrame is really simple. Yet there are many ways to do them. In this blog post, we elaborate on each and every one of the possible solutions. By looking at the codebase of pandas,…
It’s an operation I use quite a lot, but I never took the time to find out which solution is fastest. That’s why I decided to dedicate this blog post on how to concatenate columns in pandas (Python) and their execution speed. I’ve been working with the “donors” dataset from…
On my list of things that frustrate me when using notebooks is the fact that you do not see the output of every expression of a cell by default. For example, during an exploratory analysis phase, I want to see the result of two pandas operations or function within the…