There are very few occasions when you really need a hierarchical column index in Pandas. Mostly, you just want to identify a column with a single column name. In this blog post, we’ll learn how to properly flatten a DataFrame with MultiIndex columns. One of the things that annoys me…
Reading CSVs is always a little bit living on the edge, especially when multiple regions are involved in producing them. In this blog post, we’re solving UnicodeDecodeError: ‘utf-8’ codec can’t decode byte […] in position […]: invalid continuation byte. Important, I’m assuming you got the error when you used Pandas’…
There are many situations that you have an 1:n relationship, where you would have wanted an 1:1 relationship. Sometimes, it makes sense to concatenate the strings from rows belonging to the same group. Let’s do that in Python, using Pandas. Imagine, you have the following table that contains an ID…
This blog post is the first post in a two-part series on subsetting Pandas DataFrame rows using chained conditions. In this post, we tackle the following ValueError. Part 1: Bitwise operators Part 2: Parentheses Filtering (or subsetting) a DataFrame can easily be done using the loc property, which can access…
This blog post is the second post in a two-part series on subsetting Pandas DataFrame rows using chained conditions. In this post, we tackle the following TypeError. Part 1: Bitwise operators Part 2: Parentheses Filtering (or subsetting) a DataFrame can easily be done using the loc property, which can access…
Recently, I’ve been working with Hive a lot and I bumped into this simple thing that I wasn’t aware of existed in SQL. So I decided to share in this blog post. Let’s say you need to extract a month from a timestamp. The following code snippet will return a…
Various SaaS notebook solutions have row numbers enabled by default. It’s a pity that this is not the case within Jupyter. In this blog post, two ways to enable row numbers in Jupyter notebooks. Enable row numbers via hotkeys You can enable row numbers in one code cell by pressing…
For my latest assignment, I have to work with PostGIS/Postgres a lot. I tried a query that inserts the result of a SELECT statement into a new table. However, I ran into a peculiar error. The error I ran into was: A scenario where this error can arise is when…
In this blog post we’ll calculate the share of a row per group. There are multiple ways to do this: with aggregate functions and with analytic functions. We’ll go over both and demonstrate how analytic functions are far superior in terms of readability. First, let’s create a table to demonstrate…
In this blog post, you’ll learn how you can run scraping jobs using Selenium from Google Cloud Run, a fully managed solution that eliminates the burden of configuring and managing a Kubernetes cluster. It simplifies your production pipeline and fully automates your scraping process. Take notice, I’m not an expert…