Recently, I needed to calculate the relative change of a value compared to the first value within a DataFrame group, a GroupBy object. With SQL, I would use a window function (e.g. PARTITION BY in PostgreSQL). In this article, we’ll do it in Pandas (Python). To demonstrate, let’s start with…
In this blog post, I explore the options for taking the row-wise sum of a subset of columns from a Pandas DataFrame. Let’s try to take the row-wise sum of the columns first_column, second_column, and third_column. This means we’re leaving other_column out. But first, let’s take a step back. If…
In this blog post we’ll do a small thing to remove trailing or leading whitespaces from a DataFrame’s column names. This is something that can occur when working with files that haven’t been properly formatted. There are multiple things you can do to fix this. First, let’s create some dummy…
In this article, you’ll learn how to execute shell commands using the subprocess package in Python. This article is part of a two-part series related to running shell commands from within Python. Part 1: Execute shell commands with the os package Part 2: Execute shell commands with the subprocess package…
Running shell (e.g. Bash) commands in Python is fairly easy using the os package. In this article, I outline two ways to run shell commands in Python: using the system method and the popen method. This article is part of a two-part series related to running shell commands from within…
I would like to briefly elaborate on an error I ran into, which is due to limited documentation on the Google Cloud Platform API. With this article, I hope to save you the time and effort to find the solution. I’ve been following an infrastructure-as-code approach to deploying GCP resources…
Because there are (too) many ways to cast a Pandas DataFrame from long to wide format, I decided to list four ways to achieve that goal. The four functions I describe in this article are the following. Function Object aggregation Can handle NaNs pivot DataFrame no no pivot_table DataFrame yes…
In this article we elaborate on the multiple ways to remove the final line of a CSV in Python when loading it with Pandas’ read_csv function. Remove final row(s) after loading the file Removing the final row from a Pandas DataFrame can be done with a simple slice. The following…
I’ve been working with Airflow recently, and I needed to configure an operator by passing it the execution time. Or at least I thought so. I really felt like writing findings down, because if you’re in the dark here, like I was today, there are some fundamental Airflow principles you…
Sometimes, just sometimes, you really want to get rid of the legend(s) in your Folium Choropleth map. However, there is no method within the Choropleth object to achieve that. Currently, the most straightforward way of removing the legend from your map is by looping over all the layers in the…