In this blog post, we’ll use GeoPandas and Folium in Python to plot a choropleth map of New York’s unemployment rate per borough. Not because it’s extremely interesting, but because it took me a while to get all the elements in order, and I want to help you save some…
There are very few occasions when you really need a hierarchical column index in Pandas. Mostly, you just want to identify a column with a single column name. In this blog post, we’ll learn how to properly flatten a DataFrame with MultiIndex columns. One of the things that annoys me…
Reading CSVs is always a little bit living on the edge, especially when multiple regions are involved in producing them. In this blog post, we’re solving UnicodeDecodeError: ‘utf-8’ codec can’t decode byte […] in position […]: invalid continuation byte. Important, I’m assuming you got the error when you used Pandas’…
There are many situations that you have an 1:n relationship, where you would have wanted an 1:1 relationship. Sometimes, it makes sense to concatenate the strings from rows belonging to the same group. Let’s do that in Python, using Pandas. Imagine, you have the following table that contains an ID…
This blog post is the first post in a two-part series on subsetting Pandas DataFrame rows using chained conditions. In this post, we tackle the following ValueError. Part 1: Bitwise operators Part 2: Parentheses Filtering (or subsetting) a DataFrame can easily be done using the loc property, which can access…
This blog post is the second post in a two-part series on subsetting Pandas DataFrame rows using chained conditions. In this post, we tackle the following TypeError. Part 1: Bitwise operators Part 2: Parentheses Filtering (or subsetting) a DataFrame can easily be done using the loc property, which can access…
Various SaaS notebook solutions have row numbers enabled by default. It’s a pity that this is not the case within Jupyter. In this blog post, two ways to enable row numbers in Jupyter notebooks. Enable row numbers via hotkeys You can enable row numbers in one code cell by pressing…
In this blog post, you’ll learn how you can run scraping jobs using Selenium from Google Cloud Run, a fully managed solution that eliminates the burden of configuring and managing a Kubernetes cluster. It simplifies your production pipeline and fully automates your scraping process. Take notice, I’m not an expert…
This is a brief overview of the possibilities for setting environment variables in a virtual environment (venv). Set environment variables in Python code Environment variables can be set inside your Python script. All child processes will inherit the environment variables and their values. Set environment variables in activate.bat or activate.ps1…
I was running a scraping job from a Docker container using Python and the psycopg2 adapter that wrote the results to a Postgres database managed on DigitalOcean (DO). On random occasions, the container died, for some reason. I started debugging and these are my findings. The error I ran into…