Home » python


How to deploy a scraping script and Selenium in Google Cloud Run

  • by
  • 8 min read

In this blog post, you’ll learn how you can run scraping jobs using Selenium from Google Cloud Run, a fully managed solution that eliminates the burden of configuring and managing a Kubernetes cluster. It simplifies your production pipeline and fully automates your scraping process. Take notice, I’m not an expert… 

Set environment variables in a Python venv (Windows)

  • by
  • 1 min read

This is a brief overview of the possibilities for setting environment variables in a virtual environment (venv). Set environment variables in Python code Environment variables can be set inside your Python script. All child processes will inherit the environment variables and their values. Set environment variables in activate.bat or activate.ps1… 

Using TA-Lib in Deepnote

  • by
  • 2 min read

Using TA-Lib in a Jupyter notebook isn’t straightforward, and using it in a hosted solution like Deepnote could use a tutorial too. In this blog post, we’ll set up the popular Python wrapper for TA-lib. A prerequisite for installing the TA-lib Python wrapper is that you have the TA-lib library… 

Solve Pandas “ValueError: cannot reindex from a duplicate axis”

Recently, I’ve been working with Pandas DataFrames that had a DateTime as the index. When I tried reindexing the DataFrame (using the reindex method), I bumped into an error. Let’s find out what causes it and how to solve it. The Python error I’m talking about is: A “duplicate axis”?… 

Solve “Message: ‘chromedriver’ executable needs to be in PATH”

  • by
  • 2 min read

All of us data hoarders get to a point where we need to circumvent in-place rules that prevent us from scraping the web. Enter: Selenium. The world’s favorite tool for automating tasks in a browser. Selenium uses ChromeDriver, initially created for automated testing, but also a splendid scraping tool. In… 

Solve “PermissionError: [Errno 1] Operation not permitted” when deploying to DigitalOcean

I’m a data scientist. Deploying is not part of my core skills, but something I need to be able to do when working on hobby projects. I recently embraced DigitalOcean to deploy my Django (Python) Project and want to elaborate on a problem I ran into. The error that appeared…