Home » gcp

gcp

How to deploy a scraping script and Selenium in Google Cloud Run

  • by
  • 8 min read

In this blog post, you’ll learn how you can run scraping jobs using Selenium from Google Cloud Run, a fully managed solution that eliminates the burden of configuring and managing a Kubernetes cluster. It simplifies your production pipeline and fully automates your scraping process. Take notice, I’m not an expert… 

Solve “invalid table reference” when streaming into BigQuery

  • by
  • 2 min read

Recently I decided to start streaming clickstream data (from this blog, indeed) into BigQuery using the insertAll method. It is a relatively cheap solution that requires almost no technical configuration. Until I bumped into an error. Let’s tackle it in this blog post. The error I encountered is: “google.api_core.exceptions.BadRequest: 400… 

Solving “Permission ‘cloudfunctions.functions.get’ denied on resource”

  • by
  • 2 min read

Your first steps in setting up Cloud Build for deploying Google Cloud Functions but running into yet another error? You’re not alone. With this blog post, I hope to put you on your way again. Have you run into the following error? “Permission ‘cloudfunctions.functions.get’ denied on resource […]” Here’s what’s… 

Connect cumul.io to a MySQL Database

  • by
  • 4 min read

Being a little bit bored with the default visualization tools in R and Python, I decided to look into the available visualization tools that are out there in the data ecosystem. The Belgian visualization tool cumul.io focusses on integration in platforms, but it also seems like a solid entry-level visualization… 

Solved: “The caller does not have permission” – Using the API with a private Google Spreadsheet

Ahh, user rights. The cause and solution to all of life’s identification problems. In this blog post I explain how you can access (private) google spreadsheets using the Python gspread library. Before you get started: make sure you have administrator rights to the spreadsheets you are trying to work with.… 

Reading from and writing files to GCP Storage in an AI Platform job

  • by
  • 1 min read

When you’re trying to access a CSV file stored in Google Cloud Storage when submitting a job to AI Platform, your first reflex is probably to use pandas’ read_csv. However, this will produce the following error: ImportError: The gcsfs library is required to handle GCS files That’s because pandas is… 

Creating a GCP Service Account and linking it to your system

When you haven’t gone through the Google Cloud Platform authentication process in your development environment, you might run into the following error message. In this blog post, I explain how you can solve it. google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the…