In this article, you’ll learn how you can get metadata from your pipeline run inside your pipeline’s notebooks (or other activities). The documentation on this subject is relatively limited, so I hope it will prevent a lot of wasted time. In Airflow, you have macros, to refer to the metadata…
In R you’ll use rbind, in Pandas you’ll use append, in SQL you’ll use UNION. But how do you append files in PowerQuery? In this example I will combine multiple files that contain data for every month of 2019. For convenience, I use the refundable pharmaceutical products in Belgium. Before…
Let’s start with saying that importing MS Access files into R can be easy, but it isn’t necessarily so. There’s some compatibility issues that you need to know about. This blog post elaborates on these errors and how I solved them. Import tables from an .mdb file Opening a connection…
A couple of months ago I wrote a short blog post about a new library I discovered: openxlsx. Here’s a very short blog post as a followup. To read in Excel files in R using the openxlsx library, I use the following function: However, one day, I ran into the…
As you could have guess from my previous blog post, I started using Microsoft’s Power Platform intensively for a project in healthcare. A thing that surprised me is how easy it is to generate PDFs, for example for generating specific lists, using PowerAutomate. That is, if you’re fond of OneDrive.…
I recently took a course in Microsoft PowerQuery, a visual user interface to data wrangling. It’s nowhere as powerful as Python’s pandas for example, but it does provide a convenient way for analysts without coding skills to clean small to medium-sized datasets. One aspect I found to be really intuitive…
Out of the blue, when coding Python in Visual Studio Code, you might run into the following error. “inconsistent use of tabs and spaces in indentation” This happens, because somewhere in your code, indentation is different than in the rest of your code. Generally, Python doesn’t care what kind of…
It happened too many times before I write this blog post. Oftentimes, when I read Excel (xls or xlsx) files into R, I encounter this strange phenomenon where dates are converted to a 5 number integer. Here is how to fix it. For example, 02/01/2017 (dd/MM/yyyy) would be converted to…
Historically, browsers have had a great deal of control over the online experience of end users. Since their genesis, several different browsers have competed for dominant market share. With the arrival of Intelligent Tracking Prevention (ITP), it seems that big tech is now using browser standards to target each other’s…
Another management book? Not just a management book, but the story of Satya Nadella. Who? The son of a marxist economist and a drama professor but mostly known as Microsoft’s less famous, less rich, but current CEO. A brief review. Nadella’s book ‘Hit Refresh’ is a triptych. The first part…