One of the most frequent used Excel functions is probably SUMIF and its SUMIFS variant. In this article, you’ll learn how to do exactly the same in PySpark. What is the sumif function? In Excel, the SUMIF function is an aggregation function for summing values from a column, but only…
Why do people put data in a list in the first place? Because it can be really darn handy. In this blog post I elaborate on some good use cases for putting data frames in a list. Loading a lot of files In many situations you will be confronted with…
Replacing, excluding or imputing missing values is a basic operation that’s done in nearly all data cleaning processes. In my third blog post on Julia I give an overview of common solutions for replacing missing values. First, let’s create a dummy DataFrame as an example. Both columns, a and b,…
I recently started working with Julia. I’ve been reading some great things about it and I felt I needed to try it out. As I’m working myself through a data science project, I plan on writing a couple of blog posts on stuff that needs some explanation. In this first…
In this 2-part blog post I want to discuss two files that allow you to customize your R experience. In this part I discuss the .Renviron file. First I answer what it does, where you can find it and finally we get to the part where we actually use it.…
In this blog post I elaborate on the reality that statistical models often obscure the fact that some problems can actually be solved in a way that requires less time, resources and headaches. By using reference models and heuristics, decision makers can evaluate if those seemingly brilliant consultants are solving…
Having R as my data science lingua franca, some things in pandas feel less intuitive. As I’m slowly transitioning to pandas, I’ll be posting a lot of examples over the next couple of weeks. Here’s the first one: concatenating columns values of a pandas DataFrame into another (new) column. First…
Yesterday, Iceland gained a point against Argentina by putting up an icy wall in front of their goal. Nothing adds more spice to a football game than an underdog holding out against a football superpower. Sometimes details are responsible for second tier teams to win their group phase and for…