Over six months ago I published a blog post about the existing ‘unfairness’ in the Eurovision Song Contest. In several graphs I showed how there is a bias for viewers and jury to vote for candidates and their acts who performed at a later time during the contest. Although that conclusion is definitely the right one, my methods used back then deserve some scrutiny.
To prove that an average contestant’s final score increased depending on their act’s appearing order during the night, I used the following plot. It is a linear regression line between the two variables, where the standardized score is the dependent variable (on the y-axis).
However, there is something crucially wrong with this plot: their is no bivariate normality. In more popular terms, the residuals are not distributed normally: there are more dots under than above the curve. There are many ways to transform the data to make the residuals more normal but (1) the result loses interpretability and (2) I did not find a proper transformation to meet the normal residuals assumptions.
Because the data set is fairly large, there is still some proof in the results as we can definitely reject the null hypothesis. However, claims about effect size, confidence intervals and predictions are more tricky. I also have some questions about the used methods in this paper by Wändi Bruine de Bruine so that’s why I ran another test to prove my case.
Instead of trying to find a trend in the data I split the data set in two groups: acts that were in the first half of the show, and acts that were in the second half of the show. The difference is 0.055 standardized points, or in today’s voting system: 53 out of 948 points. If we consider the used data as a representative data set for all Eurovision contests, in the past and in the future, then we should put a confidence interval around this difference. Using bootstrapping methods I was able to calculate a (p=0.95) confidence interval of 32 to 77. So yes, there is definitely a ‘recency effect’!
Please find my code here. For the first time I used Python, Pandas and Numpy in a blog post!
Header image is ‘Fud Leclerc’ who revisited the Eurovision Song Contest 4 times.