Wednesday, April 29, 2015

An Analytical Perspective on Referee Bias

Celtic supporters have a reputation for being conspiracy theorists when it comes to referees.

I'll admit, it's hard not to think referees are biased when an opposing defender deliberately uses his hand to stop a ball in the box and two referees looking directly at the incident fail to award a penalty, which contributes to Celtic losing the game and a chance at the treble!

There are countless other incidents and decisions that have gone against Celtic over the years. According to some supporters, these examples constitute a body of evidence in support of the view that referees in Scotland are biased against Celtic.

But anecdotes such as these are hardly objective evidence, and there is always the potential for confirmation bias in the use of such anecdotes.

Fortunately, there is ample publicly available data on Scottish referee decisions at sites like football-data.co.uk. So we don't have to rely just on anecdotes; we can assess the patterns rigorously and quantitatively with analytics.

I will focus in this post on fouls because awarding fouls is the most frequent referee action. As such, fouls give us the best opportunity to detect bias (signal), as opposed to random variation (noise). Bottom line: if we can't find evidence of referee bias in fouls, we are unlikely to find it in cards or penalties, which have much smaller sample sizes.

The metric I will focus on is Fouls Ratio, or

Fouls Conceded/(Fouls Conceded + Fouls Awarded).

Teams that tend to conceded more fouls than their opponents will have higher Fouls Ratios than teams that tend to concede fewer fouls. Note, there is no single expected value for Fouls Ratio, such as 0.50. The value will depend on the strategies, tactics, and discipline level of each team.

Now for the obligatory colorful charts.


Figure 1
Click to Enlarge

Figure 2
Click to Enlarge


Figures 1 and 2 plot the Fouls Ratio for each league match for both Celtic and Rangers going back to the 2000-01 season. As you can see, whether the match is home or away, the pattern looks completely random, no discernible trends or predictable oscillations.

Also notice that the lines overlap substantially between Celtic and Rangers. The mean Fouls Ratios for Celtic, home and away, are 0.44 and 0.47 respectively, compared to 0.45 and 0.49 for Rangers. These relatively minor differences are nonetheless statistically significant in the case of away matches.

Thus, if anything, referees appear to be biased against Rangers, not Celtic. However, I suspect these differences reflect real differences in discipline between the two clubs, rather than referee bias. It's important to note also that both Celtic and Rangers have below average Fouls Ratios compared to the league as a whole (pro Old Firm bias anyone?).

So there doesn't appear to be a detectable bias against Celtic in fouls overall, but what about individual referees?

The next two charts plot Fouls Ratios for Celtic and Rangers by referee while taking into consideration the number of games each referee has worked. The number of games worked is essential information because random variation in Fouls Ratio is expected to decrease as the number of games increases. Indeed, this appears to be the case.


Figure 3
Click to Enlarge



Figure 4
Click to Enlarge


As you can see from the charts above, Fouls Ratios tend to converge to a similar value for both clubs as the number of games a referee has worked increases. This is an example of regression to the mean. No individual referee biases can be detected in these plots for referees that have worked 10 or more games. All of the referees tend to converge on a single value, which is the mean Fouls Ratio for the team, be it Celtic or Rangers.

It's interesting to note that because of typos in the source files, some of the data points represent the same referee (i.e., Hugh Dallas, H Dallas). Because of these data entry errors, we can clearly see how Fouls Ratios regress to the mean as the number of games worked increases.

Take-home message: there is no detectable evidence of bias against Celtic by referees in Scotland, at least in regards to the proportion of fouls conceded in league matches between 2000 and 2014.

While there may be some referees who hold personal biases against Celtic, or other clubs, these biases do not appear to influence their behavior on the pitch in any reliable way.

Moreover, given the randomness of variation in Fouls Ratio over time, it appears that refereeing decisions do even out in the long run.

Friday, April 24, 2015

The Most Underrated Team in Scotland

Scottish Cup finalists Inverness Caledonian Thistle (ICT) play former SPFL Premiership "title challengers" Aberdeen tomorrow in the Highlands.

A few of us have been talking up ICT on Twitter for a while now. I was even ridiculed by one Twitter pundit for daring to suggest that ICT are just as good as Aberdeen, if not better. 

When ICT subsequently drew and then won against Celtic in the Premiership and Cup respectively, I admit I did feel more than a bit vindicated (obviously my ego trumps my club allegiance).

Looking at the numbers behind the Premiership season so far, I have concluded that ICT are the most underrated team in Scotland, and in this post I'll explain why.

The table below presents performance data for both sides during Aberdeen's "title challenge," from the start of the New Year until their 4-0 defeat to Celtic. As usual, I will focus on the two main drivers of goal dominance, Shots on Target Ratio (SoTR) and PDO, as well as their components (click here to see how these variable are defined).


Click to Enlarge


As you can see from the table, with regard to shots on target (SoT), ICT were actually better than Aberdeen overall during this period of the season; The Jags had a higher SoTR driven by a superior defense (lower SoT conceded per game). Interestingly, despite Aberdeen's much vaunted attack, the number of goals scored per SoT (score rate) was virtually identical for both teams.

The only aspect of performance in which Aberdeen was significantly better than ICT was save rate (goals per SoT conceded). The Dons' keeper Scott Brown was as good as Craig Gordon (Celtic) or Alan Mannus (St. Johnstone) during this period. However, unlike Gordon and Mannus, Brown was not able to sustain such a high level of performance, and was subsequently dropped for Jamie Langfield. Ouch.

In summary, during Aberdeen's two-month title challenge, ICT performed equal to or better than the Dons in every important performance metric, except save rate. So why were/are the Jags so underrated by pundits and supporters alike?

Here are three possible reasons that I'm throwing out there as food for thought rather than definitive answers.

  1. The Table-Never-Lies Fallacy: This is the mistaken idea that results are a true measure of a team's quality. Aberdeen are in 2nd place and ICT in 3rd, so the Dons must be better than the Jags, right? Wrong. Football doesn't work that way. There is too much luck involved. If you don't believe me, then I suggest you read The Numbers Game by Anderson and Sally.
  2. The Wee-Club Bias: ICT are a small, provincial club, one of only two Highland clubs in the SPFL Premiership. Aberdeen, in comparison, are a much larger club, part of the so-called New Firm (along with Dundee United). Bigger is better, right? Not necessarily.
  3. Wengeritis: This is a mental disease that afflicts many football supporters. It leads to the dogmatic view that "attractive" football is superior to "winning ugly." ICT have been described as "dire" by some afflicted with Wengeritis. Although, to be fair, they do play a form of attacking football, just not the one preferred by some proponents of the Beautiful Game apparently. Aberdeen, in contrast, have been playing some very attractive, free flowing football. Thus, in the minds of some purists, ICT can never be as good as Aberdeen, because the Dons play the "right" way and ICT play the "wrong" way.
Again, these cheeky explanations for ICT's underrated status are meant only as food for thought. 

Here's another morsel for your mind: Who do you think will do better in Europe next season, ICT or Aberdeen? I know who I'm picking.