Exploring football predictions using Bayesian modelling

I learnt about Bayes' theorem during my studies, but didn't give it much of a second thought - I didn't then understand just how powerful the simple concept could be.

Traditional ML algorithms can be fit as black-boxes - plug in your inputs/outputs and almost anyone can produce a half-decent model (of course it's never as simple as this!). Bayesian techniques, by contrast, require explicit definitions, and often a lot more thought into the data and distribution from which it came.

You have to think about latent variables - what's really driving the outcomes under the hood? And how your assumptions about these variables change the way you model outcomes, and the effect of the observed data.

The payoff for defining assumptions before ever touching a dataset is a posterior distribution, from which statistical metrics about your model can be derived in a way impossible using traditional ML. Inference such as: what's a 95% confidence interval for my outcome? Or what's the probability of observing extreme outcomes?

This is the motivation for using Bayesian approaches for predicting football outcomes - a task known for being predictably unpredictable. It's clear why, under these high-variance conditions, additional inference becomes extremely valuable: rather than a model spitting out an 'Arsenal win', I want to understand exactly why, and how confident such a prediction is. Perhaps they have been very strong attacking-wise, or their opponent has been weak defensively recently - or maybe it is just a coin toss where the model thinks Arsenal will edge it.

With all that out the way, let's delve a bit deeper into the model.

Latent Variables, random processes and priors

In order to build a model, we need to have a basic understanding of how football works - or at least a simplified theory which we can use!

Teams play matches, during which they have the possibility of scoring and (/or) conceding. They can play matches at their home stadium or away at their opponents stadium.

So one way to model this is by introducing: an attacking strength for each team to help assess the likelihood of scoring, a defensive strength to assess likelihood of conceding, and a home advantage, which tells the model how much better a team plays when at their own stadium. Although the latter seems trivial, it is known to play a factor - and may not be equal across all teams. Imagine a team with a massive stadium and roaring fans vs a team with a half empty stadium.

So the idea, therefore, is to model each match's goals by using the attack and defence scores of the teams playing. Specifically, for a given match:

Home team's expected goals:

$$\theta_{home} = attack_H - defence_A + homeadv_H$$

Away team's expected goals:

$$\theta_{away} = attack_A - defence_H$$

where $H$ represents the home team and $A$ represents the away team.

Of course a model is a simplification of reality, and there are many more factors which you can include! There are reasons both for, and against including these additional variables.

The attack and defence variables are distributed as per-team AR(1) processes - meaning they follow the formula:

$$attack_t = \rho \cdot attack_{t-1} + \sigma \cdot z_t$$

where $z_t \sim N(0, 1)$ is the raw noise, and $\rho$, $\sigma$ are learned hyperparameters ($\rho$ has a Beta prior, $\sigma$ is HalfNormal).

Essentially, all this is saying is the attack variable depends on the attack variable at the last timestep plus some noise.

This is replicated for defence symmetrically.

From our attack and defence, we can then compute a $\theta$ variable, which represents the linear combination of our variables as:

$$\theta_{home} = attack_{t,team} - defence_{t,opp} + homeadv_{team} \cdot isHome$$ $$\theta_{away} = attack_{t,opp} - defence_{t,team}$$

(One for the home team and another for the away team for a given match.)

Note 1: We also define $homeadv$ as a half-normal, fixed-per-team variable.

Note 2: This assumes home goals scored is independent from away goals scored, which is not true. We can add an extra variable to account for this (Dixon-Coles); for simplicity I've left it out here.

Once we have all our priors, and attack and defence processes set up, we can model goals as Poisson distributions as follows:

$$\lambda_{home} = exp(\theta_{home}), \quad \lambda_{away} = exp(\theta_{away})$$ $$goals_{home} \sim Poisson(\lambda_{home})$$ $$goals_{away} \sim Poisson(\lambda_{away})$$

Note: The use of exp() here allows the linear combination $\theta$ to be mapped into a valid rate parameter for the Poisson distribution; more information about this can be found online.

The idea here is that once we observe data, we can combine that with our likelihood function (from the Poisson distribution) to produce thousands of posterior draws. So we now have a new, data-informed distribution we can use for statistical inference.