fight-rite · exploratory analysis · ufcstats 1994–2026
A walk through building a UFC fight predictor, and the five checks that caught it cheating. Every number here comes from 8,808 real bouts — and the headline finding is that two of the most obvious features in the dataset are leaks, one of them hiding inside the missing values.
Leakage
In this dataset each bout is stored as "Fighter A vs. Fighter B" with an
OUTCOME of W/L or L/W. Modern UFC assigns the red corner —
listed first — to the betting favourite, so seeing red win 58.0% of the time is real signal.
But plot that rate by year and the pre-2010 era isn't a bias. It's a flat 100%.
All 1,250 older bouts carry W/L; not one carries L/W.
Corner order is the label.
Why this matters A leak does not make a model crash or look wrong. It makes it look excellent. Had this gone unnoticed, corner order would have delivered a 64.1% "accuracy" that collapses the moment it meets a fight whose result isn't already encoded in the row. Leaks are caught by plotting your label against time, not by reading code.
Baselines
Scored over 12 walk-forward folds — train on everything before year N,
test on year N, advance, repeat — across 5,655 post-2015 bouts. Note that
red_corner wins on accuracy and loses badly on log-loss, scoring worse than a coin flip.
| Model | Accuracy | Log-loss | Brier | Verdict |
|---|
Why this matters
red_corner is right more often than Elo, yet it is the worse model. It
claims 64.1% confidence — the rate it learned from contaminated history — while modern truth is
58.0%. Being confidently wrong is punished by log-loss and invisible to accuracy. Report a single
accuracy number and this is exactly the failure you will ship.
Calibration
Bucket every prediction by the probability it claimed, then check what actually happened. A calibrated model sits on the diagonal. Elo sits consistently below it — every bucket wins more often than Elo predicted, by 5.8–6.7 points.
| Bucket | Bouts | Predicted | Observed | Gap |
|---|
Why this matters
The cause is visible upstream: mean |elo_diff| is just 31.8 points, so ratings
barely spread apart. UFC careers are short — a median of about four bouts — and Elo never gets
the repetitions it needs to converge. It is a rating system designed for chess players with
hundreds of games. This is a structural mismatch with the sport, not a tuning problem.
Drift
The sport itself changed. Submissions were 65% of finishes in 1994 and are 18% now; decisions went from near-zero to roughly half of all bouts as rounds, judging and weight classes professionalised. A model trained on all of it is learning a sport that no longer exists.
Why this matters This is concept drift, and it has a counter-intuitive consequence. An expanding training window — the obvious default — estimated the red-corner rate at ~62% because it averaged in the old era, against a modern truth of 58.0%. Cutting training data improved the model. When the process generating your data is non-stationary, more history is not more information.
Informative missingness
Reach is missing for nearly half of all fighters, and not at random. Among fighters who had a single UFC bout, it is missing 76.7% of the time; among those who had ten or more, 1.1%. The fighter table is a snapshot taken the day the data was downloaded — so “reach was never recorded” is a fact about the whole career, including the part that had not happened yet when the fight was contested.
This matters because the textbook fix for missing data is a “was missing” indicator column. Strip out every feature and keep only those flags, and the model still scores 58.2% accuracy at 0.6750 log-loss — better than any baseline on this page. The flags on career features earn 57.9% honestly: a debut really is a debut, and it is known in advance. The flags on physical attributes earn 57.4% by knowing the future. The numbers cannot tell the two apart. Only knowing how the column was produced can.
| Strategy | Accuracy | Log-loss | Brier | Verdict |
|---|
Why this matters
The leaky variant wins on accuracy by 1.0 points and is disqualified anyway. What replaces it
is shrinkage: pull each fighter's rates toward the population mean with weight
bouts / (bouts + 3), so a debutant gets the average, a veteran keeps their own
numbers, and nobody needs a flag. Missingness and unreliability stop being separate problems —
which is the same trick as the inflated K for new fighters in Elo, moved from ratings to features.
Where this stands
Most of this page is scaffolding rather than prediction: a leak filter pinned to
2010-01-01, chronological walk-forward splits, three baselines to beat,
calibration on every output, and now a rule that missingness from a snapshot column is not a
feature. Fifty-one tests enforce it — including one that shuffles the fight order and asserts
that Elo refuses to run, and one that recomputes every rolling feature on truncated
data to prove no fight can see its own result.
The bar, and the first thing over it
The bar was 57.0% accuracy and 0.6824 log-loss — clear both or lose to
a one-line rule or a rating system from 1960. Pre-fight rolling box scores and fight history,
imputed by shrinkage, reach 62.4% and 0.6444.
That clears both. It is also a plain logistic regression on differenced features, which is
the least interesting model available and the point: the work was in the features, not the fit.