Model transparency · xG V1
How well does xG measure scoring chances?
We measure probability quality and whether summed xG matches goals. Most attempts do not score, so “no goal” accuracy alone tells very little.
Since V1 was frozen
Completed regular-season games starting after Oct 14, 2026, 06:58 UTC. Historical backfills before this date stay out of this comparison.
2 / 2 eligible completed games fully scored. Reports refresh within five minutes of new scores.
| Probability quality | V1 | Constant baseline |
|---|---|---|
| Log loss ↓ | 0.2573 | 0.2820 |
| Brier score ↓ | 0.0701 | 0.0744 |
| ROC-AUC ↑ | 0.7292 | — |
| Average precision ↑ | 0.2423 | — |
| Calibration error (ECE) ↓ | 0.0353 | — |
Live calibration
Predicted goal chanceActual goal rate
Similar lengths mean better agreement.
0–10%118 attempts
10–20%34 attempts
20–30%8 attempts
30–40%1 attempts
Saved development benchmark
What we saw while building V1
These saved chronological splits helped us develop and compare the model. The test results were reviewed during model choice; treat this as a development benchmark, not an untouched final evaluation.
| Split | Log loss ↓ | Baseline loss ↓ | Brier ↓ | ROC-AUC ↑ | Average precision ↑ |
|---|---|---|---|---|---|
| Training | 0.2076 | 0.2459 | 0.0566 | 0.7946 | 0.2413 |
| Validation | 0.2112 | 0.2446 | 0.0570 | 0.7771 | 0.2212 |
| Test | 0.2183 | 0.2489 | 0.0590 | 0.7653 | 0.2044 |
Development test calibration
Predicted goal chanceActual goal rate
Similar lengths mean better agreement.
0–10%69554 attempts
10–20%19615 attempts
20–30%4490 attempts
30–40%772 attempts
40–50%314 attempts
50–60%139 attempts
60–70%29 attempts
70–80%16 attempts
80–90%2 attempts
Saved goal totals by split
| Split | Attempts | Goals | Summed xG | xG minus goals |
|---|---|---|---|---|
| Training | 155807 | 10443 | 10445.51 | 2.51 |
| Validation | 93002 | 6190 | 6381.94 | 191.94 |
| Test | 94931 | 6471 | 6912.65 | 441.65 |
What do the metrics mean?
Log loss and Brier score: how close the predicted chances were to what happened. Lower is better. Log loss penalizes very confident mistakes more strongly.
Constant baseline: every attempt gets the same 6.70% chance, using only V1's training goal rate.
ROC-AUC and average precision: can the model give goals higher scores than non-goals? Higher is better. They need goals in the sample; ROC-AUC also needs non-goals.
Calibration: among attempts given a 10% chance, roughly 10 in 100 should score. Bin counts show how much evidence each bar has. Small groups are noisy. ECE averages those gaps and can hide problems in rare high-chance shots.
Summed xG: expected goals across included attempts. A positive xG-minus-goals value means the model estimated more goals than occurred. Empty-net attempts, blocked shots and shootouts are excluded throughout.