Model Calibration · v3.0 Walk-Forward 3192 OOS fights

Expected Calibration Error

0.157

Average gap between predicted and actual win rates. Lower = better. Miscalibrated

Brier Score

0.1895

0.25 = coin flip

Log Loss

0.5613

0.693 = coin flip

Accuracy

76.6%

walk-forward

Buckets

confidence tiers

Reliability by Confidence Bucket Predicted Actual

50-55%

P 52.5%

A 67.7%

+15.2pp

55-60%

P 57.1%

A 89.7%

+32.5pp

60-65%

P 62.2%

A 88.7%

+26.5pp

65-70%

P 66.3%

A 85.7%

+19.4pp

70-75%

P 74.1%

A 88.0%

+13.9pp

75-80%

P 76.3%

A 92.0%

+15.7pp

80-85%

P 84.4%

A 98.2%

+13.8pp

Calibration reliability by confidence bucket
Bucket	Fights	Predicted average	Actual win rate	Delta percentage points
50-55%	2023	52.5%	67.7%	+15.2
55-60%	58	57.1%	89.7%	+32.5
60-65%	53	62.2%	88.7%	+26.5
65-70%	63	66.3%	85.7%	+19.4
70-75%	158	74.1%	88.0%	+13.9
75-80%	614	76.3%	92.0%	+15.7
80-85%	223	84.4%	98.2%	+13.8

Model is under-confident — its picks win more often than advertised. Real edge is larger than displayed confidence.

Predicted vs Actual Diagonal = calibrated

▸ Exact numbers table

Predicted Range	Fights	Avg Predicted	Actual Win Rate	Delta	Status
50-55%	2023	52.5%	67.7%	+15.2%	Under-confident (wins MORE than predicted — extra value)
55-60%	58	57.1%	89.7%	+32.5%	Under-confident (wins MORE than predicted — extra value)
60-65%	53	62.2%	88.7%	+26.5%	Under-confident (wins MORE than predicted — extra value)
65-70%	63	66.3%	85.7%	+19.4%	Under-confident (wins MORE than predicted — extra value)
70-75%	158	74.1%	88.0%	+13.9%	Under-confident (wins MORE than predicted — extra value)
75-80%	614	76.3%	92.0%	+15.7%	Under-confident (wins MORE than predicted — extra value)
80-85%	223	84.4%	98.2%	+13.8%	Under-confident (wins MORE than predicted — extra value)

Walk-forward = compact per-fight validation rows (3192 fights). True out-of-sample, zero lookahead.

Method-call reliability

Actual Method	Sample	Method Calls Correct
DEC	1589	47.8%
KO/TKO	1032	59.9%
SUB	571	24.2%

When a fight actually ends by submission, our full read (winner + method) matched it only ~24% of the time; by KO ~60%, by decision ~48% (n=3,192). Submissions are our hardest outcome to project - treat the finish-method line as directional. Method rows are descriptive reliability checks and do not alter the win-probability model.

Empirical Hit Rates by Public Zone 145 scored

Zone	Scored	Record	Expected	Hit Rate	95% Wilson CI
LOCK	33	23/33	79.0%	69.7%	52.7%-82.6%
STRONG	3	2/3	65.9%	66.7%	20.8%-93.9%
SOLID	11	9/11	61.9%	81.8%	52.3%-94.9%
LEAN	8	4/8	58.7%	50.0%	21.5%-78.5%
COIN FLIP	90	57/90	52.5%	63.3%	53.0%-72.6%

These rows are read-only audit data from resolved prediction-log entries. Empty zones stay visible so missing sample areas are obvious.

How to Read These Metrics

Brier Score — measures probability calibration. 0.25 = coin flip, lower = better. Excellent

Log Loss — information-theoretic quality. 0.693 = coin flip, lower = better. Excellent

ECE (Expected Calibration Error) — avg gap between predicted and actual rates. Lower = better calibrated. Miscalibrated

Live Tracking Data (145 predictions scored — click to collapse)

145

Scored

66%

Accuracy

0.236

Brier

Calibration by Confidence Bucket

Predicted Range	Count	Avg Predicted	Actual Win Rate	Delta	Status
50%-55%	85	52.3%	62.4%	+10.1%	Under-confident
55%-60%	13	57.7%	61.5%	+3.8%	Well calibrated
60%-65%	11	61.9%	81.8%	+19.9%	Under-confident
65%-70%	3	65.9%	66.7%	+0.7%	Well calibrated
70%-75%	3	72.1%	66.7%	-5.4%	Over-confident
75%-80%	15	75.6%	73.3%	-2.3%	Well calibrated
80%-85%	6	81.8%	50.0%	-31.8%	Over-confident
85%-90%	9	85.0%	77.8%	-7.2%	Over-confident

Rolling Accuracy (10-fight window)

Fight 138

70%

Fight 139

70%

Fight 140

70%

Fight 141

80%

Fight 142

80%

Fight 143

90%

Fight 144

90%

Fight 145

80%

Calibration Metrics