The Math Behind the Market
Model error and calibration
8 min · A model is judged by whether its percentages come true at the rate it claims.
- Model error and calibration
- Confidence and uncertainty
The stat
A model is judged by whether its percentages come true at the rate it claims.
Learn the number
Every model is wrong in some way. The useful question is not "was it right about this game?" but "are its numbers honest over many games?"
Calibration is that test. Take everything a model called 70%, and check how often those things actually happened. Well calibrated means about 70% of them did.
Being poorly calibrated has a direction. Claiming 70% and hitting 50% is overconfidence. Claiming 70% and hitting 90% is underconfidence.
One result can never measure calibration. A 70% call that misses is completely expected — it should miss about three times in ten.
This is exactly what POWERRANK's own calibration tool measures on your journal picks, and what the Beat the Model module measures on your estimates.
Read the stat
A learner makes 40 predictions at roughly 70% confidence and gets 27 right.
- Claimed rate: 70%.
- Actual rate: 27 ÷ 40 = 0.675, which is 67.5%.
- Gap: 67.5% − 70% = −2.5 percentage points.
- A gap that small, over 40 predictions, is close agreement.
Well calibrated. Their 70% means roughly what it says — and the 13 misses are not errors, they are the 30% showing up.
Your turn
Another learner makes 40 predictions at 90% confidence and gets 26 right. What is their actual rate, and what does it say?
Explain what it means
Calibration separates being right from being honest about how sure you were. A modest, well-calibrated forecaster is more useful than a confident one who is wrong about their own confidence.
Make a prediction
Before your next ten predictions, write the confidence for each. Predict how many you will get right at each level.
Review what happened
Compare claimed against actual per confidence level, not overall. Overconfidence usually hides in the highest bucket.
Check your understanding
0 of 4 correct
Every question has to be right for this lesson to count as complete. Miss one and you can try it again.