v2 Phase 3: rigorous backtest + Methodology page
- scripts/buildBacktest.ts: honest walk-forward validation — params fit on pre-2018 internationals, tested out-of-sample on 7,988 matches (2018-2026) predicting each game from prior data only. Proper scoring (Brier/log-loss/RPS/ accuracy) vs uniform / base-rate / Elo-only baselines + a calibration analysis (reliability bins + ECE). Results: 60% accuracy, RPS 0.171 (beats all baselines), ECE 0.01 (excellent calibration). - Methodology page (/methodology): plain-language model walkthrough, the backtest scorecard, a custom-SVG reliability diagram, and honest limits. Transparency is the differentiator — no market odds, no overclaiming. - ReliabilityDiagram component; 'Model' nav entry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -160,6 +160,22 @@ export interface TeamProfile {
|
||||
qualifyOdds: number | null;
|
||||
}
|
||||
|
||||
// ---- model backtest (Methodology page) ----
|
||||
|
||||
export interface BacktestScores { brier: number; logloss: number; rps: number; accuracy: number }
|
||||
export interface BacktestReport {
|
||||
generatedAt: string;
|
||||
trainEnd: string;
|
||||
testTo: string;
|
||||
tested: number;
|
||||
params: { goalsPerElo: number; avgGoals: number; rho: number; homeAdvElo: number };
|
||||
model: BacktestScores;
|
||||
baselines: { uniform: BacktestScores; baseRate: BacktestScores; eloOnly: BacktestScores };
|
||||
reliability: { predicted: number; observed: number; count: number }[];
|
||||
ece: number;
|
||||
worldCup: { tested: number; accuracy: number };
|
||||
}
|
||||
|
||||
// ---- data-story viz (StatsBomb open data, a fixed historical match) ----
|
||||
|
||||
export interface VizShot {
|
||||
|
||||
Reference in New Issue
Block a user