# Cup26 v3 — "GIGA": the data lake, the ensemble, the scoreboard

## The three asks, translated honestly

| Ask | Reality | What we build |
|---|---|---|
| **Perfect accuracy** | Impossible — match outcomes are irreducibly random. A true 60% favourite loses 40% of the time. | Perfect **calibration** (ECE → ~0.01, proven on the reliability diagram) + maximal sharpness from an ensemble. "Perfect" = our probabilities mean exactly what they say. |
| **More accurate than betting sites** | The closing line is the strongest public forecaster; nobody beats it systematically. | A **live public Model-vs-Market scoreboard**: capture DraftKings odds (free via ESPN core API) for every WC match, de-vig them, score market vs our model with RPS/Brier after each result. Target: within noise of the market; stretch: beat it on slices (draws, knockouts). Nobody else shows this publicly — win or lose, we show the gap. The model itself stays **pure** (odds are a benchmark, never an input — per the earlier decision). |
| **Literal gigabytes of data** | Verified: StatsBomb open data alone is **7.06 GB** of event JSON. | A real data lake: 7 GB StatsBomb corpus + ESPN historical sweep + odds-history capture + Transfermarkt squads/values/injuries. Raw lake lives on the dev machine; compact aggregates (tens of MB) ship to prod; a `/data` page shows live counters (bytes, events, matches, players, freshness) — "show all the data." |

## Pillar A — The Data Lake (Phase A)

1. **StatsBomb full corpus** (`github.com/statsbomb/open-data`, ~7 GB): clone locally, process EVERY competition (WC 2018/2022, Euros, La Liga, EPL, WSL, Copa, …) into:
   - `player_profiles` — per-90 xG, xA, shots, key passes, pressures, duels, set-piece involvement for every player in the corpus; matched to 2026 squads by name.
   - `team_fingerprints` — directness, press height, set-piece share, xG-by-zone (the "gaps & strengths" engine, now from real event data).
   - `score_state_rates` — empirical goal rates by score state/minute (powers in-play v2).
   - `shootout/penalty priors` — conversion rates (with martj42 `shootouts.csv`).
2. **ESPN historical sweep**: scoreboards + summaries for WC 2010–2022 (verified working) → fills H2H/form archive with venues, stats.
3. **ESPN odds capture** (`sports.core.api.espn.com/.../odds`, verified live): VPS scheduler snapshots odds for every 2026 fixture a few times daily + near kickoff → `odds_history` (line movement!). Benchmark only.
4. **Transfermarkt adapter** (planned in v2, now built): squads, market values, injuries/suspensions → `squad`, `injuries` tables (daily cadence).
5. **`/api/data/stats` + `/data` page**: total bytes processed, events, matches, players, per-source freshness/health. The gigabytes, visible.

## Pillar B — Model v3: the GIGA ensemble (Phase B)

Three independent ratings, ensembled:
1. **Elo–Dixon-Coles v2** (current model + covariates): tuned time-decay, rest days, venue geo/altitude, stage effects.
2. **Squad-value model**: Transfermarkt market values + age curves + injury deductions → goal expectations (literature: rivals Elo; low correlation with it → great ensemble partner).
3. **Player-aggregate model**: squad xG production/defence from StatsBomb player profiles (coverage-weighted; honest about sparse squads).

Then: **log-opinion-pool ensemble** (weights tuned on 2018–2022 via the existing backtest harness) → **isotonic calibration** on held-out years → only ship what wins the bake-off. Plus: shootout model in the Monte Carlo (knockouts), in-play v2 (score-state intensity, red cards).

Measurement: extended walk-forward backtest scores every variant (RPS/Brier/log-loss/ECE); the Methodology page shows the bake-off table. Live: the scoreboard (below) is the ultimate test.

## Pillar C — UI: clear, clean, labelled (Phase C)

1. **Glossary + tooltips**: every metric (xG, Elo, RPS, ECE…) gets an inline "?" → plain-language explainer; one shared `<Term>` component.
2. **Provenance chips everywhere**: "ESPN · 2 min ago", "150-year archive", "StatsBomb corpus", "model v3".
3. **`/scoreboard`** — Model vs Market: per-match cards (our probs vs de-vigged DraftKings), running RPS/Brier totals, "who was closer" badges. The headline feature.
4. **`/data`** — the lake, visible: counters, coverage, freshness, browsable source tables.
5. **Squad views** on team pages: players with club, age, market value, injury status, corpus stats where available.
6. **Clarity pass**: consistent units/labels, collapsible advanced sections, density/mobile polish.

## Phasing

- **Phase A — Lake** (StatsBomb ingest + Transfermarkt + ESPN history/odds capture + /data) ← start here; odds capture must begin ASAP (every day of line history is unrecoverable).
- **Phase B — Ensemble** (covariates, squad-value + player models, ensemble + calibration, bake-off, shootout, in-play v2).
- **Phase C — UI** (scoreboard, data explorer, glossary/tooltips, squads, clarity pass).
- **Phase D — Deploy & live validation** through the group stage; scoreboard accumulates.

## Honest success criteria

- Calibration: ECE ≤ 0.015 out-of-sample (already 0.01; keep it while adding sharpness).
- Sharpness: beat v2's RPS 0.171 out-of-sample (every point matters; the market sits only a little better).
- Market: publish the head-to-head; target = statistical tie; stretch = ahead on any meaningful slice after ≥48 matches.
- Data: > 7 GB processed, with the proof on `/data`.