cup26/docs/V2-PLAN.md

# Cup26 "v2" — the data leap (internally v7)

Drafted June 12, 2026 (matchday 2) from a six-source reconnaissance, probed
from both the residential IP and the production VPS. The visual overhaul is
handled separately in Claude Design; this plan reserves explicit handoff
slots for it and keeps all new features design-ready (data + logic first,
plain markup that a design pass can reskin).

## Reconnaissance results (verified June 12)

| Source | Local | VPS | Verdict |
|---|---|---|---|
| **FotMob** `/api/data/*` | ✅ 200 | ✅ 200 — **no signed header anymore** | **INGEST NOW** |
| **FIFA official** `api.fifa.com/api/v3` | ✅ | ✅ — no auth, no block | **INGEST NOW** |
| **ESPN deep fields** (already cached!) + propBets | ✅ | ✅ | **INGEST NOW** |
| **Open-Meteo** weather+elevation | ✅ | ✅ — keyless | **INGEST NOW** |
| SofaScore | browser-only (403 to scripts) | ❌ hard IP block | defer (FotMob supersedes) |
| FBref | ❌ Cloudflare wall | ❌ | skip (no 2026 xG yet anyway) |

What the three rich sources carry (all verified against real June 2026 matches):

- **FotMob `matchDetails`** (~244KB/match): per-shot xG + xGOT with pitch
  x/y, team xG split (open play/set play), **player ratings** (e.g. 7.5) for
  all 52 players, player-of-the-match, momentum curve, attacking zones, match
  facts/insights, lineups with formations. Works for live matches too.
  Key ids: league 77; matches by `?date=YYYYMMDD` (e.g. 4667751 = MEX–RSA).
- **FIFA v3**: IdCompetition=17, IdSeason=285023. Official **confirmed
  lineups with stable player IDs + headshot URLs**, typed event timeline with
  pitch X/Y, official attendance (80,824 at the opener), team tactics,
  officials, standings with QualificationStatus, live match clock.
- **ESPN, unused but already stored**: per-player match stat lines (14 stats:
  shots, on-target, saves, fouls, cards, goals conceded…), formation +
  formationPlace (pitch positions), team leaders, ~109-entry commentary
  ticker, broadcasters, match recap articles. Plus one cheap new fetch:
  **propBets — DraftKings any/first-goalscorer odds, ~962 props per match**.
- **Open-Meteo**: one batched call covers all 16 venues; returns elevation
  (Azteca 2,245m!), hourly temp/precip/wind in venue-local time; archive API
  for played matches.

## Phase D — capture first (urgent: live data is unrecoverable)

FotMob is unofficial — it could re-lock any day. Capture beats features.

- `server/src/ingest/fotmob.ts`: daily `matches?date=` sweep → id map
  (fixture ↔ matchId via team-pair+date, the ESPN pattern); `matchDetails`
  per finished match (once, re-fetch at FT+2h for settled ratings) and per
  live match every ~90s. Store raw-ish normalized JSON in new `match_fotmob`
  table (whole payload is fine — ~25MB for all 104 matches). Respectful
  cadence: ≥2s between calls, circuit breaker, health-tracked like ESPN.
- `server/src/ingest/fifa.ts`: calendar sweep (id map), `live/football/...`
  per live match (clock/lineups), `timelines` at FT. Tables: `match_fifa`
  (lineups/officials/attendance/tactics), reuse health.
- `server/src/ingest/espnExtra`: widen `espnNormalize` to keep player stat
  lines, formation+formationPlace, leaders, commentary, attendance when
  present (normalizer v3 + the existing boot backfill re-stores everything).
  New `scorer_props` table fed from propBets on the odds cadence (3h) —
  line history is unrecoverable, same urgency as 1X2 odds was in v3.
- `server/src/ingest/weather.ts`: forecast at enrich cadence for fixtures
  <48h away; archive once at FT. `match_weather` table. Venue lat/lon/alt
  as a committed static (`scripts/buildVenues.ts` → `public/data/venues.json`).
- Storage: everything also lands in the nightly storage-box snapshot.

## Phase M — model upgrades (CV-gated, same discipline as v4/v5)

Every experiment runs through `scripts/cvSearch.ts` (5-fold rolling CV,
pre-registered ship bar ≥0.0005 mean-fold RPS) — ship only what validates.

- **M1 xG-informed Team-DC**: backfill FotMob `matchDetails` for recent
  *international* matches (last ~4 years; local script into the lake, weeks
  of polite fetching — start immediately, run in background). Experiment:
  fit attack/defence on a goals/xG blend (xG is less noisy than goals).
  This is the single largest model upside available.
- **M2 lineup-aware kickoff refresh**: when FIFA confirmed lineups land
  (~75min before kickoff), adjust the match forecast for missing key players
  (availability share from minutes×ratings, NOT market values). Snapshot for
  the scoreboard stays frozen at T-15min as today — honesty unchanged.
- **M3 in-play v2**: blend the live model with score-state rates from the
  StatsBomb lake, stratified by pre-match strength gap (fixes the v3
  selection-bias concern), validated on lake matches; FotMob momentum as a
  display layer, not a model input.
- **M4 scorer model (new headline)**: per-player goal-rate model (minutes ×
  team λ share from historical scorer data + tournament form) → predicted
  Golden Boot table, benchmarked against DraftKings scorer props on a
  **Scorers: model vs market** scoreboard — the existing honest-benchmark
  franchise, extended to players.

## Phase F — features (each lands design-ready; final skin = design handoff)

Tier 1 (data already flowing after D):
1. **Match center, full**: xG race chart (reuse the Story page viz), shotmap
   (per-shot xG, x/y), attack-momentum strip, lineup pitch with formations +
   headshots (FIFA), per-player stat lines + ratings table, POTM card,
   officials + attendance + broadcasters, weather/altitude badge,
   commentary ticker. This page becomes the product's core.
2. **Player pages + tournament leaderboards**: G+A (assists verified
   structured in ESPN participants), ratings-based Best XI, saves/cards
   boards from ESPN stat lines; FIFA headshots; links everywhere.
3. **Luck-adjusted table**: group standings re-computed on xG ("the table
   if chances decided games") next to the real one — very shareable.
4. **Golden Boot: model vs market** (M4 + scorer_props).
5. **Weather strip** on Live/match pages; altitude notes (Azteca 2,245m).

Tier 2 (after M experiments settle):
6. xG-aware form chips and team pages (xG trend last 5).
7. In-play v2 probability + momentum fusion on live pages.
8. Lineup-drop push notification ("XI is in — model moved to 54/24/22").

## Design handoff slots (Claude Design → me for implementation)

- Match center layout (shotmap + xG race + momentum + lineup pitch + ticker)
- Player page / player card / Best XI grid
- Leaderboards (boot race, G+A, ratings) and the luck-adjusted table
- Weather/altitude badges and the match-info panel
- Any reskin of existing pages — all current components are plain Tailwind
  with tokenized colors (`--app-*`), so a token+component redesign drops in.

## Order & verification

D (capture-first, deploy same day) → M1 backfill kicked off in background →
F1–F5 in batches → M experiments as data accrues → Tier 2. Per batch: tests +
exit-code build gate + preview matrix (EN/DE × mobile/desktop × themes) +
prod verification against the live tournament. Honest framing everywhere:
xG and ratings are third-party estimates; scorer odds remain benchmark-only.

## Risks

- FotMob is unofficial: could re-add signed headers or block datacenter IPs
  any day (SofaScore precedent). Mitigations: capture-first, raw payload
  storage, circuit breaker + health, app never depends on it being live.
- FIFA API is unauthenticated but official-internal; same mitigations.
- Identity mapping (fixture ↔ fotmobId ↔ fifaId ↔ espnId) is the foundation —
  one `source_map` per provider, team-pair+date matching, tested.
- Player identity across sources (names vs FIFA IdPlayer): normalize via
  name+team with a manual alias map fallback; keep per-source ids.