GET /api/match-rich/:num projects the captured provider data for the
match center: every shot with pitch coords and xG oriented to
home/away (FotMob payloads now carry team ids, v2 + settle-versioned
refetch), team xG sums, the momentum curve, player of the match with
rating, FIFA's official attendance/stadium/referee, and kickoff-hour
weather. Verified on real data: South Korea 2.31-0.83 Czechia on 22
oriented shots, POTM Hwang 8.9, referee Amin Mohamed Omar.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first heal attempt missed both matches: a 02:00 UTC kickoff files
under the PREVIOUS day in ESPN's ?dates= param (ET-based), and the
backfill skipped today's UTC date entirely. Now any started-but-
forgotten fixture triggers fetches for its UTC date and the day
before, covering the ET offset in both directions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The persistence table shipped empty just as ESPN's default scoreboard
window rolled past matchday 1 — prod forgot both results. Rather than
patching the DB by hand, boot now re-fetches the date-scoped
scoreboard for any past day whose fixtures still look scheduled and
merges + persists the results. Heals fresh databases and any future
data loss the same way.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four new feeds, all verified against real tournament matches:
- FotMob (unofficial, works from the VPS again): per-shot xG/xGOT with
pitch coords, team xG, player ratings, POTM, momentum, attacking
zones — trimmed payloads (~87KB/match) on a gentle 90s-live/30m-idle
cadence. Capture-first: this source could lock again any day.
- FIFA official v3: confirmed lineups with player IDs, typed XY event
timelines (80 events for the opener), attendance + named referees,
tactics — id-mapped via the calendar, hot while live.
- Open-Meteo: kickoff-hour weather per venue from a build-time geocoded
venues.json (with elevation — Azteca 2,240m); fetched in the enrich
loop inside the 48h horizon.
- DraftKings scorer props (first/last/anytime goalscorer per athlete)
via ESPN's core propBets, insert-if-changed line history, athlete
names resolved through a 30-day cache.
Also fixes a real resilience gap found along the way: fixture results
now persist in SQLite and restore at boot — previously a restart after
ESPN's scoreboard window moved on would silently forget a match day
(standings, model and scoreboard would all regress).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A restart during a live match sat behind the multi-minute odds sweep
before the first scoreboard poll. The live loop now starts immediately
after fixture mapping; odds/backfill/enrich follow on their own
rate-limit keys.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The boot refit fires before live state is restored, so restarts lost
this tournament's matches from the fit until the nightly timer. A
one-shot follow-up at +5 minutes closes the gap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fully progressive: without VAPID keys in the environment the API says
404, the bell never renders, and the service worker addition (a push
handler via workbox importScripts — no SW strategy change) is inert.
With keys: a bell on each team page requests permission, subscribes
(one team per device) and the server diffs fixture states on every
broadcast to push Kickoff / Goal / Full time to that team's followers.
Dead endpoints (404/410) self-clean. Round-trip verified locally with
dev VAPID keys; disabled mode verified too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ESPN's keyEvents carry participants[].athlete for goals — no text
parsing needed. The normalizer now extracts scorer/scoring/shootout
(payload versioned, with a boot backfill that re-stores summaries
saved by the old normalizer), /api/goldenboot aggregates per-player
goals (own goals and shootout kicks excluded), and the Teams page
leads with the top-10 race. Verified against tonight's real goals.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Crawler-visible <title> + og:title/description are injected into the
SPA shell server-side: match links carry the score or the model's
probability split, team links the current title odds — honest framing
included. Plain shell for every other route.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
While a match runs, every poll records the in-play model probability
per game state (insert-if-new on minute+score) into inplay_history.
The match page draws it as a stacked area chart — home fills from the
bottom, away from the top, the gap is the draw — with minute ticks and
a 50% guide. Shows live (under the win-probability bar) and after full
time as its own card. EN/DE strings included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first prod snapshot landed as a 0-byte file with a stale journal:
SQLite needs byte-range locking on the VACUUM INTO destination, which
the CIFS storage box refuses. Now: vacuum into a local temp file, then
a plain sequential copy to the archive (always CIFS-safe), cleaning up
the temp and any leftover journal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 'external drive' turned out to be a CIFS-mounted Hetzner storage
box — SQLite cannot run on a network mount (WAL shared memory and
locking break), and the live DB is only ~7MB anyway. So: the hot
database stays on the local volume, and a nightly VACUUM INTO snapshot
(transactionally consistent, a plain sequential file write — perfect
for CIFS) lands in /data/archive, which compose binds to the storage
box via HOST_ARCHIVE_DIR. 14-day retention, plus a snapshot at boot.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HOST_DATA_DIR in .env can now point /data at a host path (the 1TB
external drive) instead of the named volume, so odds history, caches
and snapshots stop eating the 128GB root disk. A daily prune drops
cache entries stale for over a week and ingest-log rows older than 14
days — faster polling no longer means unbounded DB growth.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Validated on the strict 2018-2022 window and confirmed on the untouched
2022-2026 test set (RPS 0.1703 vs 0.1721 over 4,448 matches):
- the Elo member now blends 30% of a 3x-faster Elo walk, so recent
results move ratings much harder
- ensemble weight shifts from 75/25 toward Elo to 45/55 toward the
time-decayed Team-DC member — the recent-form model now leads
- Team-DC refits nightly (and at boot) on the 15-year window plus every
finished 2026 match, via a new committed-at-build dcTrain.json
(server-only, excluded from the PWA precache)
- a recent-form goal multiplier was also tested and did NOT validate;
it ships disabled, and backtest.json records the whole decision
buildBacktest.ts grew the experiment harness: wider xi/k grids, the
fast-Elo and form variants, a pre-registered ship bar (>=0.0005
validation RPS), and a 7-day refit-cadence check. Older ratings.json
files still work — all new model fields are optional with golden
regression coverage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The live staleness bug: liveTick refetched ESPN summaries every 20s but
the cache TTL was 5 minutes, so in-play stats/timeline barely moved.
Summaries now take a live flag (15s TTL in play), the scoreboard TTL
drops 20s->5s, and polling gets three tiers (8s in-play / 20s near
kickoff / 10m idle) via a pure, tested cadence module. The
football-data fallback is throttled to one attempt per 15s so the 8s
cadence can never breach its 10 req/min cap. Enrichment runs every 5m
(was 20m); client refetch intervals tighten to 10s (live match page)
and 30s (scoreboard).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- scripts/processStatsbomb.ts: one pass over the 17GB StatsBomb lake (4,235
matches) -> lakestats.json (12.82GB events / 14.9M events / 6,919 players),
scorestate.json (goal-rate multipliers by game state — shipped as an insight,
NOT wired into in-play: the raw effect is selection-biased toward stronger
teams and would distort a calibrated model), fingerprints.json (40/48 nations'
shots/xG/set-piece share from real event data). Artifacts committed; raw lake
stays local (gitignored).
- /data page + /api/data/stats: the gigabytes made visible — lake counters,
historical archive, live-collection counts (odds lines, frozen forecasts,
enrichment), per-source health with freshness. Footer links.
- Term component: plain-language glossary tooltips (RPS, Brier, ECE, Elo, xG,
de-vig, ensemble) attached where metrics appear.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- prediction_snapshots: model + de-vigged market frozen ≤15min pre-kickoff
(or at first sight of a live match, flagged 'late' and excluded from totals).
- src/lib/model/scoring.ts: ONE shared RPS/Brier/log-loss implementation for
the backtest and the live scoreboard.
- /api/scoreboard + /scoreboard page: per-match model-vs-bookmaker bars, RPS
for both after FT, 'closer' badges, running head-to-head with honest rules
printed (small-sample caveat included). Nav: 'vs Market'.
- Methodology page now shows the three-way split + ensemble bake-off variants.
- Verified end-to-end with a simulated match: snapshot froze model 80.7% and
DraftKings 67.0% (de-vig correct), both scored at FT, late-flag honored.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- src/lib/model/teamDc.ts: per-team attack/defence Dixon-Coles via weighted MLE
(Maher iterations), exponential time-decay, shrinkage for thin histories,
learned home multiplier; log-pool of score matrices. 7 vitest cases.
- buildBacktest.ts rewritten as a strict three-way-split bake-off: params
pre-2018, tune (xi, k, w) on 2018-2022 validation, final scores on untouched
2022-2026 test (4,448 matches). Result: ensemble RPS 0.1721 beats Elo-DC
0.1726 and Team-DC 0.1801; ECE stays 0.01. Tuned: xi=0.5 k=5 w=0.75.
- Runtime now IS the backtested model: ratings.json carries teamDc+ensembleW;
matchMatrix() pools both members for predictions, Monte Carlo and advance
probs; ModelEngine re-rating preserves ensemble params; lambdas reported as
matrix expectations (drives in-play).
- scripts/buildSquadValues.ts: Transfermarkt FIWC participants page -> all 48
squad market values (France 1.52bn ... Qatar 20m). Display/availability layer
only — June-2026 values are NOT in the backtested model (would leak).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- docs/V3-GIGA-PLAN.md: the v3 plan (data lake to 7+GB, GIGA ensemble, scoreboard,
UI clarity) with the honest reframe: perfect accuracy = perfect calibration;
'beat the betting sites' = a live public head-to-head, scored with proper rules.
- src/lib/odds.ts: American-ml → decimal, de-vig (normalize away the margin),
overround; 3 vitest cases incl. tonight's real opening lines.
- ESPN core odds adapter (server/src/ingest/espnOdds.ts): captures DraftKings
1X2 moneylines + O/U + spread per fixture; follows $ref pointer items.
- odds_history table (insert-if-changed → clean line-movement history);
scheduler odds sweep every 3h over a 14-day horizon, first sweep at boot;
/api/odds + /api/odds/:num. Odds are a benchmark ONLY — never a model input.
- Boot speed-up: fixture↔event mapping skips dates already fully mapped.
- Verified: 54 fixtures captured with real moneylines locally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- src/lib/model/inplay.ts: in-play win probability — remaining goals modelled as
time-scaled Poisson processes convolved with the current score; recomputed each
tick. 4 vitest cases (locks at FT, lead grows decisive over time, sums to 1).
- ESPN summary normalizer extended with the live event timeline (keyEvents);
preview now carries liveStats + events. Scheduler refreshes LIVE matches'
summaries every live tick (~20s) for fresh stats/timeline.
- /match/:num is now a live match center: when live it shows in-play win
probability (driven by the WS score/clock), match stats bars and an event
timeline, polling while live; preview content for scheduled games, recap when
finished. Verified with a simulated live state (94% live vs 81% pre-match).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Persistence via Node's built-in node:sqlite (zero native deps) on a Docker
volume: response cache, source health, fixture↔provider id map, per-fixture
enrichment, ingest log. Runtime bumped to node:24-slim + --experimental-sqlite.
- Resilient fetcher: DB cache + per-source rate-limit + jittered backoff +
circuit-breaker (blocked sources trip + skip; UI reads DB, never breaks).
- ESPN hidden API as the PRIMARY rich source (works from the VPS where SofaScore
403s): scoreboard (live scores w/ clock) + summary (venue, H2H, recent form,
lineups, team stats). football-data / SofaScore are fallbacks.
- Scheduler: maps all 72 group fixtures to ESPN event ids, polls live on a
dynamic cadence, enriches imminent fixtures into match_ext. /api/sources/health.
- Verified: DB populates (real H2H + form for opening matches), 22 tests pass,
Docker image runs node:sqlite on the volume and persists across restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
NilsBriggen