Isotonic calibration layer — validated leave-one-fold-out, shipped

The CV harness gained a calibration experiment: per-outcome isotonic
maps (pool-adjacent-violators) fit walk-forward on the five folds and
judged leave-one-fold-out against a pre-registered bar. It cleared it:
LOFO ECE improves 16% (0.0146 → 0.0122) with RPS slightly better too
(0.1724 → 0.1722). The final maps (fit on all folds) are committed as
a research artifact, embedded into ratings.json by buildRatings when
the shipped config matches, and applied in matchMatrix by scaling the
score-matrix outcome regions — scorelines, Monte Carlo and displayed
probabilities all stay mutually consistent. Without the field, output
is bit-identical (golden tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-12 01:08:20 +02:00
parent 005b4d72ca
commit 88afbfad74
6 changed files with 1714 additions and 5 deletions
+12 -2
View File
@@ -1,5 +1,5 @@
{
"generatedAt": "2026-06-11T22:41:08.860Z",
"generatedAt": "2026-06-11T23:06:22.047Z",
"protocol": {
"folds": [
{
@@ -56,5 +56,15 @@
"testRps": 0.1647
},
"shipChange": false,
"runtimeSec": 72
"isotonic": {
"bar": "LOFO RPS worsens ≤0.0002 AND LOFO ECE improves ≥10%",
"lofo": {
"rawRps": 0.1724,
"calRps": 0.1722,
"rawEce": 0.0146,
"calEce": 0.0122
},
"ship": true
},
"runtimeSec": 73
}