Isotonic calibration layer — validated leave-one-fold-out, shipped
The CV harness gained a calibration experiment: per-outcome isotonic maps (pool-adjacent-violators) fit walk-forward on the five folds and judged leave-one-fold-out against a pre-registered bar. It cleared it: LOFO ECE improves 16% (0.0146 → 0.0122) with RPS slightly better too (0.1724 → 0.1722). The final maps (fit on all folds) are committed as a research artifact, embedded into ratings.json by buildRatings when the shipped config matches, and applied in matchMatrix by scaling the score-matrix outcome regions — scorelines, Monte Carlo and displayed probabilities all stay mutually consistent. Without the field, output is bit-identical (golden tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
File diff suppressed because it is too large
Load Diff
+12
-2
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"generatedAt": "2026-06-11T22:41:08.860Z",
|
||||
"generatedAt": "2026-06-11T23:06:22.047Z",
|
||||
"protocol": {
|
||||
"folds": [
|
||||
{
|
||||
@@ -56,5 +56,15 @@
|
||||
"testRps": 0.1647
|
||||
},
|
||||
"shipChange": false,
|
||||
"runtimeSec": 72
|
||||
"isotonic": {
|
||||
"bar": "LOFO RPS worsens ≤0.0002 AND LOFO ECE improves ≥10%",
|
||||
"lofo": {
|
||||
"rawRps": 0.1724,
|
||||
"calRps": 0.1722,
|
||||
"rawEce": 0.0146,
|
||||
"calEce": 0.0122
|
||||
},
|
||||
"ship": true
|
||||
},
|
||||
"runtimeSec": 73
|
||||
}
|
||||
Reference in New Issue
Block a user