soma-cot-compression / base_rate_test.json
XXMiner's picture
v90: loss mechanism pinned (100% of lost terms only in call bodies); two wrong conclusions corrected; held-out gain +0.259 char, a tenth of the oracle; 419 tests
519d6ed verified
Raw
History Blame Contribute Delete
638 Bytes
{
"quartiles": [
{
"quartile": 1,
"n": 37,
"losers": 0,
"loser_rate": 0.0,
"mean_uniq_at_risk": 7.9
},
{
"quartile": 2,
"n": 37,
"losers": 8,
"loser_rate": 0.2162,
"mean_uniq_at_risk": 32.7
},
{
"quartile": 3,
"n": 37,
"losers": 9,
"loser_rate": 0.2432,
"mean_uniq_at_risk": 82.5
},
{
"quartile": 4,
"n": 39,
"losers": 4,
"loser_rate": 0.1026,
"mean_uniq_at_risk": 239.8
}
],
"rate_spread": 0.2432,
"flat_prediction_holds": false,
"corr_loss_vs_atrisk_among_losers": -0.2078
}