covtoken / gate_reports /gate_4_block3.json
Chucks90's picture
covtoken: label-free lesion-subspace token economy (reframed) + gated eval + paper draft
3510f1d verified
Raw
History Blame Contribute Delete
3.6 kB
{
"gate": 4,
"phase": "Phase 4 - Constraint binds + budget adapts (mechanism + money plot) [block-3]",
"status": "PARTIAL",
"status_detail": "Mechanism PASSES (dual stable + constraint binds/satisfies). The difficulty-adaptive budget (money plot) does NOT emerge for small lesions.",
"metrics": [
{
"name": "dual_mu_stability", "modality": "LIDC-IDRI", "budget": null,
"value": 1.00, "ci95": null, "test": "running_var_last20%<=first20%",
"threshold": 0.80, "threshold_status": "FIXED", "passed": true,
"detail": "mu trajectory stabilizes on 100% of cases; no divergence. The dual variable is a stable controller (no RL)."
},
{
"name": "constraint_satisfaction_rate", "modality": "LIDC-IDRI", "budget": null,
"value": 0.995, "ci95": null, "test": "delta_C<=epsilon rate",
"threshold": 0.95, "threshold_status": "FIXED", "passed": true,
"detail": "At epsilon=12.5 (tight), the coverage floor is satisfied on 99.5% of cases; the constraint binds (mean delta_C ~9.96)."
},
{
"name": "adaptive_budget_k_pos_minus_k_neg", "modality": "LIDC-IDRI", "budget": null,
"value": -2.03, "ci95": [-2.97, -1.04], "test": "bootstrap_diff + Cohen_d",
"threshold": 0.0, "threshold_status": "CALIBRATE(>0, d>=0.5)", "passed": false,
"detail": "k_pos=141.3 (72%) vs k_neg=143.4 (73%): NO adaptive budget (in fact slightly lower on positives). Cohen d=-0.41."
},
{
"name": "Cstar_pos_vs_neg", "modality": "LIDC-IDRI", "budget": null,
"value": 1.013, "ci95": null, "test": "ratio C*_pos/C*_neg",
"threshold": null, "threshold_status": "diagnostic", "passed": false,
"detail": "C*_pos=250.4 vs C*_neg=247.2 (1.3% diff). The ROOT CAUSE: aggregate lesion-subspace coverage barely differs between lesion-positive and -negative slices."
}
],
"thresholds_locked_ref": null,
"seeds": [0],
"data_gaps": [],
"root_cause": "The coverage functional C(S;x)=effective-rank(P_L Z) is an AGGREGATE over all 196 tokens. A small nodule (1-3 patches) is highly localizable at the TOKEN level (Gate 1 AUROC 0.87) but contributes negligibly to the aggregate effective rank, which is dominated by the ~190 non-lesion tokens' projections onto P_L. So pathological slices do NOT have materially higher C*, and the difficulty-adaptive budget (formalization 6) has no signal. Same root cause as Gate 2's borderline faithfulness.",
"interpretation": "The constrained-optimization MECHANISM is sound and validated (interpretable dual mu is a stable controller; per-image coverage floor binds and is satisfied; per-image certificate is well-defined). But the EMERGENT difficulty-adaptive budget property does not materialize for SMALL lesions under effective-rank coverage. It is expected to emerge where lesions span many patches (larger lesions) -- testable on liver/pancreas/kidney where tumors are larger. An energy-based coverage (||P_L Z|| rather than effective rank) is the natural design change to make the budget lesion-sensitive for small lesions; left as a calibrated option.",
"decision_rule": "PASS iff dual stable (>=80%) AND constraint satisfied (>=95%) AND k(pos)>k(neg) [CI excludes 0, Cohen d>=0.5].",
"decision": "PARTIAL. Constraint binds and the dual is a stable, satisfied controller (Metric A + satisfaction PASS). The money plot (Metric B) FAILS on small LIDC nodules because aggregate coverage is lesion-insensitive at this lesion size -- an honest scope condition, not a mechanism failure. Headline pruning benefit (Gate 3) is unaffected.",
"human_signoff": null
}