| # Claim 6: Hyperparameter sensitivity (lambda_p) |
| |
| |
| --- |
| <!-- trackio-cell |
| {"type": "markdown", "id": "cell_7dada6d7e69e", "created_at": "2026-08-02T12:07:40+00:00", "title": "Claim 6: Hyperparameter sensitivity (lambda_p)"} |
| --> |
| **Verdict: TOY-VERIFIED** |
|
|
| Script: `lambda_p_sweep.py`. 12 reps, `hierarchical_shrinkage=True`, λs=25/λn=50 fixed, λp swept |
| over the paper's own values, accuracy + FP/FN reoccurrence at N=50 and N=100. |
|
|
| | λp | Accuracy | FP reocc. | FN reocc. | |
| |---|---|---|---| |
| | 0 | 96.92% | 6.65% | 0.77% | |
| | 0.5 | 96.99% | 6.55% | 0.79% | |
| | 1.0 | 97.03% | 6.55% | 0.76% | |
| | 5.0 | **98.28%** (peak) | 2.84% | **0.16%** (best) | |
| | 10.0 | 97.84% | 2.66% | 0.71% | |
| | 20.0 | 97.80% | 2.66% | 0.92% | |
| | 50.0 | 97.26% | **2.31%** (best) | 1.56% | |
|
|
| Paper: "higher λp can benefit accuracy, productive monotonicity, and error reoccurrence. λp=25.0 |
| retains these benefits while maintaining high precision near 100%." Direction matches cleanly: |
| moving from λp=0 to any λp≥5 roughly halves FP reoccurrence and substantially improves accuracy, |
| with a slight FN uptick only at the largest value tested (λp=50). This is consistent with the |
| paper's own framing of λp=25 as a middle-ground choice rather than a monotonic "more is always |
| better" relationship. λp=25 itself was not one of the paper's own swept values ({0, 0.5, 1, 5, 10, |
| 20, 50}), so no direct single-point comparison is possible — matches the paper's own sweep grid. |
|
|
