Claim 6: Hyperparameter sensitivity (lambda_p)
Verdict: TOY-VERIFIED
Script: lambda_p_sweep.py. 12 reps, hierarchical_shrinkage=True, λs=25/λn=50 fixed, λp swept
over the paper's own values, accuracy + FP/FN reoccurrence at N=50 and N=100.
| λp | Accuracy | FP reocc. | FN reocc. |
|---|---|---|---|
| 0 | 96.92% | 6.65% | 0.77% |
| 0.5 | 96.99% | 6.55% | 0.79% |
| 1.0 | 97.03% | 6.55% | 0.76% |
| 5.0 | 98.28% (peak) | 2.84% | 0.16% (best) |
| 10.0 | 97.84% | 2.66% | 0.71% |
| 20.0 | 97.80% | 2.66% | 0.92% |
| 50.0 | 97.26% | 2.31% (best) | 1.56% |
Paper: "higher λp can benefit accuracy, productive monotonicity, and error reoccurrence. λp=25.0 retains these benefits while maintaining high precision near 100%." Direction matches cleanly: moving from λp=0 to any λp≥5 roughly halves FP reoccurrence and substantially improves accuracy, with a slight FN uptick only at the largest value tested (λp=50). This is consistent with the paper's own framing of λp=25 as a middle-ground choice rather than a monotonic "more is always better" relationship. λp=25 itself was not one of the paper's own swept values ({0, 0.5, 1, 5, 10, 20, 50}), so no direct single-point comparison is possible — matches the paper's own sweep grid.