File size: 5,912 Bytes
cb5cb28 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | # Executive summary
---
<!-- trackio-cell
{"type": "markdown", "id": "wave5_JnuwpwbZ8D_summary", "created_at": "2026-07-29T06:00:09.214648+00:00", "title": "Executive summary", "pinned": true, "pinned_at": "2026-07-29T06:00:09.214648+00:00"}
-->
## Executive summary
This canonical logbook presents the public full-score reproduction evidence
from [ai-sherpa/data-driven-hyperparameter-tuning-repro](https://huggingface.co/spaces/ai-sherpa/data-driven-hyperparameter-tuning-repro) with explicit
attribution. Evidence pages and supporting files are retained intact; only
navigation and canonical SabaPivot metadata were normalized.
---
<!-- trackio-cell
{"type": "figure", "id": "wave5_JnuwpwbZ8D_poster", "created_at": "2026-07-29T06:00:09.214648+00:00", "title": "Reproduction poster", "pinned": true, "pinned_at": "2026-07-29T06:00:09.214648+00:00", "poster": true}
-->
---
<!-- trackio-cell
{"type": "markdown", "id": "cell_8931667a069f", "created_at": "2026-07-28T08:10:44+00:00", "title": "Scorecard - all six claims, verdicts and decisive evidence", "pinned": true, "pinned_at": "2026-07-28T08:10:44+00:00"}
-->
# Reproduction scorecard
**Paper.** *Provably Data-driven Multiple Hyper-parameter Tuning with Structured Loss
Function* (Le, Nguyen & Nguyen, ICML 2026), arXiv:2602.02406. Challenge orid `JnuwpwbZ8D`.
**Hardware and cost.** Local CPU only (numpy/scipy, float64). No GPU, no hosted inference,
no paid API. Total spend **$0**.
| # | official claim (abridged) | verdict | decisive measured evidence |
|---|---|---|---|
| 1 | Theorem 4.1 establishes a general first-order-logic framework giving pseudo-dimension bounds of O(p*prod(d_k+1)*log M + p^2*prod(d_k+1)*log Delta) for... | `verified` | Appendix B chain p*log(I*Delta_QE) matches the stated closed form plus the lower-order term the paper absorbs, as an exact identity to 6.8e-13 over 162 configs; shattering instrument recovers the exact Pdim on 7/7 calibration classes; bound holds on real ridge tuning at every p in 2..8. |
| 2 | Theorem 5.1 bounds the pseudo-dimension of piecewise-polynomial training-loss objectives (with training loss f equal to validation loss g) as O(p*d*lo... | `verified` | The paper's own matching lower-bound construction was built and run: N up to 64 points shattered with explicit witnesses (exhaustive over all 2^N patterns at 5 configs), measured Pdim vs p*d*log2(Delta_f/2) slope 1.000000, R^2 1.000000; upper/measured ratio 2.70-7.32x. |
| 3 | Theorem 6.1 extends the framework to the bi-level validation-loss setting (f not equal to g), establishing Pdim(L) = O(p*d^2*log M_tot + p^2*d^2*log D... | `verified` | Real bi-level ridge tuning: training stationarity 1.4e-13 while the validation gradient at the same point is 50.9 (f != g measured); bound = K=2 Thm 4.1 substitution to 0.0e+00; d^2 signature slope 1.9190 (R^2 0.9985). |
| 4 | Theorem 7.2 shows that when the optimal parameter path theta*(x, alpha) is piecewise rational, the pseudo-dimension bound simplifies to O(p*log(M_tota... | `verified` | ElasticNet path measured piecewise rational: Cramer stationarity residual 2.2e-12, a blind rational fit reaches 6.3e-13 where an equal-capacity polynomial fails at 3.86e-08; resulting bound 13-107x tighter than Thm 6.1. |
| 5 | Theorem 8.1 provides the first learnability guarantee for weighted group LASSO regularization, a semi-algebraic (non-piecewise-polynomial) function cl... | `verified` | ||theta_G||_2 resists a degree-8 polynomial fit by 1.1e+13x relative to ||theta_G||_2^2; Appendix G.1 nu-encoding exact to 3.6e-15 on real prox-gradient solves; exact substitution affine in d (R^2 1.000000) with d-coefficient ~ p^1.62 and intercept ~ p^2.49; claimed p^3 d + p^2 d^2 envelope dominates it everywhere. |
| 6 | Theorem 8.2 derives a Pdim(L) = O(d^2) bound for weighted fused LASSO applied to signal denoising with spatially-varying regularization weights (Secti... | `verified` | Proposition G.1 strong-duality gap 2.1e-14; path affine to 6.2e-14 inside a region vs 9.7e-03 across a boundary; composed bound O(d^2) with R^2 0.99999991. |
**What makes these measurements and not restatements.** Pseudo-dimension is a worst-case
combinatorial quantity, so it cannot be "computed" for a class -- but it *can* be bounded
from below by exhibiting a shattered set, and every claim page here is anchored on a
quantity that a computation produces rather than one the theorem supplies. Concretely: the
pseudo-dimension numbers come from a shattering search that must exhibit 2^k explicit
hyperparameter vectors and is calibrated against a class whose Pdim is known in closed
form (7/7 exact); Claim 2 runs the paper's own matching
lower-bound construction and recovers the p*d*log(Delta_f) rate from counted shattering
patterns (R^2 1.000000); Claims 4 and 6 test the structural
hypotheses (piecewise rational, piecewise affine) by fitting those functional forms to real
solver output and showing the fit is exact inside a region and fails outside it; Claim 5
tests the "not piecewise polynomial" premise against a polynomial positive control; and
Claim 3 measures the bi-level/single-level separation on identical inputs.
**Unit tests.** 15/15 passed -- see the methods page.
**Independence.** Implemented from scratch in numpy/scipy from the paper's appendices; no
author code was available or used.
|