bundle/v0/HYPERGRADIENT_VALIDATION.md

Hypergradient component validation

Scope

This is bounded validation evidence for the analytic gradient oracle used by the independent reproduction. It is not a paper-scale experiment and does not establish either C1 or C2 by itself. C1, C2, and C3 remain HOLD.

The validation covers every implemented lower-level route family:

1. Gaussian portfolio with the exact Gaussian risk coefficient
2. Gaussian portfolio with the moment-class sensitivity coefficient
3. Empirical W1 portfolio
4. W1 absolute regression
5. W2 squared regression, including the root objective and post-solve square chain rule

Each route checks the distance derivative, lower-value derivative at the identity metric, lower-value derivative at the active penalty point, smoothed coverage-violation derivative, active penalty derivative, and total hypergradient. Squared regression also checks the root lower-value derivative.

Numerical contract

• Lower-triangular central differences use steps 1e-4, 1e-5, and 1e-6.
• 1e-5 is the predeclared primary step.
• A component passes only if the primary step passes and at least two of three steps pass.
• Each step passes when relative Frobenius error is at most 1e-3 or maximum absolute error is at most 1e-6.
• Conic and transport residual thresholds are both 1e-7.
• Every plus and minus scalar evaluation retains its solver or transport residual receipt.
• Evaluation failures are retained in the artifact failure ledger. They are never dropped from aggregation.

Executed result

The immutable artifact was generated from clean source commit 348fb859246fd9f42bf71928ff9b654e0978c587.

• 4 route families, 5 route variants
• 31 analytic component checks
• 93 numerical gradient matrices
• 882 retained plus/minus scalar evaluations
• 764 conic residual receipts
• 4,005 transport residual receipts
• 0 failures
• All 31 components passed all three step sizes
• Largest primary-step relative error: 7.625598602017288e-06, empirical W1 portfolio active lower value
• Largest relative error anywhere in the step grid: 3.5048066444505585e-05, squared-regression total hypergradient at 1e-6
• Maximum conic residual: 3.9171789151799395e-11
• Maximum transport residual: 3.6529009592165657e-14

Artifact bindings:

• Artifact SHA-256: 89010cd8d96aa144f62953550cbf92061fc966ae41dd303e48ea683158632511
• Adjacent sidecar: 89010cd8d96aa144f62953550cbf92061fc966ae41dd303e48ea683158632511 hypergradient.json
• Evidence payload: sha256:a822c97905c460c0b914a119195ef19e8d25834ea23d0930027194cf781215a0
• Base Git tree: ba5c9b5c1acad7cd56cf46678208e38f51e19695
• Scientific files proved against Git blobs: 41
• Source tree: sha256:c08bf440638d1d68a3ad2a76442ab6cfd68d0aeeab2d4520d3bdde8615b4325a
• Validation config: sha256:a14d7dc8a9b17cd2555eeb48a91045a4ca1df0aad6533cf73836243dbf70e6e4
• Paper plan: sha256:f4312037cbf0a49c291ec624c8598d04507524b2d522a00be4adaacc6d716a54
• Locked packages: NumPy 2.4.6, SciPy 1.17.1, CVXPY 1.9.2, Clarabel 0.11.1, POT 0.9.7
• CPU: Intel(R) Core(TM) Ultra 9 275HX

Reproduce and validate

From this paper lane:

$env:OMP_NUM_THREADS="1"
$env:MKL_NUM_THREADS="1"
$env:OPENBLAS_NUM_THREADS="1"
$env:NUMEXPR_NUM_THREADS="1"
$env:PYTHONPATH="$PWD/src"
& .venv/Scripts/python.exe -B scripts/run_hypergradient_validation.py `
  --config configs/hypergradient_validation_v1.json `
  --output .openresearch/artifacts/validation/hypergradient.json

& .venv/Scripts/python.exe -B scripts/validate_hypergradient.py `
  .openresearch/artifacts/validation/hypergradient.json `
  --config configs/hypergradient_validation_v1.json

Create .venv with CPython 3.11 and install environment/scientific-freeze.txt exactly. environment/requirements.lock lists the direct scientific dependencies; the transitive freeze is the canonical regeneration environment. The runner and full validator fail closed unless NumPy, SciPy, CVXPY, Clarabel, and POT match both locks. They also force the four declared BLAS thread variables to one before importing NumPy.

The runner refuses an existing artifact or sidecar and requires a clean paper lane. It writes hypergradient.json plus the adjacent hypergradient.sha256. The validator requires that sidecar, proves every bound scientific file against the named Git commit's tree and blob objects, reconstructs each numerical gradient from retained scalar evaluations, and independently regenerates the complete evidence payload.

--structural-only is limited to adversarial tests. Its success message explicitly states that scientific regeneration was not run. It cannot serve as a regeneration receipt.

Test receipt

• Hypergradient and adversarial validator suite: 12 passed
• Directly affected scientific regression suite: 78 passed
• Python compilation check: passed

Eight additional historical-canary validator tests were attempted and failed only because this isolated worktree does not contain the ignored appendix-canary-v4 and scientific-canary-v12 directories they copy as fixtures. They are recorded as unavailable, not as passes and not as regressions caused by this repair. The directly affected suite named above is the test evidence for this artifact.