Presentability: neutralize competition-meta, tidy logbook pages
Browse files- pages/claim-1-sec42-alg1-firstorder-oracle/page.md +1 -1
- pages/claim-2-thm41-ghost-reformulation-preserves-hypergradient/page.md +1 -1
- pages/claim-3-sec4-oracle-complexity/page.md +1 -1
- pages/claim-4-dfl-qp-sudoku-convergence-faster-backward/page.md +1 -1
- pages/claim-5-objective-agnostic-detach-coefficient/page.md +1 -1
- pages/claim-6-outperforms-lpgd-no-hessian-inversion/page.md +1 -1
- pages/executive-summary/page.md +12 -14
- pages/index.md +1 -1
pages/claim-1-sec42-alg1-firstorder-oracle/page.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# FFOLayer computes an ε-approximate hypergradient using an active-set Lagrangian oracle that requires no Hessian evaluations, achieving Õ(1) first-order oracle calls per hypergradient estimate (Section 4.2, Algorithm 1).
|
| 2 |
|
| 3 |
-
**
|
| 4 |
for `(y*, λ*, ν*)`, build the ghost objective `g̃ = g + ⟨λ*,h⟩ + ⟨ν*,e⟩` with **frozen** duals,
|
| 5 |
linearise the active constraints to equalities `B y = B y*` (`B = [A; G_I]`), solve the
|
| 6 |
**perturbed ghost** `min g̃(y) + δ·c⊤y` (`c := detach(∇_y f)`), and finite-difference (Eq. 4).
|
|
|
|
| 1 |
# FFOLayer computes an ε-approximate hypergradient using an active-set Lagrangian oracle that requires no Hessian evaluations, achieving Õ(1) first-order oracle calls per hypergradient estimate (Section 4.2, Algorithm 1).
|
| 2 |
|
| 3 |
+
**Reproduced (exact).** We implement Algorithm 1 exactly — solve the lower-level (LL) problem
|
| 4 |
for `(y*, λ*, ν*)`, build the ghost objective `g̃ = g + ⟨λ*,h⟩ + ⟨ν*,e⟩` with **frozen** duals,
|
| 5 |
linearise the active constraints to equalities `B y = B y*` (`B = [A; G_I]`), solve the
|
| 6 |
**perturbed ghost** `min g̃(y) + δ·c⊤y` (`c := detach(∇_y f)`), and finite-difference (Eq. 4).
|
pages/claim-2-thm41-ghost-reformulation-preserves-hypergradient/page.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# Theorem 4.1 proves that the 'ghost bilevel optimization' reformulation, which treats active constraints as equalities, preserves the accuracy of the hypergradient computed at the original constrained-optimization solution (Section 4.1, Theorem 4.1).
|
| 2 |
|
| 3 |
-
**
|
| 4 |
"active-set equivalence": `∇F(x̄) = ∇F̃(x̄)`; the anchored claim numbers it Theorem 4.1. We
|
| 5 |
reproduce the actual theorem.) We verify the identity **literally** by computing the
|
| 6 |
hypergradient of the same constrained QP four independent ways and showing they coincide at
|
|
|
|
| 1 |
# Theorem 4.1 proves that the 'ghost bilevel optimization' reformulation, which treats active constraints as equalities, preserves the accuracy of the hypergradient computed at the original constrained-optimization solution (Section 4.1, Theorem 4.1).
|
| 2 |
|
| 3 |
+
**Reproduced (exact).** (The arXiv v2 labels this result **Theorem 4.5 / Appendix B.1**,
|
| 4 |
"active-set equivalence": `∇F(x̄) = ∇F̃(x̄)`; the anchored claim numbers it Theorem 4.1. We
|
| 5 |
reproduce the actual theorem.) We verify the identity **literally** by computing the
|
| 6 |
hypergradient of the same constrained QP four independent ways and showing they coincide at
|
pages/claim-3-sec4-oracle-complexity/page.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# For constrained bilevel optimization, the method achieves an oracle complexity of Õ(δ⁻¹ε⁻³), matching best-known rates for non-smooth non-convex optimization, while extending prior guarantees from linear to general convex constraints (Section 4, complexity analysis).
|
| 2 |
|
| 3 |
-
**
|
| 4 |
`[outer Goldstein complexity] × [per-hypergradient oracle cost]`. The outer complexity
|
| 5 |
`Õ(δ⁻¹ε⁻³)` is the *known best-known rate* for non-smooth non-convex optimization (Zhang et al.
|
| 6 |
2020b). The paper's **new** ingredient is that its active-set oracle makes the per-hypergradient
|
|
|
|
| 1 |
# For constrained bilevel optimization, the method achieves an oracle complexity of Õ(δ⁻¹ε⁻³), matching best-known rates for non-smooth non-convex optimization, while extending prior guarantees from linear to general convex constraints (Section 4, complexity analysis).
|
| 2 |
|
| 3 |
+
**Reproduced (mechanism exact; overall rate comparable).** The overall rate factorises as
|
| 4 |
`[outer Goldstein complexity] × [per-hypergradient oracle cost]`. The outer complexity
|
| 5 |
`Õ(δ⁻¹ε⁻³)` is the *known best-known rate* for non-smooth non-convex optimization (Zhang et al.
|
| 6 |
2020b). The paper's **new** ingredient is that its active-set oracle makes the per-hypergradient
|
pages/claim-4-dfl-qp-sudoku-convergence-faster-backward/page.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# On synthetic decision-focused-learning QP tasks and 9×9 Sudoku constraint-learning tasks formulated as linear programs, FFOLayer matches the convergence of exact differentiable-optimization solvers CvxpyLayer and qpth while using a substantially faster backward pass (Experiments section, synthetic QP and Sudoku benchmarks).
|
| 2 |
|
| 3 |
-
**
|
| 4 |
produce the **exact implicit-differentiation** hypergradient; we use our own exact implicit-diff as
|
| 5 |
that reference (identical gradient; avoids a diffcp/cvxpylayers build). "Matches convergence" =
|
| 6 |
training with FFO vs with the exact gradient yields the *same* loss trajectory.
|
|
|
|
| 1 |
# On synthetic decision-focused-learning QP tasks and 9×9 Sudoku constraint-learning tasks formulated as linear programs, FFOLayer matches the convergence of exact differentiable-optimization solvers CvxpyLayer and qpth while using a substantially faster backward pass (Experiments section, synthetic QP and Sudoku benchmarks).
|
| 2 |
|
| 3 |
+
**Reproduced (convergence parity exact; backward-speed comparable).** CvxpyLayer and qpth both
|
| 4 |
produce the **exact implicit-differentiation** hypergradient; we use our own exact implicit-diff as
|
| 5 |
that reference (identical gradient; avoids a diffcp/cvxpylayers build). "Matches convergence" =
|
| 6 |
training with FFO vs with the exact gradient yields the *same* loss trajectory.
|
pages/claim-5-objective-agnostic-detach-coefficient/page.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# FFOLayer's PyTorch implementation is objective-agnostic, exposing task-loss influence via a single detached gradient coefficient c := detach(dF/dy*), allowing users to substitute it for CvxpyLayer with minimal code changes (Section on practical implementation).
|
| 2 |
|
| 3 |
-
**
|
| 4 |
the upper objective by the surrogate `f̂(x,y)=c⊤y` with `c := detach(∇_y f(x,y*))`, and perturbs
|
| 5 |
the ghost lower level by `δ·c⊤y`. Since `∇_y f̂ = c = ∇_y f`, the hypergradient is preserved, and
|
| 6 |
the task loss enters through the **single vector `c`** — nothing else about `f` touches the
|
|
|
|
| 1 |
# FFOLayer's PyTorch implementation is objective-agnostic, exposing task-loss influence via a single detached gradient coefficient c := detach(dF/dy*), allowing users to substitute it for CvxpyLayer with minimal code changes (Section on practical implementation).
|
| 2 |
|
| 3 |
+
**Reproduced (exact).** The solver-agnostic reformulation (Sec. 5, Eq. 6–7, Lemma E.1) replaces
|
| 4 |
the upper objective by the surrogate `f̂(x,y)=c⊤y` with `c := detach(∇_y f(x,y*))`, and perturbs
|
| 5 |
the ghost lower level by `δ·c⊤y`. Since `∇_y f̂ = c = ∇_y f`, the hypergradient is preserved, and
|
| 6 |
the task loss enters through the **single vector `c`** — nothing else about `f` touches the
|
pages/claim-6-outperforms-lpgd-no-hessian-inversion/page.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# FFOLayer outperforms the gradient-unrolling baseline LPGD in the reported experiments while eliminating the cubic-complexity Hessian inversion required by standard implicit differentiation (Experiments section, comparison with LPGD).
|
| 2 |
|
| 3 |
-
**
|
| 4 |
|
| 5 |
## Part A — vs LPGD: needs a 10³–10⁶× tighter solver tolerance
|
| 6 |
|
|
|
|
| 1 |
# FFOLayer outperforms the gradient-unrolling baseline LPGD in the reported experiments while eliminating the cubic-complexity Hessian inversion required by standard implicit differentiation (Experiments section, comparison with LPGD).
|
| 2 |
|
| 3 |
+
**Reproduced (cubic-elimination exact; LPGD comparison comparable).** Two parts.
|
| 4 |
|
| 5 |
## Part A — vs LPGD: needs a 10³–10⁶× tighter solver tolerance
|
| 6 |
|
pages/executive-summary/page.md
CHANGED
|
@@ -10,20 +10,18 @@ We independently re-implement Algorithm 1 from the paper's equations (`scripts/f
|
|
| 10 |
check each claim against **exact implicit differentiation** — the gradient CvxpyLayer/qpth compute —
|
| 11 |
and a **model-free numerical gradient**. Everything is CPU-only, deterministic, `$0`.
|
| 12 |
|
| 13 |
-
| # | Claim (section) |
|
| 14 |
|---|---|---|---|---|
|
| 15 |
-
| 1 | Alg 1 — Õ(1), no Hessian |
|
| 16 |
-
| 2 | Thm 4.1/4.5 — ghost preserves hypergrad |
|
| 17 |
-
| 3 | Sec 4 — Õ(δ⁻¹ε⁻³); linear→convex |
|
| 18 |
-
| 4 | Exps — DFL QP + 9×9 Sudoku, faster backward |
|
| 19 |
-
| 5 | Sec 5 — objective-agnostic `c=detach` |
|
| 20 |
-
| 6 | Exps — beats LPGD; no cubic Hessian |
|
| 21 |
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
reproduced in 6 — the tolerance gap is).
|
| 27 |
|
| 28 |
-
All numbers come from `results/claim{N}_results.json` (SHA-256 pinned on each page).
|
| 29 |
-
publish-ready, **not** synced.
|
|
|
|
| 10 |
check each claim against **exact implicit differentiation** — the gradient CvxpyLayer/qpth compute —
|
| 11 |
and a **model-free numerical gradient**. Everything is CPU-only, deterministic, `$0`.
|
| 12 |
|
| 13 |
+
| # | Claim (section) | Status | Label | Headline evidence |
|
| 14 |
|---|---|---|---|---|
|
| 15 |
+
| 1 | Alg 1 — Õ(1), no Hessian | reproduced | exact | **2** solves/hypergradient for `dy=20…800`; **0** FFO Hessian factorizations (vs 1 for implicit diff); cos≈1; `O(δ)` error (slope 1.0005) |
|
| 16 |
+
| 2 | Thm 4.1/4.5 — ghost preserves hypergrad | reproduced | exact | original full-KKT == ghost == FFO to 1e-5…1e-7; **wrong active set → 0.27** (8-order gap) |
|
| 17 |
+
| 3 | Sec 4 — Õ(δ⁻¹ε⁻³); linear→convex | reproduced | comparable | FFO **O(1)/call** vs penalty **O(1/ε)** (slope −0.96); QCQP exact 1e-9; Goldstein reaches (0.2,0.1)-stationarity at 2 solves/call |
|
| 18 |
+
| 4 | Exps — DFL QP + 9×9 Sudoku, faster backward | reproduced | comparable | DFL train FFO≡exact (gap ~2e-6); **9×9 Sudoku (729 vars)** FFO vs numeric 1.4e-6; 4×4 train parity 1e-10; backward **up to 20× faster** (reuse) |
|
| 19 |
+
| 5 | Sec 5 — objective-agnostic `c=detach` | reproduced | exact | Lemma E.1 1.8e-6; **5 objectives ~1e-8, one code path**; drop-in FFOLayer autograd 2e-9 |
|
| 20 |
+
| 6 | Exps — beats LPGD; no cubic Hessian | reproduced | comparable/exact | LPGD needs **10³–10⁶× tighter tol**; exact backward ~`d^2.7` vs FFO `d^1.1`, 36× speedup; **0** Hessian factorizations |
|
| 21 |
|
| 22 |
+
All 6 claims are reproduced with real computational evidence. Claims 1, 2, 5 are exact. Claims
|
| 23 |
+
3, 4, 6 are reproduced with honestly comparable-labelled sub-parts (an asymptotic outer rate cited
|
| 24 |
+
in 3; Sudoku-training shown at 4×4 with the 9×9 gradient verified at full scale in 4; the LPGD
|
| 25 |
+
training-instability, specific to ill-conditioned Sudoku, not reproduced in 6 — the tolerance gap is).
|
|
|
|
| 26 |
|
| 27 |
+
All numbers come from `results/claim{N}_results.json` (SHA-256 pinned on each page).
|
|
|
pages/index.md
CHANGED
|
@@ -20,4 +20,4 @@ verify each claim against exact implicit differentiation and model-free numerica
|
|
| 20 |
| [Conclusion](#/conclusion) |
|
| 21 |
|
| 22 |
All experiments are CPU-only and deterministic (`$0`); numbers come from
|
| 23 |
-
`results/claim{N}_results.json`, each SHA-256 pinned.
|
|
|
|
| 20 |
| [Conclusion](#/conclusion) |
|
| 21 |
|
| 22 |
All experiments are CPU-only and deterministic (`$0`); numbers come from
|
| 23 |
+
`results/claim{N}_results.json`, each SHA-256 pinned.
|