File size: 2,927 Bytes
537d7fa f758236 537d7fa f758236 537d7fa f758236 537d7fa f758236 537d7fa f758236 537d7fa | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | {
"schema_version": 1,
"title": "Reproduction: The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks",
"emoji": "🧭",
"space_id": "ProCreations/repro-the-implicit-bias-of-adam-and-muon-on-smooth-homogeneous-neural-networks",
"paper": {"arxiv_id": "2602.16340"},
"tags": ["icml2026-repro", "paper-DpIc1cpNKG"],
"claims": [
"Theorem 3.2 proves that any limit point of the normalized parameter trajectory θₜ/‖θₜ‖ under normalized steepest descent is the direction of a KKT point of the corresponding max-margin problem (Theorem 3.2).",
"Theorem 3.3 extends the margin-maximization convergence result to momentum steepest descent under decaying learning rates, using an Approximate Steepest Descent framework introduced in Definition 5.1 (Theorem 3.3, Definition 5.1).",
"Corollary 3.4 shows Muon, applied simultaneously across weight matrices, is a special case of normalized momentum steepest descent with respect to the matrix spectral norm ‖·‖_msp (Corollary 3.4).",
"Corollary 3.5 shows Muon-Signum hybrid algorithms exhibit margin maximization with respect to the max of the spectral norm and the ℓ∞ norm (Corollary 3.5).",
"Theorem 3.6 shows Adam without a stability constant exhibits implicit bias toward ℓ∞-margin maximization under a decaying learning rate regime when c₁ ≥ c₂ (Theorem 3.6).",
"Theorem 3.1 establishes that under normalized steepest descent with a learning rate schedule satisfying ∫₀^∞ η(t)dt = ∞, the soft margin γ̃(θₜ) increases monotonically (Theorem 3.1)."
],
"updated_at": "2026-07-21T15:49:36+00:00",
"root": {
"slug": "index",
"title": "Reproduction: The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks",
"file": "pages/index.md",
"children": [
{"slug": "executive-summary", "title": "Executive summary", "file": "pages/executive-summary/page.md", "children": []},
{"slug": "claim-1-kkt", "title": "Claim 1: KKT limit points", "file": "pages/claim-1-kkt/page.md", "children": []},
{"slug": "claim-2-momentum", "title": "Claim 2: momentum approximate descent", "file": "pages/claim-2-momentum/page.md", "children": []},
{"slug": "claim-3-muon", "title": "Claim 3: Muon spectral geometry", "file": "pages/claim-3-muon/page.md", "children": []},
{"slug": "claim-4-hybrid", "title": "Claim 4: Muon-Signum hybrid", "file": "pages/claim-4-hybrid/page.md", "children": []},
{"slug": "claim-5-adam", "title": "Claim 5: Adam infinity bias", "file": "pages/claim-5-adam/page.md", "children": []},
{"slug": "claim-6-soft-margin", "title": "Claim 6: soft-margin monotonicity", "file": "pages/claim-6-soft-margin/page.md", "children": []},
{"slug": "conclusion", "title": "Conclusion", "file": "pages/conclusion/page.md", "children": []}
]
},
"agent_view_tokens": 4700,
"revision": "1784648976000000000"
}
|