File size: 2,953 Bytes
124c088 5cc1d21 124c088 5cc1d21 124c088 5cc1d21 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | {
"schema_version": 1,
"title": "Repro: A Theoretical Framework for Modular Learning of Robust Generative Models",
"emoji": "jigsaw",
"space_id": "snaykey/repro-modular-robust-gen",
"paper": {
"openreview_id": "ToCArdndt6"
},
"tags": [
"icml2026-repro",
"paper-ToCArdndt6"
],
"updated_at": "2026-07-26T00:00:00+00:00",
"root": {
"slug": "index",
"title": "Repro: A Theoretical Framework for Modular Learning of Robust Generative Models",
"file": "pages/index.md",
"children": [
{
"slug": "executive-summary",
"title": "Executive summary",
"file": "pages/executive-summary/page.md",
"children": []
},
{
"slug": "claim-1-robust-gate",
"title": "The robust gate existence result is proven via a saddle point of a linearized minimax game over the normalized gate space G_1 and the mixture simplex, established using Kakutani's Fixed Point Theorem (Theorem 3, Section 4.2).",
"file": "pages/claim-1-robust-gate/page.md",
"children": []
},
{
"slug": "claim-2-kl-bound",
"title": "Theorem 3 bounds the KL divergence of the robust gated model to any mixture by log-sum-exp of the per-expert guarantees epsilon_k minus an overlap term (target-weighted conditional entropy) minus a diversity term equal to the Jensen-Shannon Divergence, so the modular approach can outperform aggregate retraining by a gap characterized by JSD (Section 4.2).",
"file": "pages/claim-2-kl-bound/page.md",
"children": []
},
{
"slug": "claim-3-static-capacity",
"title": "Any static (non-adaptive) mixture-weight gate is shown to be fundamentally capacity-limited, with a capacity lower bound of log(sum_k e^{epsilon_k}) for disjoint domains, motivating the need for an input-dependent gate (Theorem 16, Appendix B).",
"file": "pages/claim-3-static-capacity/page.md",
"children": []
},
{
"slug": "claim-4-algorithm-distillation",
"title": "The paper introduces a Stochastic Primal-Dual algorithm to solve the constrained minimax game at scale and a Structural Distillation method that maps the non-causal gate to a causal router for efficient autoregressive inference (Sections 5 and 7).",
"file": "pages/claim-4-algorithm-distillation/page.md",
"children": []
},
{
"slug": "claim-5-empirical",
"title": "Empirical results on synthetic benchmarks and real-world datasets (Wikipedia, Code, FineWeb) show the modular gated architecture mitigates gradient conflict and outperforms monolithic baselines in high-interference regimes (Section 8, Section 8.1).",
"file": "pages/claim-5-empirical/page.md",
"children": []
},
{
"slug": "conclusion",
"title": "Conclusion",
"file": "pages/conclusion/page.md",
"children": []
}
]
}
} |