Spaces:
Running
Running
File size: 3,634 Bytes
4a12cb5 011d8c0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | {
"schema_version": 1,
"title": "Reproduction: A theory of learning data statistics in diffusion models, from easy to hard",
"emoji": "🌀",
"space_id": "snaykey/repro-embedding-defense",
"paper": {
"arxiv_id": "2603.12901",
"openreview_id": "xPdpfcJ65T"
},
"tags": [
"icml2026-repro",
"paper-xPdpfcJ65T"
],
"updated_at": "2026-07-28T00:00:00+00:00",
"root": {
"slug": "index",
"title": "Reproduction: A theory of learning data statistics in diffusion models, from easy to hard",
"file": "pages/index.md",
"children": [
{
"slug": "executive-summary",
"title": "Executive summary",
"file": "pages/executive-summary/page.md",
"children": []
},
{
"slug": "claim-1-figure-1-distributional-simplicity-bias",
"title": "Diffusion denoisers trained on natural images exhibit a distributional simplicity bias: on CIFAR-10, a U-Net achieves nearly identical test loss on real images versus Gaussian clones matched in mean and covariance for roughly 10^3 training steps before performance diverges, indicating pair-wise statistics are learned first (Figure 1).",
"file": "pages/claim-1-figure-1-distributional-simplicity-bias/page.md",
"children": []
},
{
"slug": "claim-2-proposition-4-3-sample-complexity-threshold",
"title": "For a k-th order cumulant target, the sample complexity threshold for weak recovery via projected gradient descent is n_hat(d,k) = omega(d^(k-1) log^2 d), with |v . w(n_hat)| approaching 1 as d approaches infinity after this many steps (Proposition 4.3).",
"file": "pages/claim-2-proposition-4-3-sample-complexity-threshold/page.md",
"children": []
},
{
"slug": "claim-3-proposition-4-4-negative-result",
"title": "If the number of samples n(d) = o(n_hat(d,k*)), standard SGD provably fails to achieve weak recovery of the k*-th cumulant, establishing a sharp learnable/unlearnable sample-complexity separation (Proposition 4.4).",
"file": "pages/claim-3-proposition-4-4-negative-result/page.md",
"children": []
},
{
"slug": "claim-4-proposition-4-6-independent-latents",
"title": "In a mixed-cumulant latent model, pair-wise statistics are recoverable with linear sample complexity n = Theta(d * polylog(d)), while recovering the fourth cumulant requires Omega(d^3) samples when latent variables are independent (Proposition 4.6).",
"file": "pages/claim-4-proposition-4-6-independent-latents/page.md",
"children": []
},
{
"slug": "claim-5-proposition-4-6-correlated-latents",
"title": "When latent variables underlying the pair-wise and higher-order statistics are positively correlated, the fourth-cumulant recovery sample complexity accelerates from cubic to linear in d (Proposition 4.6).",
"file": "pages/claim-5-proposition-4-6-correlated-latents/page.md",
"children": []
},
{
"slug": "claim-6-proposition-4-7-spherical-constraint",
"title": "Unconstrained SGD exhibits contraction dynamics toward w=0 for many activation functions, causing learning failure, whereas constraining updates to the unit sphere (spherical projection) preserves the ability to learn higher-order statistics (Proposition 4.7).",
"file": "pages/claim-6-proposition-4-7-spherical-constraint/page.md",
"children": []
},
{
"slug": "conclusion",
"title": "Conclusion",
"file": "pages/conclusion/page.md",
"children": []
}
]
}
} |