{
  "schema_version": 1,
  "title": "Reproduction: A theory of learning data statistics in diffusion models, from easy to hard",
  "emoji": "🌀",
  "space_id": "snaykey/repro-embedding-defense",
  "paper": {
    "arxiv_id": "2603.12901",
    "openreview_id": "xPdpfcJ65T"
  },
  "tags": [
    "icml2026-repro",
    "paper-xPdpfcJ65T"
  ],
  "updated_at": "2026-07-28T00:00:00+00:00",
  "root": {
    "slug": "index",
    "title": "Reproduction: A theory of learning data statistics in diffusion models, from easy to hard",
    "file": "pages/index.md",
    "children": [
      {
        "slug": "executive-summary",
        "title": "Executive summary",
        "file": "pages/executive-summary/page.md",
        "children": []
      },
      {
        "slug": "claim-1-figure-1-distributional-simplicity-bias",
        "title": "Diffusion denoisers trained on natural images exhibit a distributional simplicity bias: on CIFAR-10, a U-Net achieves nearly identical test loss on real images versus Gaussian clones matched in mean and covariance for roughly 10^3 training steps before performance diverges, indicating pair-wise statistics are learned first (Figure 1).",
        "file": "pages/claim-1-figure-1-distributional-simplicity-bias/page.md",
        "children": []
      },
      {
        "slug": "claim-2-proposition-4-3-sample-complexity-threshold",
        "title": "For a k-th order cumulant target, the sample complexity threshold for weak recovery via projected gradient descent is n_hat(d,k) = omega(d^(k-1) log^2 d), with |v . w(n_hat)| approaching 1 as d approaches infinity after this many steps (Proposition 4.3).",
        "file": "pages/claim-2-proposition-4-3-sample-complexity-threshold/page.md",
        "children": []
      },
      {
        "slug": "claim-3-proposition-4-4-negative-result",
        "title": "If the number of samples n(d) = o(n_hat(d,k*)), standard SGD provably fails to achieve weak recovery of the k*-th cumulant, establishing a sharp learnable/unlearnable sample-complexity separation (Proposition 4.4).",
        "file": "pages/claim-3-proposition-4-4-negative-result/page.md",
        "children": []
      },
      {
        "slug": "claim-4-proposition-4-6-independent-latents",
        "title": "In a mixed-cumulant latent model, pair-wise statistics are recoverable with linear sample complexity n = Theta(d * polylog(d)), while recovering the fourth cumulant requires Omega(d^3) samples when latent variables are independent (Proposition 4.6).",
        "file": "pages/claim-4-proposition-4-6-independent-latents/page.md",
        "children": []
      },
      {
        "slug": "claim-5-proposition-4-6-correlated-latents",
        "title": "When latent variables underlying the pair-wise and higher-order statistics are positively correlated, the fourth-cumulant recovery sample complexity accelerates from cubic to linear in d (Proposition 4.6).",
        "file": "pages/claim-5-proposition-4-6-correlated-latents/page.md",
        "children": []
      },
      {
        "slug": "claim-6-proposition-4-7-spherical-constraint",
        "title": "Unconstrained SGD exhibits contraction dynamics toward w=0 for many activation functions, causing learning failure, whereas constraining updates to the unit sphere (spherical projection) preserves the ability to learn higher-order statistics (Proposition 4.7).",
        "file": "pages/claim-6-proposition-4-7-spherical-constraint/page.md",
        "children": []
      },
      {
        "slug": "conclusion",
        "title": "Conclusion",
        "file": "pages/conclusion/page.md",
        "children": []
      }
    ]
  }
}