neural_physics_engine_roadmap.md

From Tied-Basis Subspace Sims to a Neural Physics Engine

A scaling roadmap grounded in what we actually measured

0. What we proved, in one abstraction

Every system we built follows one skeleton:

per-element LOCAL PROJECTION  →  GLOBAL RECONCILIATION
(tied/shared across elements)    (cheap solve or reduction)

System	Local projection (tied)	Global step
Solid (tower, balls)	co-rotate → strain → shared 4×2 basis → target	prefactored PD solve, or r×r reduced
Fluid (PBF water)	density constraint λ (same kernel for all particles)	position corrections + neighbor grid
Coupling	acceleration penalty (same rule for all pairs)	folded into both

Measured facts to build on:

• One 8-float strain matrix served every triangle and transferred to unseen load cases at 99%+ coverage — local physics is scenario-independent.
• Co-rotation reduced strain to an exactly 3-dim space (4th singular value = 0) — handling symmetry analytically shrinks what must be learned.
• Constraint folding was algebraically exact — free 3.25×.
• The failures were never the learned parts: they were fixed hyperparameters (iteration budget), missing guards (inversion), and integration details (affine offset, warm-start).

The neural engine thesis: replace each hand-derived projection with a learned one, keep the skeleton, keep the analytic symmetry handling.

---

1. The core architectural bet: constraint projectors as tied embeddings

The 4×2 PCA basis is a linear, shared encoder/decoder. The neural generalization:

z = Enc_θ(p, m)          # p: co-rotated local state, m: material token
p̂ = Dec_θ(z, m)          # Dec = Enc^T in the tied-linear case
target = R(I + p̂)        # rotation stays ANALYTIC

• θ shared across every element — this is your embedding-table insight, and it's why the model stays tiny (KBs, not MBs) and mesh-independent.
• Material tokens m: a learned embedding per material/constraint type (rubber, steel, water-density, volume-preservation, contact). One projector network, conditioned — literally tied embeddings across physics. New material = new token, not new network.
• Weight tying Enc/Dec: in the linear case it guaranteed a symmetric PSD reduced operator (solver stability). Keep it in the nonlinear case as Dec = Jacobian-transpose structure or an explicit energy formulation: learn a scalar energy E_θ(z, m) and take gradients — symmetry then comes for free and PD convergence theory mostly survives.

Do not learn the rotation. Polar decomposition (closed-form 2D, Müller-iterative 3D) is exact, cheap, and removing it from the learning problem is what made a 2-dim basis sufficient. Learned rotation handling is where prior neural-physics attempts burn capacity.

2. What each measured lesson becomes at scale

Lesson (measured)	Neural-engine consequence
Frozen bases fine OOD, iteration budget wasn't	Learn the iteration policy too: a tiny controller reads constraint residuals and decides iterations per frame (or use residual-threshold stopping). Never ship fixed iteration counts.
Latent-freezing ≡ fewer iterations, but slower	Don't cache cheap things. Profile before compressing; compress the global step first — it dominated.
Reduced global basis regularized (3× more accurate)	Nested/hierarchical latents double as LOD and stabilizers. Train coarse-to-fine.
Random-poke training fixed interaction OOD	Data curriculum must include forcing diversity from day one: random impulses, contacts, boundary changes — not just free dynamics.
Inversion broke closed-form polar	Bake guards into the architecture: strain clamps, SVD-safe fallbacks, contact impulse caps. Guards are part of the physics, not patches.
False-alarm NaN in my own test	Every component ships with a headless invariant suite (finiteness, momentum, energy bounds, penetration) — test-the-test included.

3. Concrete architecture

Level 1 — Local (the tied projector). MLP or tiny SIREN, 2–4 layers, ~1–10k params total, shared across all elements. Input: co-rotated deformation/state + material token. Output: latent z (k≈4–8 for 3D symmetric strain space of dim 6) or energy scalar. This replaces: PCA basis, PBF kernel constraint, contact penalty rule.

Level 2 — Global (reconciliation). Three options, in order of trust:

1. Exact reduced solve (what we shipped): Galerkin-project the PD system onto a learned position basis. Keep this as the default — it's fast, exact within the subspace, and differentiable.
2. Learned field basis (CROM-style): replace the linear Φ with a coordinate-network decoder x(X, q) mapping material points + latent q to positions — mesh-free, resolution-independent, handles large rotation better than linear POD.
3. Learned iterator: a GNN that amortizes the local/global loop (MeshGraphNets-style message passing = learned Jacobi). Highest ceiling, weakest guarantees — use it to accelerate option 1/2 (learned initial guess or preconditioner), not replace it. A learned warm-start that halves iterations is pure profit with zero correctness risk.

Level 3 — Contact & coupling. Keep acceleration-based penalties initially (proven stable). Neural upgrade: a shared contact projector taking (gap, relative velocity, material tokens of both parties) → impulse. Same tied-across-all-pairs structure. Train it against a high-quality contact solver's output.

Fluid unification. PBF's density constraint is already a tied local projector with an analytic kernel. Neural version: learn the kernel/scorr correction as a small shared network over (distance, local density) — known to fix PBF's clumping artifacts. Fluid and solid then differ only by material token and neighbor topology (dynamic grid vs fixed mesh).

4. Training strategy

Differentiable PD as the substrate. The whole pipeline — polar, projection, folded target, reduced solve — is differentiable (SVD/polar needs care near degenerate singular values; use the co-rotated formulation to avoid them). Backprop through K rollout steps.

Losses, in priority order:

1. Trajectory matching vs a ground-truth solver (yours: corotated FEM / SPFlow / Warp / Taichi-MPM), on positions AND velocities — velocity error is what caught our over-damping bug.
2. Constraint residual (density error, strain outside admissible set) — physics-informed, no labels needed.
3. Long-horizon stability: rollout loss over 100+ steps with pushforward-style noise injection (the GNS trick) so the model learns to correct its own drift.
4. Energy behavior: penalize energy gain hard, drift softly. Our over-damping and over-energetic bugs were both energy-signature bugs — make the training loop watch it automatically.

Data generation on your stack: 2D corpus on the 3060 (thousands of trajectories, minutes each — everything in this project ran on CPU in seconds, so 2D data is nearly free). 3D corpus on Modal H100 with Warp or Taichi (both give you differentiable ground truth). Curriculum: single material free dynamics → random forcing → contact pairs → fluid → coupled scenes.

5. Scaling steps, in order

1. 3D solids first. Tets, 9-dim deformation gradient, 6-dim symmetric co-rotated strain. Müller-style cached rotation extraction replaces closed-form polar. This is where the tied-basis payoff grows: 6D strain compressed to k=3–4, shared across 100k+ tets.
2. Nonlinear tied projector replacing PCA, trained on the 3D corpus. First experiment: does one projector trained on 5 materials (via tokens) match 5 separate PCAs? If yes, the embedding-table thesis scales.
3. CROM-style global decoder replacing linear POD — removes the linear-basis rotation limitation and the scenario-specific ROM weakness (train across many scenes; latent becomes a scene-conditioned field).
4. Learned warm-start / preconditioner for the solve — attack the measured bottleneck with the safest neural insert.
5. Unified constraint zoo: strain + density + volume + contact as tokens through one projector. This is the "engine" milestone: one network, all phenomena.
6. GPU port. The per-element loop is embarrassingly parallel; WebGPU compute for demos, CUDA/Warp for training. RHS scatter is the only sync point — segmented reduction, solved problem.
7. LOD via nested latents: we already showed nested POD columns give a live quality dial (r=4…24). Train the neural bases nested (ordered dropout on latent dims) and you get continuous LOD for free.

6. Where this sits in the literature (and the actual gap)

Adjacent work: CROM (continuous reduced-order fields), Neural/differentiable Projective Dynamics, subspace neural physics (Holden et al. — learned dynamics inside a subspace), GNS/MeshGraphNets (learned simulators, no structure guarantees), NCLaw (learned constitutive models inside a classical solver — closest in spirit).

The combination that appears genuinely under-explored: weight-tied per-element constraint projectors in co-rotated local frames, shared across constraint types via tokens, inside an exact reduced global solve. NCLaw ties constitutive law across elements but doesn't do the latent-projection/PD framing or multi-physics tokens; GNS learns everything and guarantees nothing; CROM reduces globally but keeps local physics analytic. The middle ground — learn only the local projection, keep the solver exact — is your lane, and this project is already its 2D proof of concept.

7. Risk register (each one already bitten us once)

• Distribution shift in global latents → CROM-style multi-scene training + online basis enrichment; keep full-solve fallback path in the engine.
• Solver-budget OOD → residual-based stopping, always.
• Inversion / degenerate elements → analytic guards inside the architecture; never let the network see raw F without co-rotation.
• Energy pumping from learned components → energy-based projector formulation (learn E, take gradients) makes passivity provable rather than hoped.
• Contact stiffness tuning hell → learn the contact projector early; it was the most hand-tuned part of everything we built.
• Silent metric lies → phase drift inflated our errors 5×; use amplitude/period/energy diagnostics alongside L2, and test the tests.

8. Suggested 8-week sequence (your hardware)

• W1–2: 3D tet PD engine (Warp or Taichi on 3060), Müller rotation extraction, headless invariant suite from day one.
• W3: 3D data corpus + linear tied basis (PCA) baseline — replicate every 2D result in 3D. This is also a strong standalone Substack post.
• W4–5: neural tied projector with material tokens (Modal H100 training runs, models are tiny so runs are cheap); the 5-materials-1-network experiment.
• W6: learned warm-start for the global solve; measure iterations saved at fixed accuracy.
• W7: PBF-3D + fluid token through the same projector; coupled dam-break-on-soft-body scene as the flagship demo.
• W8: WebGPU interactive demo + writeup. The demo is the paper figure.

Honest overall assessment: the local-projector thesis has strong evidence behind it now; the global-reduction thesis is proven in-distribution and has a known, literature-supported fix (CROM) for its OOD weakness; the fully-learned-solver path is the speculative tail — sequence it last, and let the exact solver remain the spine of the engine throughout.