README.md

---
license: mit
library_name: pytorch
tags:
- neural-operator
- pde-solver
- physics-informed
- computational-mechanics
- elasticity
- linear-attention
- transolver
- stress-prediction
metrics:
- relative-l2
model-index:
- name: transolver-linearno-elasticity-eqreg
  results:
  - task:
      type: operator-learning
      name: Elastic von Mises stress-field prediction (Geo-FNO Elasticity)
    dataset:
      type: geo-fno-elasticity
      name: Geo-FNO Elasticity (Li et al., 2022)
    metrics:
    - type: relative-l2
      value: 0.00597
      name: Test relative L2 (shipped checkpoint, mean over 200 meshes)
    - type: r-squared
      value: 0.99986
      name: Pooled R^2 vs FEM ground truth
---

# Equilibrium-Regularized LinearNO — Elastic Stress Surrogate

A neural-operator surrogate that predicts the per-node **von Mises stress field** of a hyper-elastic
unit cell with an arbitrary central void, on the **Geo-FNO Elasticity** benchmark. It maps a 2-D
unstructured mesh (972 nodes) directly to the stress field in **milliseconds on CPU**, replacing a
per-geometry finite-element (FEM) solve for fast design-space exploration.

Verified node-by-node against FEM: pooled **R² = 0.99986**, test relative-L2 **0.00597** on the shipped
checkpoint. **Interactive demo:** https://huggingface.co/spaces/Efradeca/elastic-stress-surrogate

> **At a glance.** The deployed value is *accuracy + speed*: **R² = 0.99986** vs FEM, milliseconds on CPU.
> The equilibrium regularizer is a physical-consistency component with a well-characterized
> accuracy/consistency **trade-off** (see *Evaluation*).

## Model Details

### Model Description

The model is a **Transolver-family transformer operator** whose attention block is the asymmetric
**LinearNO** linear-attention operator. It predicts the full symmetric Cauchy stress tensor
(σ_xx, σ_yy, σ_xy) per node; the supervised target is the derived scalar von Mises stress. Its
distinguishing component is a soft **equilibrium-residual regularizer** — the training loss penalizes the
discrete divergence ‖∇·σ‖² so the predicted field approaches static mechanical equilibrium (∇·σ ≈ 0),
with the divergence discretized by a **meshfree moving-least-squares (MLS)** operator that needs no mesh
connectivity.

- **Developed by:** Efradeca
- **Model type:** Neural operator (transformer PDE surrogate) for 2-D static hyper-elastic stress
- **Inputs / outputs:** node coordinates `(B, 972, 2)` → per-node von Mises stress `(B, 972)`
- **Parameters:** 582,275 (the 3-channel equilibrium model; sized to match the official LinearNO
  Elasticity configuration, ≈585k, and ≤ our Transolver baseline's 713,665)
- **License:** MIT
- **Built on (not original to this work):** the **Transolver** solver (Wu et al., ICML 2024, MIT — its
  attention block is reused verbatim) and an independent, from-equations reimplementation of the
  **LinearNO** attention block (Hu et al., AAAI 2026).

### Novelty and prior work

The soft equilibrium-residual penalty `λ‖∇·σ‖²` on a predicted stress field is **established prior art**
(PI-UNet 2022; PiFNO 2024; and especially **P-DivGNN 2025**, which uses essentially the same
soft-divergence loss and the same three-component Cauchy-stress target on an unstructured mesh — a
periodic micro-RVE). We do **not** claim the idea. This
repository's actual contribution is (1) a specific, previously-unattested **instantiation** — the penalty
on a *transformer* operator with a *meshfree, connectivity-free* MLS divergence (exact on affine fields by
the degree-1 GMLS reproduction property), distinct from the FE / FFT / finite-difference operators of all
prior work — and (2) a **cross-operator analysis** that quantifies when that penalty actually helps.

### Model Sources

- **Reference papers:** Transolver (arXiv:2402.02366, MIT); LinearNO (arXiv:2511.06294); Geo-FNO dataset
  (arXiv:2207.05209, MIT); closest prior work P-DivGNN (arXiv:2507.05291).
- **Interactive demo (Space):** https://huggingface.co/spaces/Efradeca/elastic-stress-surrogate
- **Full technical report + committed evidence:** the project repository (`docs/PAPER.md`,
  `docs/RESULTS.md`, and the `results/` run logs).

## Uses

### Direct Use

Millisecond-scale, CPU prediction of the von Mises stress field of a hyper-elastic unit cell with a
central void — for **design-space exploration / void-shape optimization**: screen many candidate
geometries with the surrogate, then verify only the best few with FEM.

### Downstream Use

A starting point for fine-tuning to related geometries, materials, or boundary conditions — which
**requires retraining** on the corresponding FEM data.

### Out-of-Scope Use

This is a **narrow surrogate, not a general stress solver**. Without retraining it is **not** valid for
other materials (e.g. steel), other boundary conditions, or geometries outside the training distribution,
and it **must not** be used as a certified solver for safety-critical decisions.

## Bias, Risks, and Limitations

- **Narrow domain.** One benchmark, one hyper-elastic (rubber-like Rivlin–Saunders) material, one geometry
  family (unit cell, central void, radius 0.2–0.4, clamped bottom, tensile top). Not a foundation model.
- **Units.** Stress is in **consistent non-dimensional benchmark units, not MPa** (traction = 100,
  C₁ = 1.86×10⁵); a physical-units result requires re-scaling to the target material.
- **Run-to-run variance.** The training set is small (1000 samples), so per-seed results vary noticeably.
  We report median + mean ± std with the **full per-seed distribution** and do not hide outliers. The
  shipped checkpoint is the **best of 3** eqreg seeds (0.00597 < 0.00653 < 0.00754).
- **Regularizer scope.** The supervised target is scalar von Mises; the tensor components are *latent*,
  shaped by a *discrete* ∇·σ penalty (a physical-consistency prior, not exact continuous momentum balance).
- **The physics benefit is a trade-off, and partly operator-specific.** See *Evaluation → Findings*: the
  large residual reduction is mostly measured by the model's own operator, it costs ~+30% accuracy, and it
  nearly vanishes when the stress tensor is directly supervised.
- **Out-of-distribution.** Accuracy degrades on geometries far from the training distribution (extreme
  voids). The regularizer keeps *its own MLS* residual low OOD, but — per the cross-operator analysis —
  that consistency gain is itself largely operator-specific, and it does **not** improve OOD accuracy.

### Recommendations

Use within the training distribution; verify any safety-critical prediction with FEM; treat the
equilibrium residual ‖∇·σ‖² as a physical-consistency indicator, not a correctness guarantee.

## How to Get Started with the Model

```python
from huggingface_hub import snapshot_download
import sys

repo = snapshot_download("Efradeca/transolver-linearno-elasticity")
sys.path.insert(0, repo)                              # model.py is bundled with the repo
from model import load_checkpoint, predict_stress

model, normalizer, info = load_checkpoint(f"{repo}/model.safetensors", device="cpu")

# coords: (N, 2) node coordinates of a unit cell with a central void
stress = predict_stress(model, coords, normalizer, info)   # (N,) von Mises stress, CPU, < 50 ms
```

## Training Details

### Training Data

**Geo-FNO Elasticity** (Li et al., 2022, arXiv:2207.05209, MIT): FEM simulations of an incompressible
Rivlin–Saunders hyper-elastic (`C₁ = 1.863×10⁵`, `C₂ = 9.79×10³`) `[0,1]²` unit cell with a random central
void (radius 0.2–0.4), 972 nodes per sample; target = per-node von Mises stress. Split: **1000 train /
200 test** (first-1000 / last-200 of 2000, the upstream Transolver protocol).

**Fixed boundary conditions (identical for every sample):** the bottom edge is **clamped** (zero
displacement) and a uniform **tensile traction `t = [0, 100]`** acts on the top edge (`+y`); left/right
edges are free. The model's only input is the 972 node coordinates — loads, supports, and material are
**not** inputs (they are constant; only the void geometry varies).

### Training Procedure

- **Preprocessing:** global z-score normalization of the target (de-normalized before the metric). The
  regularized model outputs 3 stress-tensor channels and derives von Mises.
- **Hyperparameters:** 8 layers, hidden 128, 8 heads, `dim_head` 16, `M = 64` slices; `shared_qk` LinearNO
  attention, 3-channel output. AdamW (lr 1e-3, weight decay 1e-5), cosine annealing, **500 epochs**, batch
  size 1, gradient clipping 0.1.
- **Loss:** relative-L2 (on von Mises) + `λ·‖∇·σ‖²` on interior nodes, **λ = 0.01**, MLS divergence with
  `k = 12` neighbors.
- **Compute:** single NVIDIA A10 GPU (Modal); total project compute ≈ $90. CPU inference for the demo.

## Evaluation

### Testing Data, Factors & Metrics

- **Testing data:** the 200 held-out FEM meshes.
- **Factors:** in-distribution vs. out-of-distribution (geometry-stratified by void size).
- **Metrics:** relative-L2 (primary); the discrete equilibrium residual ‖∇·σ‖²; pooled Pearson r and R²
  vs. FEM; peak-stress relative error.

### Results (Geo-FNO Elasticity test relative L2)

We report **median** (robust to this dataset's high run-to-run variance) alongside mean ± std and seed
count `n`. Accuracy-comparison rows use `torch.compile`; the equilibrium-regularized row is eager. Full
committed per-seed distributions are in the repository (`docs/RESULTS.md`, `results/`).

| Model | median | mean ± std | n | params |
|---|---|---|---|---|
| Published Transolver (Wu et al. 2024) | — | 0.0064 | — | ~0.7M |
| Published LinearNO (Hu et al. 2026, M=64) | — | 0.0050 | — | — |
| Transolver baseline (this work) | 0.00766 | 0.00827 ± 0.00222 | 6 | 713,665 |
| LinearNO, M=64 (this work) | **0.00606** | 0.00645 ± 0.00173 | 8 | 713,089 |
| LinearNO, M=256 (this work) | 0.00635 | 0.00622 ± 0.00071 | 3 | 765,313 |
| **+ equilibrium regularizer (this model)** | 0.00653 | 0.00668 ± 0.00065 | 3 | 582,275 |

**Accuracy: comparable to the baseline.** In this sample LinearNO's median (0.00606) is lower than our
reproduced baseline's (0.00766) at fewer parameters, and it reaches the published ~0.0050 level on good
seeds. Both models show **high run-to-run variance** (`torch.compile` non-determinism on 1000 samples),
with the baseline's higher mean driven by 2 outlier seeds; an attempted mechanistic explanation
(slice-attention instability) was tested and **not supported** by the evidence. LinearNO is therefore best
read as **statistically comparable** to the baseline at ≤ its parameters, both reaching ~0.0064 on good
seeds, without claiming an accuracy improvement. As a cross-check, we re-evaluated the authors' released
official checkpoint on our data with their code and reproduced **0.005007**, confirming the dataset, split,
and metric are identical, so the table is mutually comparable.

**Verification against FEM ground truth** (200 test meshes; shipped checkpoint = best of 3 eqreg seeds):
per-sample relative-L2 mean **0.00597** (median 0.00533), pooled Pearson **r = 0.99993**, **R² = 0.99986**,
peak-stress relative error **0.27%** — reproducible via `scripts/verify_model.py` → `results/verify_model.json`.

### Research findings

**1. The meshfree MLS operator is rigorously grounded.** It is a degree-1 GMLS derivative: *exact on
affine fields* (~1e-14 in float64) and *O(h)-convergent* on smooth fields (fitted slope 1.01), validated
on connectivity-free point clouds — no mesh required.

![MLS operator O(h) validation](fig5_mls_convergence.png)

**2. The headline residual reduction is largely operator-specific ("gaming").** On a clean,
same-architecture, epoch-matched ablation the regularizer cuts the residual **~325× as measured by the
model's own (MLS) operator**, but an **independent FE operator** sees only **~6–53×** — at a **measured
accuracy cost** (test rel-L2 0.00479 → 0.00621, +30%). A model-free control on exactly-divergence-free
analytic fields confirms the FE operator is *comparable* to MLS off the void-rim slivers (the large raw
gap is a sliver artifact), so the operator-agnostic reduction (~53×) still sits **an order of
magnitude below** the trained-operator 325×. The gaming reproduces on the *literal* official LinearNO
block (gap ~25×), so it is a property of the soft-penalty objective, not our implementation.

![cross-operator gaming](fig6_cross_operator.png)

**3. On a supervised tensor the effect nearly vanishes.** On a second benchmark, **Tensile2d** (PLAID),
where the full Cauchy tensor is *directly supervised*, the gaming gap collapses to **~1.3×** and the
regularizer becomes nearly redundant — the supervised data already pins the field near its
discrete-equilibrium floor. (This Tensile2d transfer is a **single-seed indicative** result and is **not**
comparable to the PLAID Tensile2d leaderboard, which scores a different RRMSE metric on a withheld test.)

**4. Out-of-distribution: a reported negative result.** The regularizer does **not** improve OOD accuracy
(degradation +51.5% vs. LinearNO's +52.8%; the plain baseline degrades least, +45.1%). It keeps *its own*
equilibrium residual low OOD (+7%), but on an orthogonal field-smoothness proxy it is indistinguishable
from the unregularized models.

**Takeaway:** the soft equilibrium penalty looks most impressive exactly where it is least physical (an
unsupervised latent tensor), and its benefit largely disappears under direct tensor supervision. The
deployed model's value is its accuracy (R² = 0.99986) and millisecond CPU speed.

## Technical Specifications

### Model Architecture and Objective

Transolver-family transformer (coordinate encoder → 8 pre-norm blocks → linear decoder head) with the
LinearNO asymmetric linear-attention block `φ(Q)·(ψᵀ(K)·V)` (φ softmax over the M slices, ψ over the N
points). Objective = relative-L2 data loss on von Mises + λ·‖∇·σ‖² equilibrium penalty (meshfree MLS
divergence). Output: 3 latent Cauchy-stress channels → von Mises.

### Compute Infrastructure

- **Hardware:** single NVIDIA A10 GPU (Modal) for training; CPU for demo inference.
- **Software:** PyTorch, einops, safetensors.

## Licenses & Attribution

This model, its weights, and the bundled architecture code are **MIT**-licensed. It builds on and uses
data from the following sources — please honor their licenses:

- **Transolver** (Wu et al., ICML 2024) — attention block reused verbatim, **MIT**.
- **Geo-FNO Elasticity** dataset (Li et al., 2022) — the training/evaluation data, **MIT**.
- **Tensile2d** — the second-benchmark data ([`PLAID-datasets/Tensile2d`](https://huggingface.co/datasets/PLAID-datasets/Tensile2d),
  Casenave et al.), licensed **CC-BY-SA-4.0**. Any derived Tensile2d fields we redistribute (e.g. the demo
  data in the companion Space) are shared under the **same CC-BY-SA-4.0 (share-alike)** with attribution.
  The MIT license of this model applies to the code and weights, **not** to the CC-BY-SA-4.0 Tensile2d data.

## Citation

If you use this model, please cite the underlying works.

```bibtex
@inproceedings{wu2024transolver,
  title={Transolver: A Fast Transformer Solver for PDEs on General Geometries},
  author={Wu, Haixu and Luo, Huakun and Wang, Haowen and Wang, Jianmin and Long, Mingsheng},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2024}
}
@inproceedings{hu2026linearno,
  title={Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attention},
  author={Hu and Liu and Qiao and Sun and Dou},
  booktitle={AAAI Conference on Artificial Intelligence},
  year={2026}
}
@article{li2022geofno,
  title={Fourier Neural Operator with Learned Deformations for PDEs on General Geometries},
  author={Li, Zongyi and others},
  journal={arXiv:2207.05209},
  year={2022}
}
@article{guevaragarban2025pdivgnn,
  title={Physics-Informed Graph Neural Networks to Reconstruct Local Fields Considering Finite Strain Hyperelasticity},
  author={Guevara Garban, Manuel R. and Chemisky, Yves and others},
  journal={arXiv:2507.05291},
  year={2025}
}
```

## Model Card Authors & Contact

Efradeca — https://huggingface.co/Efradeca