Upload folder using huggingface_hub
Browse files- .gitattributes +2 -0
- README.md +188 -112
- __pycache__/model.cpython-312.pyc +0 -0
- fig5_mls_convergence.png +3 -0
- fig6_cross_operator.png +3 -0
- stress_operator/data/tensile2d.py +137 -0
- stress_operator/losses/equilibrium.py +55 -3
- stress_operator/losses/fe_divergence.py +80 -0
- stress_operator/models/linear_no.py +19 -3
- stress_operator/models/transolver.py +5 -0
- stress_operator/train_tensile.py +144 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
fig5_mls_convergence.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
fig6_cross_operator.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -5,124 +5,166 @@ tags:
|
|
| 5 |
- neural-operator
|
| 6 |
- pde-solver
|
| 7 |
- physics-informed
|
|
|
|
| 8 |
- elasticity
|
| 9 |
- linear-attention
|
| 10 |
- transolver
|
|
|
|
| 11 |
metrics:
|
| 12 |
- relative-l2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
-
#
|
| 16 |
|
| 17 |
-
A neural-operator surrogate that predicts the per-node von Mises stress field of a hyper-elastic
|
| 18 |
-
unit cell with
|
| 19 |
-
mesh (972 nodes) directly to the stress field in milliseconds on CPU, replacing a
|
| 20 |
-
finite-element (FEM) solve for fast design-space exploration.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## Model Details
|
| 23 |
|
| 24 |
### Model Description
|
| 25 |
|
| 26 |
-
The model is a Transolver-family transformer operator whose attention block is the asymmetric
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
|
|
|
|
|
|
| 31 |
|
| 32 |
- **Developed by:** Efradeca
|
| 33 |
- **Model type:** Neural operator (transformer PDE surrogate) for 2-D static hyper-elastic stress
|
| 34 |
-
- **Inputs / outputs:** node coordinates (B, 972, 2) → per-node von Mises stress (B, 972)
|
|
|
|
| 35 |
- **License:** MIT
|
| 36 |
-
- **Built on (not original to this work):** the **Transolver** solver (Wu et al., ICML 2024
|
| 37 |
-
|
| 38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
### Model Sources
|
| 41 |
|
| 42 |
-
- **
|
| 43 |
-
-
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
## Uses
|
| 46 |
|
| 47 |
### Direct Use
|
| 48 |
|
| 49 |
Millisecond-scale, CPU prediction of the von Mises stress field of a hyper-elastic unit cell with a
|
| 50 |
-
central void
|
| 51 |
-
geometries, then verify the
|
| 52 |
|
| 53 |
### Downstream Use
|
| 54 |
|
| 55 |
-
A starting point for fine-tuning to related geometries, materials, or boundary conditions
|
| 56 |
-
requires retraining on the corresponding FEM data.
|
| 57 |
|
| 58 |
### Out-of-Scope Use
|
| 59 |
|
| 60 |
-
This is a narrow surrogate, not a general stress solver.
|
| 61 |
-
|
| 62 |
-
|
| 63 |
|
| 64 |
## Bias, Risks, and Limitations
|
| 65 |
|
| 66 |
-
- **Narrow domain
|
| 67 |
-
family (unit cell, central void, radius 0.2–0.4, clamped bottom, tensile top).
|
| 68 |
-
- **
|
| 69 |
-
|
| 70 |
-
- **
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
physical-consistency prior, not exact continuous momentum balance.
|
| 75 |
-
- **
|
| 76 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
### Recommendations
|
| 79 |
|
| 80 |
-
Use within the training distribution; verify any safety-critical prediction with FEM;
|
| 81 |
-
equilibrium residual ‖∇·σ‖² as a physical-consistency indicator.
|
| 82 |
|
| 83 |
## How to Get Started with the Model
|
| 84 |
|
| 85 |
```python
|
| 86 |
from huggingface_hub import snapshot_download
|
| 87 |
-
|
| 88 |
|
| 89 |
repo = snapshot_download("Efradeca/transolver-linearno-elasticity")
|
|
|
|
|
|
|
|
|
|
| 90 |
model, normalizer, info = load_checkpoint(f"{repo}/model.safetensors", device="cpu")
|
| 91 |
|
| 92 |
# coords: (N, 2) node coordinates of a unit cell with a central void
|
| 93 |
-
stress = predict_stress(model, coords, normalizer, info) # (N,) von Mises stress
|
| 94 |
```
|
| 95 |
|
| 96 |
## Training Details
|
| 97 |
|
| 98 |
### Training Data
|
| 99 |
|
| 100 |
-
Geo-FNO Elasticity (Li et al., 2022, arXiv:2207.05209): FEM simulations of an incompressible
|
| 101 |
-
hyper-elastic (
|
| 102 |
-
void (radius 0.2–0.4), 972 nodes per sample; target = per-node von Mises stress. Split: 1000 train /
|
| 103 |
-
200 test (first-1000 / last-200 of 2000, upstream Transolver protocol).
|
| 104 |
|
| 105 |
-
**Fixed boundary conditions (identical for every sample):** the bottom edge is **clamped**
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
dataset; only the void geometry varies).
|
| 110 |
|
| 111 |
### Training Procedure
|
| 112 |
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
- **
|
| 121 |
-
attention `linearno` (variant `shared_qk`, project_out=`False`)
|
| 122 |
-
- **Optimization:** AdamW, lr 1e-3, weight decay 1e-5, cosine annealing, 500 epochs, batch size 1,
|
| 123 |
-
gradient clipping 0.1
|
| 124 |
-
- **Loss:** relative L2 + λ·‖∇·σ‖² on interior nodes (λ = 0.01)
|
| 125 |
-
- **Seeds:** {0, 1, 2} (mean ± std reported)
|
| 126 |
|
| 127 |
## Evaluation
|
| 128 |
|
|
@@ -130,70 +172,102 @@ equilibrium-regularized model outputs three stress-tensor channels and derives v
|
|
| 130 |
|
| 131 |
- **Testing data:** the 200 held-out FEM meshes.
|
| 132 |
- **Factors:** in-distribution vs. out-of-distribution (geometry-stratified by void size).
|
| 133 |
-
- **Metrics:** relative
|
| 134 |
-
|
| 135 |
|
| 136 |
-
### Results
|
| 137 |
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
distributions are in the repository (`docs/RESULTS.md`, `results/
|
| 141 |
-
accuracy-comparison rows (baseline, LinearNO M=64/M=256) use `torch.compile`; the
|
| 142 |
-
equilibrium-regularized row is eager (`compile:false`). The baseline reproduces the published 0.0064
|
| 143 |
-
on its good seeds.
|
| 144 |
|
| 145 |
| Model | median | mean ± std | n | params |
|
| 146 |
|---|---|---|---|---|
|
| 147 |
| Published Transolver (Wu et al. 2024) | — | 0.0064 | — | ~0.7M |
|
| 148 |
| Published LinearNO (Hu et al. 2026, M=64) | — | 0.0050 | — | — |
|
| 149 |
-
| Transolver baseline (this work) | 0.00766 | 0.00827 ± 0.
|
| 150 |
-
| LinearNO, M=64 (this work) | **0.00606** | 0.00645 ± 0.
|
| 151 |
-
| LinearNO, M=256 (this work) | 0.00635 | 0.00622 ± 0.
|
| 152 |
-
| **+ equilibrium regularizer (this model)** | 0.00653 | 0.00668 ± 0.
|
| 153 |
-
|
| 154 |
-
**
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
**
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
**
|
| 171 |
-
(
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 181 |
|
| 182 |
## Technical Specifications
|
| 183 |
|
| 184 |
### Model Architecture and Objective
|
| 185 |
|
| 186 |
-
Transolver-family transformer (encoder → 8 pre-norm blocks → linear decoder head) with the
|
| 187 |
-
asymmetric linear-attention block
|
|
|
|
|
|
|
| 188 |
|
| 189 |
### Compute Infrastructure
|
| 190 |
|
| 191 |
-
- **Hardware:** single NVIDIA A10 GPU (Modal).
|
| 192 |
-
- **Software:** PyTorch, einops, safetensors
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 193 |
|
| 194 |
## Citation
|
| 195 |
|
| 196 |
-
|
|
|
|
| 197 |
```bibtex
|
| 198 |
@inproceedings{wu2024transolver,
|
| 199 |
title={Transolver: A Fast Transformer Solver for PDEs on General Geometries},
|
|
@@ -213,12 +287,14 @@ asymmetric linear-attention block; objective = relative-L2 data loss + equilibri
|
|
| 213 |
journal={arXiv:2207.05209},
|
| 214 |
year={2022}
|
| 215 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 216 |
```
|
| 217 |
|
| 218 |
-
## Model Card Authors
|
| 219 |
-
|
| 220 |
-
Efradeca.
|
| 221 |
-
|
| 222 |
-
## Model Card Contact
|
| 223 |
|
| 224 |
-
https://huggingface.co/Efradeca
|
|
|
|
| 5 |
- neural-operator
|
| 6 |
- pde-solver
|
| 7 |
- physics-informed
|
| 8 |
+
- computational-mechanics
|
| 9 |
- elasticity
|
| 10 |
- linear-attention
|
| 11 |
- transolver
|
| 12 |
+
- stress-prediction
|
| 13 |
metrics:
|
| 14 |
- relative-l2
|
| 15 |
+
model-index:
|
| 16 |
+
- name: transolver-linearno-elasticity-eqreg
|
| 17 |
+
results:
|
| 18 |
+
- task:
|
| 19 |
+
type: operator-learning
|
| 20 |
+
name: Elastic von Mises stress-field prediction (Geo-FNO Elasticity)
|
| 21 |
+
dataset:
|
| 22 |
+
type: geo-fno-elasticity
|
| 23 |
+
name: Geo-FNO Elasticity (Li et al., 2022)
|
| 24 |
+
metrics:
|
| 25 |
+
- type: relative-l2
|
| 26 |
+
value: 0.00597
|
| 27 |
+
name: Test relative L2 (shipped checkpoint, mean over 200 meshes)
|
| 28 |
+
- type: r-squared
|
| 29 |
+
value: 0.99986
|
| 30 |
+
name: Pooled R^2 vs FEM ground truth
|
| 31 |
---
|
| 32 |
|
| 33 |
+
# Equilibrium-Regularized LinearNO — Elastic Stress Surrogate
|
| 34 |
|
| 35 |
+
A neural-operator surrogate that predicts the per-node **von Mises stress field** of a hyper-elastic
|
| 36 |
+
unit cell with an arbitrary central void, on the **Geo-FNO Elasticity** benchmark. It maps a 2-D
|
| 37 |
+
unstructured mesh (972 nodes) directly to the stress field in **milliseconds on CPU**, replacing a
|
| 38 |
+
per-geometry finite-element (FEM) solve for fast design-space exploration.
|
| 39 |
+
|
| 40 |
+
Verified node-by-node against FEM: pooled **R² = 0.99986**, test relative-L2 **0.00597** on the shipped
|
| 41 |
+
checkpoint. **Interactive demo:** https://huggingface.co/spaces/Efradeca/elastic-stress-surrogate
|
| 42 |
+
|
| 43 |
+
> **Honesty note.** This card reports the model's limitations as prominently as its strengths. The
|
| 44 |
+
> deployed value is *accuracy + speed*. The equilibrium regularizer is a well-characterized **trade-off**
|
| 45 |
+
> (a prior-art idea in a new meshfree instantiation), not a free win — see *Evaluation* and *Findings*.
|
| 46 |
|
| 47 |
## Model Details
|
| 48 |
|
| 49 |
### Model Description
|
| 50 |
|
| 51 |
+
The model is a **Transolver-family transformer operator** whose attention block is the asymmetric
|
| 52 |
+
**LinearNO** linear-attention operator. It predicts the full symmetric Cauchy stress tensor
|
| 53 |
+
(σ_xx, σ_yy, σ_xy) per node; the supervised target is the derived scalar von Mises stress. Its
|
| 54 |
+
distinguishing component is a soft **equilibrium-residual regularizer** — the training loss penalizes the
|
| 55 |
+
discrete divergence ‖∇·σ‖² so the predicted field approaches static mechanical equilibrium (∇·σ ≈ 0),
|
| 56 |
+
with the divergence discretized by a **meshfree moving-least-squares (MLS)** operator that needs no mesh
|
| 57 |
+
connectivity.
|
| 58 |
|
| 59 |
- **Developed by:** Efradeca
|
| 60 |
- **Model type:** Neural operator (transformer PDE surrogate) for 2-D static hyper-elastic stress
|
| 61 |
+
- **Inputs / outputs:** node coordinates `(B, 972, 2)` → per-node von Mises stress `(B, 972)`
|
| 62 |
+
- **Parameters:** 582,275 (parameter-matched to the official LinearNO Elasticity configuration)
|
| 63 |
- **License:** MIT
|
| 64 |
+
- **Built on (not original to this work):** the **Transolver** solver (Wu et al., ICML 2024, MIT — its
|
| 65 |
+
attention block is reused verbatim) and an independent, from-equations reimplementation of the
|
| 66 |
+
**LinearNO** attention block (Hu et al., AAAI 2026).
|
| 67 |
+
|
| 68 |
+
### What is and isn't novel (read this)
|
| 69 |
+
|
| 70 |
+
The soft equilibrium-residual penalty `λ‖∇·σ‖²` on a predicted stress field is **established prior art**
|
| 71 |
+
(PI-UNet 2022; PiFNO 2024; and especially **P-DivGNN 2025**, which uses the same loss and the same
|
| 72 |
+
three-component Cauchy-stress target on the same kind of holed mesh). We do **not** claim the idea. This
|
| 73 |
+
repository's actual contribution is (1) a specific, previously-unattested **instantiation** — the penalty
|
| 74 |
+
on a *transformer* operator with a *meshfree, connectivity-free* MLS divergence (exact on affine fields by
|
| 75 |
+
the degree-1 GMLS reproduction property), distinct from the FE / FFT / finite-difference operators of all
|
| 76 |
+
prior work — and (2) a **cross-operator honesty analysis** of when that penalty actually helps.
|
| 77 |
|
| 78 |
### Model Sources
|
| 79 |
|
| 80 |
+
- **Reference papers:** Transolver (arXiv:2402.02366, MIT); LinearNO (arXiv:2511.06294); Geo-FNO dataset
|
| 81 |
+
(arXiv:2207.05209, MIT); closest prior work P-DivGNN (arXiv:2507.05291).
|
| 82 |
+
- **Interactive demo (Space):** https://huggingface.co/spaces/Efradeca/elastic-stress-surrogate
|
| 83 |
+
- **Full technical report + committed evidence:** the project repository (`docs/PAPER.md`,
|
| 84 |
+
`docs/RESULTS.md`, and the `results/` run logs).
|
| 85 |
|
| 86 |
## Uses
|
| 87 |
|
| 88 |
### Direct Use
|
| 89 |
|
| 90 |
Millisecond-scale, CPU prediction of the von Mises stress field of a hyper-elastic unit cell with a
|
| 91 |
+
central void — for **design-space exploration / void-shape optimization**: screen many candidate
|
| 92 |
+
geometries with the surrogate, then verify only the best few with FEM.
|
| 93 |
|
| 94 |
### Downstream Use
|
| 95 |
|
| 96 |
+
A starting point for fine-tuning to related geometries, materials, or boundary conditions — which
|
| 97 |
+
**requires retraining** on the corresponding FEM data.
|
| 98 |
|
| 99 |
### Out-of-Scope Use
|
| 100 |
|
| 101 |
+
This is a **narrow surrogate, not a general stress solver**. Without retraining it is **not** valid for
|
| 102 |
+
other materials (e.g. steel), other boundary conditions, or geometries outside the training distribution,
|
| 103 |
+
and it **must not** be used as a certified solver for safety-critical decisions.
|
| 104 |
|
| 105 |
## Bias, Risks, and Limitations
|
| 106 |
|
| 107 |
+
- **Narrow domain.** One benchmark, one hyper-elastic (rubber-like Rivlin–Saunders) material, one geometry
|
| 108 |
+
family (unit cell, central void, radius 0.2–0.4, clamped bottom, tensile top). Not a foundation model.
|
| 109 |
+
- **Units.** Stress is in **consistent non-dimensional benchmark units, not MPa** (traction = 100,
|
| 110 |
+
C₁ = 1.86×10⁵); a physical-units result requires re-scaling to the target material.
|
| 111 |
+
- **Run-to-run variance.** The training set is small (1000 samples), so per-seed results vary noticeably.
|
| 112 |
+
We report median + mean ± std with the **full per-seed distribution** and do not hide outliers. The
|
| 113 |
+
shipped checkpoint is the **best of 3** eqreg seeds (0.00597 < 0.00653 < 0.00754).
|
| 114 |
+
- **Regularizer scope.** The supervised target is scalar von Mises; the tensor components are *latent*,
|
| 115 |
+
shaped by a *discrete* ∇·σ penalty (a physical-consistency prior, not exact continuous momentum balance).
|
| 116 |
+
- **The physics benefit is a trade-off, and partly operator-specific.** See *Evaluation → Findings*: the
|
| 117 |
+
large residual reduction is mostly measured by the model's own operator, it costs ~+30% accuracy, and it
|
| 118 |
+
nearly vanishes when the stress tensor is directly supervised.
|
| 119 |
+
- **Out-of-distribution.** Accuracy degrades on geometries far from the training distribution (extreme
|
| 120 |
+
voids). The regularizer keeps *its own MLS* residual low OOD, but — per the cross-operator analysis —
|
| 121 |
+
that consistency gain is itself largely operator-specific, and it does **not** improve OOD accuracy.
|
| 122 |
|
| 123 |
### Recommendations
|
| 124 |
|
| 125 |
+
Use within the training distribution; verify any safety-critical prediction with FEM; treat the
|
| 126 |
+
equilibrium residual ‖∇·σ‖² as a physical-consistency indicator, not a correctness guarantee.
|
| 127 |
|
| 128 |
## How to Get Started with the Model
|
| 129 |
|
| 130 |
```python
|
| 131 |
from huggingface_hub import snapshot_download
|
| 132 |
+
import sys
|
| 133 |
|
| 134 |
repo = snapshot_download("Efradeca/transolver-linearno-elasticity")
|
| 135 |
+
sys.path.insert(0, repo) # model.py is bundled with the repo
|
| 136 |
+
from model import load_checkpoint, predict_stress
|
| 137 |
+
|
| 138 |
model, normalizer, info = load_checkpoint(f"{repo}/model.safetensors", device="cpu")
|
| 139 |
|
| 140 |
# coords: (N, 2) node coordinates of a unit cell with a central void
|
| 141 |
+
stress = predict_stress(model, coords, normalizer, info) # (N,) von Mises stress, CPU, < 50 ms
|
| 142 |
```
|
| 143 |
|
| 144 |
## Training Details
|
| 145 |
|
| 146 |
### Training Data
|
| 147 |
|
| 148 |
+
**Geo-FNO Elasticity** (Li et al., 2022, arXiv:2207.05209, MIT): FEM simulations of an incompressible
|
| 149 |
+
Rivlin–Saunders hyper-elastic (`C₁ = 1.863×10⁵`, `C₂ = 9.79×10³`) `[0,1]²` unit cell with a random central
|
| 150 |
+
void (radius 0.2–0.4), 972 nodes per sample; target = per-node von Mises stress. Split: **1000 train /
|
| 151 |
+
200 test** (first-1000 / last-200 of 2000, the upstream Transolver protocol).
|
| 152 |
|
| 153 |
+
**Fixed boundary conditions (identical for every sample):** the bottom edge is **clamped** (zero
|
| 154 |
+
displacement) and a uniform **tensile traction `t = [0, 100]`** acts on the top edge (`+y`); left/right
|
| 155 |
+
edges are free. The model's only input is the 972 node coordinates — loads, supports, and material are
|
| 156 |
+
**not** inputs (they are constant; only the void geometry varies).
|
|
|
|
| 157 |
|
| 158 |
### Training Procedure
|
| 159 |
|
| 160 |
+
- **Preprocessing:** global z-score normalization of the target (de-normalized before the metric). The
|
| 161 |
+
regularized model outputs 3 stress-tensor channels and derives von Mises.
|
| 162 |
+
- **Hyperparameters:** 8 layers, hidden 128, 8 heads, `dim_head` 16, `M = 64` slices; `shared_qk` LinearNO
|
| 163 |
+
attention, 3-channel output. AdamW (lr 1e-3, weight decay 1e-5), cosine annealing, **500 epochs**, batch
|
| 164 |
+
size 1, gradient clipping 0.1.
|
| 165 |
+
- **Loss:** relative-L2 (on von Mises) + `λ·‖∇·σ‖²` on interior nodes, **λ = 0.01**, MLS divergence with
|
| 166 |
+
`k = 12` neighbors.
|
| 167 |
+
- **Compute:** single NVIDIA A10 GPU (Modal); total project compute ≈ $90. CPU inference for the demo.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 168 |
|
| 169 |
## Evaluation
|
| 170 |
|
|
|
|
| 172 |
|
| 173 |
- **Testing data:** the 200 held-out FEM meshes.
|
| 174 |
- **Factors:** in-distribution vs. out-of-distribution (geometry-stratified by void size).
|
| 175 |
+
- **Metrics:** relative-L2 (primary); the discrete equilibrium residual ‖∇·σ‖²; pooled Pearson r and R²
|
| 176 |
+
vs. FEM; peak-stress relative error.
|
| 177 |
|
| 178 |
+
### Results (Geo-FNO Elasticity test relative L2)
|
| 179 |
|
| 180 |
+
We report **median** (robust to this dataset's high run-to-run variance) alongside mean ± std and seed
|
| 181 |
+
count `n`. Accuracy-comparison rows use `torch.compile`; the equilibrium-regularized row is eager. Full
|
| 182 |
+
committed per-seed distributions are in the repository (`docs/RESULTS.md`, `results/`).
|
|
|
|
|
|
|
|
|
|
| 183 |
|
| 184 |
| Model | median | mean ± std | n | params |
|
| 185 |
|---|---|---|---|---|
|
| 186 |
| Published Transolver (Wu et al. 2024) | — | 0.0064 | — | ~0.7M |
|
| 187 |
| Published LinearNO (Hu et al. 2026, M=64) | — | 0.0050 | — | — |
|
| 188 |
+
| Transolver baseline (this work) | 0.00766 | 0.00827 ± 0.00222 | 6 | 713,665 |
|
| 189 |
+
| LinearNO, M=64 (this work) | **0.00606** | 0.00645 ± 0.00173 | 8 | 713,089 |
|
| 190 |
+
| LinearNO, M=256 (this work) | 0.00635 | 0.00622 ± 0.00071 | 3 | 765,313 |
|
| 191 |
+
| **+ equilibrium regularizer (this model)** | 0.00653 | 0.00668 ± 0.00065 | 3 | 582,275 |
|
| 192 |
+
|
| 193 |
+
**Comparable, not "beats."** In this sample LinearNO's median (0.00606) is lower than our reproduced
|
| 194 |
+
baseline's (0.00766) at fewer parameters, and it reaches the published ~0.0050 level on good seeds. But
|
| 195 |
+
both models have **high run-to-run variance** (`torch.compile` non-determinism on 1000 samples), and the
|
| 196 |
+
baseline's higher mean is driven by 2 outlier seeds. We do **not** claim LinearNO beats or is more robust
|
| 197 |
+
than the baseline; an attempted mechanistic explanation (slice-attention instability) was **empirically
|
| 198 |
+
tested and refuted**. Honest reading: LinearNO is **comparable** to the baseline at ≤ its parameters, both
|
| 199 |
+
reproducing ~0.0064 on good seeds. As a cross-check, we re-evaluated the authors' released official
|
| 200 |
+
checkpoint on our data with their code and reproduced **0.005007** — confirming the dataset, split, and
|
| 201 |
+
metric are identical, so the table is mutually comparable.
|
| 202 |
+
|
| 203 |
+
**Verification against FEM ground truth** (200 test meshes; shipped checkpoint = best of 3 eqreg seeds):
|
| 204 |
+
per-sample relative-L2 mean **0.00597** (median 0.00533), pooled Pearson **r = 0.99993**, **R² = 0.99986**,
|
| 205 |
+
peak-stress relative error **0.27%** — reproducible via `scripts/verify_model.py` → `results/verify_model.json`.
|
| 206 |
+
|
| 207 |
+
### Findings (the honest part)
|
| 208 |
+
|
| 209 |
+
**1. The meshfree MLS operator is rigorously grounded.** It is a degree-1 GMLS derivative: *exact on
|
| 210 |
+
affine fields* (~1e-14 in float64) and *O(h)-convergent* on smooth fields (fitted slope 1.01), validated
|
| 211 |
+
on connectivity-free point clouds — no mesh required.
|
| 212 |
+
|
| 213 |
+

|
| 214 |
+
|
| 215 |
+
**2. The headline residual reduction is largely operator-specific ("gaming").** On a clean,
|
| 216 |
+
same-architecture, epoch-matched ablation the regularizer cuts the residual **~325× as measured by the
|
| 217 |
+
model's own (MLS) operator**, but an **independent FE operator** sees only **~6–53×** — at a **measured
|
| 218 |
+
accuracy cost** (test rel-L2 0.00479 → 0.00621, +30%). A model-free control on exactly-divergence-free
|
| 219 |
+
analytic fields confirms the FE operator is *comparable* to MLS off the void-rim slivers (the large raw
|
| 220 |
+
gap is a sliver artifact), so the honest operator-agnostic reduction (~53×) still sits **an order of
|
| 221 |
+
magnitude below** the trained-operator 325×. The gaming reproduces on the *literal* official LinearNO
|
| 222 |
+
block (gap ~25×), so it is a property of the soft-penalty objective, not our implementation.
|
| 223 |
+
|
| 224 |
+

|
| 225 |
+
|
| 226 |
+
**3. On a supervised tensor the effect nearly vanishes.** On a second benchmark, **Tensile2d** (PLAID),
|
| 227 |
+
where the full Cauchy tensor is *directly supervised*, the gaming gap collapses to **~1.3×** and the
|
| 228 |
+
regularizer becomes nearly redundant — the supervised data already pins the field near its
|
| 229 |
+
discrete-equilibrium floor. (This Tensile2d transfer is a **single-seed indicative** result and is **not**
|
| 230 |
+
comparable to the PLAID Tensile2d leaderboard, which scores a different RRMSE metric on a withheld test.)
|
| 231 |
+
|
| 232 |
+
**4. Out-of-distribution: a reported negative result.** The regularizer does **not** improve OOD accuracy
|
| 233 |
+
(degradation +51.5% vs. LinearNO's +52.8%; the plain baseline degrades least, +45.1%). It keeps *its own*
|
| 234 |
+
equilibrium residual low OOD (+7%), but on an orthogonal field-smoothness proxy it is indistinguishable
|
| 235 |
+
from the unregularized models.
|
| 236 |
+
|
| 237 |
+
**Takeaway:** the soft equilibrium penalty looks most impressive exactly where it is least physical (an
|
| 238 |
+
unsupervised latent tensor). We report this rather than paper over it. The deployed model's value is its
|
| 239 |
+
accuracy (R² = 0.99986) and millisecond CPU speed.
|
| 240 |
|
| 241 |
## Technical Specifications
|
| 242 |
|
| 243 |
### Model Architecture and Objective
|
| 244 |
|
| 245 |
+
Transolver-family transformer (coordinate encoder → 8 pre-norm blocks → linear decoder head) with the
|
| 246 |
+
LinearNO asymmetric linear-attention block `φ(Q)·(ψᵀ(K)·V)` (φ softmax over the M slices, ψ over the N
|
| 247 |
+
points). Objective = relative-L2 data loss on von Mises + λ·‖∇·σ‖² equilibrium penalty (meshfree MLS
|
| 248 |
+
divergence). Output: 3 latent Cauchy-stress channels → von Mises.
|
| 249 |
|
| 250 |
### Compute Infrastructure
|
| 251 |
|
| 252 |
+
- **Hardware:** single NVIDIA A10 GPU (Modal) for training; CPU for demo inference.
|
| 253 |
+
- **Software:** PyTorch, einops, safetensors.
|
| 254 |
+
|
| 255 |
+
## Licenses & Attribution
|
| 256 |
+
|
| 257 |
+
This model, its weights, and the bundled architecture code are **MIT**-licensed. It builds on and uses
|
| 258 |
+
data from the following sources — please honor their licenses:
|
| 259 |
+
|
| 260 |
+
- **Transolver** (Wu et al., ICML 2024) — attention block reused verbatim, **MIT**.
|
| 261 |
+
- **Geo-FNO Elasticity** dataset (Li et al., 2022) — the training/evaluation data, **MIT**.
|
| 262 |
+
- **Tensile2d** — the second-benchmark data ([`PLAID-datasets/Tensile2d`](https://huggingface.co/datasets/PLAID-datasets/Tensile2d),
|
| 263 |
+
Casenave et al.), licensed **CC-BY-SA-4.0**. Any derived Tensile2d fields we redistribute (e.g. the demo
|
| 264 |
+
data in the companion Space) are shared under the **same CC-BY-SA-4.0 (share-alike)** with attribution.
|
| 265 |
+
The MIT license of this model applies to the code and weights, **not** to the CC-BY-SA-4.0 Tensile2d data.
|
| 266 |
|
| 267 |
## Citation
|
| 268 |
|
| 269 |
+
If you use this model, please cite the underlying works.
|
| 270 |
+
|
| 271 |
```bibtex
|
| 272 |
@inproceedings{wu2024transolver,
|
| 273 |
title={Transolver: A Fast Transformer Solver for PDEs on General Geometries},
|
|
|
|
| 287 |
journal={arXiv:2207.05209},
|
| 288 |
year={2022}
|
| 289 |
}
|
| 290 |
+
@article{guevaragarban2025pdivgnn,
|
| 291 |
+
title={Physics-Informed Graph Neural Networks to Reconstruct Local Fields Considering Finite Strain Hyperelasticity},
|
| 292 |
+
author={Guevara Garban, Manuel R. and Chemisky, Yves and others},
|
| 293 |
+
journal={arXiv:2507.05291},
|
| 294 |
+
year={2025}
|
| 295 |
+
}
|
| 296 |
```
|
| 297 |
|
| 298 |
+
## Model Card Authors & Contact
|
|
|
|
|
|
|
|
|
|
|
|
|
| 299 |
|
| 300 |
+
Efradeca — https://huggingface.co/Efradeca
|
__pycache__/model.cpython-312.pyc
CHANGED
|
Binary files a/__pycache__/model.cpython-312.pyc and b/__pycache__/model.cpython-312.pyc differ
|
|
|
fig5_mls_convergence.png
ADDED
|
Git LFS Details
|
fig6_cross_operator.png
ADDED
|
Git LFS Details
|
stress_operator/data/tensile2d.py
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Tensile2d (PLAID-datasets/Tensile2d, HF, CC-BY-SA-4.0) adapter — SECOND benchmark.
|
| 2 |
+
|
| 3 |
+
Why: unlike Geo-FNO Elasticity (scalar von Mises target -> our tensor is latent), Tensile2d directly
|
| 4 |
+
supervises the full 2D Cauchy stress tensor (sig11, sig22, sig12) per node on an irregular mesh, so our
|
| 5 |
+
equilibrium regularizer acts on a SUPERVISED tensor and we can validate the predicted tensor itself.
|
| 6 |
+
|
| 7 |
+
VERIFIED facts (golden rule, by reading real samples):
|
| 8 |
+
- Each sample is a pickled CGNS tree (no PLAID lib needed): GridCoordinates {CoordinateX, CoordinateY}
|
| 9 |
+
and PointData {sig11, sig22, sig12, U1, U2, q}; plus input scalars {P, p1..p5} and output scalars.
|
| 10 |
+
- 702 rows: samples 0-499 are the labeled `train_500` set (full stress); samples 500-701 (test 200 +
|
| 11 |
+
OOD 2) have stress fields WITHHELD (competition) -> only coords + input scalars. So we use the 500
|
| 12 |
+
labeled samples and hold out a local test split (no official test ground truth is public).
|
| 13 |
+
- Node count varies per sample (6143-11801, mean ~9400). Static, plane-strain, quasistatic.
|
| 14 |
+
|
| 15 |
+
`build_cache()` writes data/tensile2d/samples/*.npz once. `build_tensile_splits()` loads them, fits
|
| 16 |
+
z-score normalizers on the train split, and returns per-sample lists (variable N -> no stacking).
|
| 17 |
+
"""
|
| 18 |
+
from __future__ import annotations
|
| 19 |
+
|
| 20 |
+
import glob
|
| 21 |
+
import os
|
| 22 |
+
from dataclasses import dataclass
|
| 23 |
+
from typing import List
|
| 24 |
+
|
| 25 |
+
import numpy as np
|
| 26 |
+
import torch
|
| 27 |
+
|
| 28 |
+
INPUT_SCALARS = ["P", "p1", "p2", "p3", "p4", "p5"]
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def _field(tree, name):
|
| 32 |
+
stack = [tree]
|
| 33 |
+
while stack:
|
| 34 |
+
n = stack.pop()
|
| 35 |
+
if isinstance(n, list) and len(n) == 4:
|
| 36 |
+
if n[0] == name and isinstance(n[1], np.ndarray):
|
| 37 |
+
return n[1]
|
| 38 |
+
stack.extend(n[2] or [])
|
| 39 |
+
return None
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def build_cache(out_dir: str = "data/tensile2d/samples") -> int:
|
| 43 |
+
"""Download Tensile2d parquet shards and cache the 500 labeled samples as .npz. Returns count."""
|
| 44 |
+
from huggingface_hub import hf_hub_download
|
| 45 |
+
import pyarrow.parquet as pq
|
| 46 |
+
import pickle
|
| 47 |
+
|
| 48 |
+
os.makedirs(out_dir, exist_ok=True)
|
| 49 |
+
shards = [hf_hub_download("PLAID-datasets/Tensile2d",
|
| 50 |
+
f"data/all_samples-0000{i}-of-00002.parquet", repo_type="dataset")
|
| 51 |
+
for i in (0, 1)]
|
| 52 |
+
idx = 0
|
| 53 |
+
saved = 0
|
| 54 |
+
for sh in shards:
|
| 55 |
+
col = pq.read_table(sh).column("sample")
|
| 56 |
+
for i in range(len(col)):
|
| 57 |
+
try:
|
| 58 |
+
obj = pickle.loads(col[i].as_py())
|
| 59 |
+
tree = list(obj["meshes"].values())[0]
|
| 60 |
+
x, y = _field(tree, "CoordinateX"), _field(tree, "CoordinateY")
|
| 61 |
+
s11, s22, s12 = _field(tree, "sig11"), _field(tree, "sig22"), _field(tree, "sig12")
|
| 62 |
+
if any(v is None for v in (x, y, s11, s22, s12)):
|
| 63 |
+
idx += 1
|
| 64 |
+
continue # withheld test/OOD sample
|
| 65 |
+
coords = np.stack([x, y], 1).astype(np.float32)
|
| 66 |
+
sigma = np.stack([s11, s22, s12], 1).astype(np.float32)
|
| 67 |
+
vm = np.sqrt(s11 ** 2 - s11 * s22 + s22 ** 2 + 3 * s12 ** 2).astype(np.float32)
|
| 68 |
+
sc = obj.get("scalars", {})
|
| 69 |
+
scin = np.array([float(sc[k]) for k in INPUT_SCALARS], np.float32)
|
| 70 |
+
np.savez(os.path.join(out_dir, f"{saved:04d}.npz"),
|
| 71 |
+
coords=coords, sigma=sigma, vm=vm, scalars_in=scin)
|
| 72 |
+
saved += 1
|
| 73 |
+
except Exception:
|
| 74 |
+
pass
|
| 75 |
+
idx += 1
|
| 76 |
+
return saved
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
@dataclass
|
| 80 |
+
class TensileSplits:
|
| 81 |
+
train_coords: List[torch.Tensor]
|
| 82 |
+
train_sigma: List[torch.Tensor] # scaled by 1/S (N,3)
|
| 83 |
+
train_scalars: List[torch.Tensor] # z-scored (6,)
|
| 84 |
+
test_coords: List[torch.Tensor]
|
| 85 |
+
test_sigma: List[torch.Tensor]
|
| 86 |
+
test_scalars: List[torch.Tensor]
|
| 87 |
+
scale_S: float # SINGLE global stress scale; de-normalize: sigma*S
|
| 88 |
+
scalar_mean: torch.Tensor # (6,)
|
| 89 |
+
scalar_std: torch.Tensor
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
def build_tensile_splits(samples_dir: str = "data/tensile2d/samples",
|
| 93 |
+
ntrain: int = 400, ntest: int = 100, seed: int = 0) -> TensileSplits:
|
| 94 |
+
"""Load cached samples; deterministic train/test split; z-score normalizers fit on TRAIN only.
|
| 95 |
+
|
| 96 |
+
Coords are kept RAW (physical units) so the MLS divergence operator measures true distances; the
|
| 97 |
+
model conditions on raw coords + normalized input scalars (the encoder MLP handles the coord scale).
|
| 98 |
+
Stress is z-scored per component; input scalars z-scored per dim.
|
| 99 |
+
"""
|
| 100 |
+
files = sorted(glob.glob(os.path.join(samples_dir, "*.npz")))
|
| 101 |
+
if not files:
|
| 102 |
+
raise FileNotFoundError(f"no cached Tensile2d samples in {samples_dir} — run build_cache() first")
|
| 103 |
+
rng = np.random.default_rng(seed)
|
| 104 |
+
order = rng.permutation(len(files))
|
| 105 |
+
tr_idx, te_idx = order[:ntrain], order[ntrain:ntrain + ntest]
|
| 106 |
+
|
| 107 |
+
def load(i):
|
| 108 |
+
d = np.load(files[i])
|
| 109 |
+
return d["coords"], d["sigma"], d["scalars_in"]
|
| 110 |
+
|
| 111 |
+
tr = [load(i) for i in tr_idx]
|
| 112 |
+
te = [load(i) for i in te_idx]
|
| 113 |
+
|
| 114 |
+
# SINGLE global stress scale S (RMS of all train stress components) so the divergence regularizer
|
| 115 |
+
# de-normalizes with one scalar: sigma_phys = S * sigma_norm => div(sigma_phys) = S * div(sigma_norm).
|
| 116 |
+
# (Per-component z-score would scale each channel differently and corrupt the physical divergence.)
|
| 117 |
+
sig_all = np.concatenate([s for _, s, _ in tr], 0) # (sum N, 3)
|
| 118 |
+
S = float(np.sqrt((sig_all ** 2).mean())) # global RMS scale
|
| 119 |
+
sc_all = np.stack([sc for _, _, sc in tr], 0) # (ntrain, 6)
|
| 120 |
+
sc_mean, sc_std = sc_all.mean(0), sc_all.std(0) + 1e-8
|
| 121 |
+
|
| 122 |
+
def pack(rows):
|
| 123 |
+
cs, ss, scs = [], [], []
|
| 124 |
+
for c, s, sc in rows:
|
| 125 |
+
cs.append(torch.tensor(c))
|
| 126 |
+
ss.append(torch.tensor(s / S))
|
| 127 |
+
scs.append(torch.tensor((sc - sc_mean) / sc_std))
|
| 128 |
+
return cs, ss, scs
|
| 129 |
+
|
| 130 |
+
trc, trs, trsc = pack(tr)
|
| 131 |
+
tec, tes, tesc = pack(te)
|
| 132 |
+
return TensileSplits(
|
| 133 |
+
train_coords=trc, train_sigma=trs, train_scalars=trsc,
|
| 134 |
+
test_coords=tec, test_sigma=tes, test_scalars=tesc,
|
| 135 |
+
scale_S=S,
|
| 136 |
+
scalar_mean=torch.tensor(sc_mean), scalar_std=torch.tensor(sc_std),
|
| 137 |
+
)
|
stress_operator/losses/equilibrium.py
CHANGED
|
@@ -40,7 +40,7 @@ def knn_indices(coords: np.ndarray, k: int) -> Tuple[np.ndarray, np.ndarray]:
|
|
| 40 |
|
| 41 |
|
| 42 |
def build_mls_gradient_operators(
|
| 43 |
-
coords: torch.Tensor, k: int = 12, eps: float = 1e-9
|
| 44 |
) -> Tuple[torch.Tensor, torch.Tensor]:
|
| 45 |
"""Build dense (N, N) operators Gx, Gy with (Gx @ f) ~ df/dx and (Gy @ f) ~ df/dy.
|
| 46 |
|
|
@@ -67,11 +67,44 @@ def build_mls_gradient_operators(
|
|
| 67 |
Gy[i, nbr] += C[:, 1]
|
| 68 |
Gy[i, i] -= C[:, 1].sum()
|
| 69 |
return (
|
| 70 |
-
torch.tensor(Gx, dtype=
|
| 71 |
-
torch.tensor(Gy, dtype=
|
| 72 |
)
|
| 73 |
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
def build_graph_laplacian(coords: torch.Tensor, k: int = 12, eps: float = 1e-9) -> torch.Tensor:
|
| 76 |
"""Unnormalized graph Laplacian L = D - W with Gaussian kNN weights (Approach B)."""
|
| 77 |
cpu = coords.detach().cpu().numpy().astype(np.float64)
|
|
@@ -93,11 +126,30 @@ def interior_mask(coords: torch.Tensor, tol: float = 0.03) -> torch.Tensor:
|
|
| 93 |
|
| 94 |
The outer-box boundary carries the applied traction / clamp, where div(sigma)=0 need not
|
| 95 |
hold; interior nodes (including those around the void) are valid for the constraint.
|
|
|
|
|
|
|
| 96 |
"""
|
| 97 |
x, y = coords[..., 0], coords[..., 1]
|
| 98 |
return (x > tol) & (x < 1 - tol) & (y > tol) & (y < 1 - tol)
|
| 99 |
|
| 100 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
def divergence(stress3: torch.Tensor, Gx: torch.Tensor, Gy: torch.Tensor) -> torch.Tensor:
|
| 102 |
"""Divergence of the symmetric 2x2 stress tensor field.
|
| 103 |
|
|
|
|
| 40 |
|
| 41 |
|
| 42 |
def build_mls_gradient_operators(
|
| 43 |
+
coords: torch.Tensor, k: int = 12, eps: float = 1e-9, dtype: torch.dtype = torch.float32
|
| 44 |
) -> Tuple[torch.Tensor, torch.Tensor]:
|
| 45 |
"""Build dense (N, N) operators Gx, Gy with (Gx @ f) ~ df/dx and (Gy @ f) ~ df/dy.
|
| 46 |
|
|
|
|
| 67 |
Gy[i, nbr] += C[:, 1]
|
| 68 |
Gy[i, i] -= C[:, 1].sum()
|
| 69 |
return (
|
| 70 |
+
torch.tensor(Gx, dtype=dtype, device=coords.device),
|
| 71 |
+
torch.tensor(Gy, dtype=dtype, device=coords.device),
|
| 72 |
)
|
| 73 |
|
| 74 |
|
| 75 |
+
def build_mls_gradient_operators_sparse(coords: torch.Tensor, k: int = 12, eps: float = 1e-9):
|
| 76 |
+
"""Sparse (N,N) MLS gradient operators, built directly without the dense intermediate.
|
| 77 |
+
|
| 78 |
+
Identical stencils to build_mls_gradient_operators (k+1 nonzeros per row) but assembled as COO,
|
| 79 |
+
so it scales to large meshes (e.g. Tensile2d's ~9k-12k nodes, where a dense (N,N) is ~0.7 GB).
|
| 80 |
+
"""
|
| 81 |
+
cpu = coords.detach().cpu().numpy().astype(np.float64)
|
| 82 |
+
n = cpu.shape[0]
|
| 83 |
+
dist, idx = knn_indices(cpu, k)
|
| 84 |
+
sigma2 = float((dist.mean()) ** 2) + eps
|
| 85 |
+
rows = np.empty(n * (k + 1), dtype=np.int64)
|
| 86 |
+
cols = np.empty(n * (k + 1), dtype=np.int64)
|
| 87 |
+
vx = np.empty(n * (k + 1), dtype=np.float64)
|
| 88 |
+
vy = np.empty(n * (k + 1), dtype=np.float64)
|
| 89 |
+
p = 0
|
| 90 |
+
for i in range(n):
|
| 91 |
+
nbr = idx[i]
|
| 92 |
+
d = cpu[nbr] - cpu[i]
|
| 93 |
+
w = np.exp(-(d ** 2).sum(1) / sigma2)
|
| 94 |
+
A = np.einsum("k,ka,kb->ab", w, d, d)
|
| 95 |
+
C = (w[:, None] * d) @ np.linalg.pinv(A) # (k, 2)
|
| 96 |
+
m = k + 1
|
| 97 |
+
rows[p:p + m] = i
|
| 98 |
+
cols[p:p + k] = nbr; cols[p + k] = i
|
| 99 |
+
vx[p:p + k] = C[:, 0]; vx[p + k] = -C[:, 0].sum()
|
| 100 |
+
vy[p:p + k] = C[:, 1]; vy[p + k] = -C[:, 1].sum()
|
| 101 |
+
p += m
|
| 102 |
+
ij = torch.tensor(np.stack([rows, cols]), device=coords.device)
|
| 103 |
+
Gx = torch.sparse_coo_tensor(ij, torch.tensor(vx, dtype=torch.float32), (n, n)).coalesce()
|
| 104 |
+
Gy = torch.sparse_coo_tensor(ij, torch.tensor(vy, dtype=torch.float32), (n, n)).coalesce()
|
| 105 |
+
return Gx.to(coords.device), Gy.to(coords.device)
|
| 106 |
+
|
| 107 |
+
|
| 108 |
def build_graph_laplacian(coords: torch.Tensor, k: int = 12, eps: float = 1e-9) -> torch.Tensor:
|
| 109 |
"""Unnormalized graph Laplacian L = D - W with Gaussian kNN weights (Approach B)."""
|
| 110 |
cpu = coords.detach().cpu().numpy().astype(np.float64)
|
|
|
|
| 126 |
|
| 127 |
The outer-box boundary carries the applied traction / clamp, where div(sigma)=0 need not
|
| 128 |
hold; interior nodes (including those around the void) are valid for the constraint.
|
| 129 |
+
NOTE: assumes the Geo-FNO Elasticity [0,1]^2 unit cell. For arbitrary geometries (e.g. Tensile2d
|
| 130 |
+
on [-1,1]^2 with an irregular free surface) use ``interior_mask_knn`` instead.
|
| 131 |
"""
|
| 132 |
x, y = coords[..., 0], coords[..., 1]
|
| 133 |
return (x > tol) & (x < 1 - tol) & (y > tol) & (y < 1 - tol)
|
| 134 |
|
| 135 |
|
| 136 |
+
def interior_mask_knn(coords: torch.Tensor, k: int = 12, thresh: float = 0.5) -> torch.Tensor:
|
| 137 |
+
"""Geometry-agnostic, connectivity-free interior mask via k-NN stencil symmetry.
|
| 138 |
+
|
| 139 |
+
A node deep in the interior has neighbors roughly symmetric around it, so the offset of its
|
| 140 |
+
k-NN centroid is small relative to the mean neighbor distance; a boundary/free-surface node has
|
| 141 |
+
one-sided neighbors and a large centroid offset (where the MLS gradient is also least reliable).
|
| 142 |
+
Interior iff ``||mean(neighbors) - node|| / mean_neighbor_dist < thresh``. Works for any mesh.
|
| 143 |
+
"""
|
| 144 |
+
cpu = coords.detach().cpu().numpy().astype(np.float64)
|
| 145 |
+
dist, idx = knn_indices(cpu, k)
|
| 146 |
+
nbr = cpu[idx] # (N, k, 2)
|
| 147 |
+
centroid = nbr.mean(axis=1) # (N, 2)
|
| 148 |
+
offset = np.linalg.norm(centroid - cpu, axis=1) # (N,)
|
| 149 |
+
scale = dist.mean(axis=1) + 1e-12 # (N,)
|
| 150 |
+
return torch.tensor((offset / scale) < thresh, device=coords.device)
|
| 151 |
+
|
| 152 |
+
|
| 153 |
def divergence(stress3: torch.Tensor, Gx: torch.Tensor, Gy: torch.Tensor) -> torch.Tensor:
|
| 154 |
"""Divergence of the symmetric 2x2 stress tensor field.
|
| 155 |
|
stress_operator/losses/fe_divergence.py
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""FE shape-function divergence operator (the P-DivGNN-style operator), for the head-to-head ablation.
|
| 2 |
+
|
| 3 |
+
P-DivGNN computes div(sigma) with **FE P1 shape-function gradients on a mesh**; we use a **meshfree MLS**
|
| 4 |
+
gradient that needs no connectivity. To isolate that single difference, this module builds an FE nodal
|
| 5 |
+
gradient operator on the SAME points (via a Delaunay triangulation), so we can compare both operators'
|
| 6 |
+
divergence residuals on identical predicted stress fields. The FE operator REQUIRES connectivity (the
|
| 7 |
+
triangulation); the MLS operator does not — that is the differentiating point.
|
| 8 |
+
|
| 9 |
+
Build is analogous to build_mls_gradient_operators: returns dense (N,N) Gx_fe, Gy_fe with
|
| 10 |
+
(Gx_fe @ f) ~ df/dx via area-weighted recovery of per-triangle P1 gradients. Exact on affine fields
|
| 11 |
+
(P1 elements reproduce linear fields exactly) — verified in tests.
|
| 12 |
+
"""
|
| 13 |
+
from __future__ import annotations
|
| 14 |
+
|
| 15 |
+
import numpy as np
|
| 16 |
+
import torch
|
| 17 |
+
from scipy.spatial import Delaunay
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def build_fe_gradient_operators(coords: torch.Tensor):
|
| 21 |
+
"""Dense (N,N) FE nodal-gradient operators from a Delaunay triangulation of `coords` (N,2)."""
|
| 22 |
+
pts = coords.detach().cpu().numpy().astype(np.float64)
|
| 23 |
+
n = pts.shape[0]
|
| 24 |
+
tri = Delaunay(pts).simplices # (T,3)
|
| 25 |
+
Gx = np.zeros((n, n)); Gy = np.zeros((n, n)); wsum = np.zeros(n)
|
| 26 |
+
for t in tri:
|
| 27 |
+
i0, i1, i2 = t
|
| 28 |
+
p0, p1, p2 = pts[i0], pts[i1], pts[i2]
|
| 29 |
+
# 2*signed area
|
| 30 |
+
det = (p1[0] - p0[0]) * (p2[1] - p0[1]) - (p2[0] - p0[0]) * (p1[1] - p0[1])
|
| 31 |
+
if abs(det) < 1e-14:
|
| 32 |
+
continue
|
| 33 |
+
area = 0.5 * abs(det)
|
| 34 |
+
# P1 shape-function gradients (constant per triangle): grad N_i = (b_i, c_i)/(2A_signed)
|
| 35 |
+
# b_i = y_{i+1}-y_{i+2}, c_i = x_{i+2}-x_{i+1} (cyclic), using signed 2A=det
|
| 36 |
+
bs = [p1[1] - p2[1], p2[1] - p0[1], p0[1] - p1[1]]
|
| 37 |
+
cs = [p2[0] - p1[0], p0[0] - p2[0], p1[0] - p0[0]]
|
| 38 |
+
for local, node in enumerate(t):
|
| 39 |
+
gx = bs[local] / det
|
| 40 |
+
gy = cs[local] / det
|
| 41 |
+
# area-weighted nodal recovery: each incident node gets this triangle's (constant) gradient
|
| 42 |
+
for lv, nv in enumerate(t):
|
| 43 |
+
Gx[node, nv] += area * (bs[lv] / det)
|
| 44 |
+
Gy[node, nv] += area * (cs[lv] / det)
|
| 45 |
+
wsum[node] += area
|
| 46 |
+
nz = wsum > 0
|
| 47 |
+
Gx[nz] /= wsum[nz, None]
|
| 48 |
+
Gy[nz] /= wsum[nz, None]
|
| 49 |
+
return (torch.tensor(Gx, dtype=torch.float64), torch.tensor(Gy, dtype=torch.float64))
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def build_fe_gradient_operators_sparse(coords: torch.Tensor):
|
| 53 |
+
"""Sparse FE nodal-gradient operators (same stencils as the dense builder), for large meshes."""
|
| 54 |
+
pts = coords.detach().cpu().numpy().astype(np.float64)
|
| 55 |
+
n = pts.shape[0]
|
| 56 |
+
tri = Delaunay(pts).simplices
|
| 57 |
+
rows, cols, vx, vy = [], [], [], []
|
| 58 |
+
wsum = np.zeros(n)
|
| 59 |
+
for t in tri:
|
| 60 |
+
i0, i1, i2 = t
|
| 61 |
+
p0, p1, p2 = pts[i0], pts[i1], pts[i2]
|
| 62 |
+
det = (p1[0] - p0[0]) * (p2[1] - p0[1]) - (p2[0] - p0[0]) * (p1[1] - p0[1])
|
| 63 |
+
if abs(det) < 1e-14:
|
| 64 |
+
continue
|
| 65 |
+
area = 0.5 * abs(det)
|
| 66 |
+
bs = [p1[1] - p2[1], p2[1] - p0[1], p0[1] - p1[1]]
|
| 67 |
+
cs = [p2[0] - p1[0], p0[0] - p2[0], p1[0] - p0[0]]
|
| 68 |
+
for node in t:
|
| 69 |
+
wsum[node] += area
|
| 70 |
+
for lv, nv in enumerate(t):
|
| 71 |
+
rows.append(node); cols.append(nv)
|
| 72 |
+
vx.append(area * bs[lv] / det); vy.append(area * cs[lv] / det)
|
| 73 |
+
ij = torch.tensor(np.array([rows, cols]))
|
| 74 |
+
Gx = torch.sparse_coo_tensor(ij, torch.tensor(vx), (n, n)).coalesce()
|
| 75 |
+
Gy = torch.sparse_coo_tensor(ij, torch.tensor(vy), (n, n)).coalesce()
|
| 76 |
+
w = torch.tensor(np.where(wsum > 0, wsum, 1.0))
|
| 77 |
+
# normalize each row i by wsum[i]
|
| 78 |
+
Gx = torch.sparse_coo_tensor(Gx.indices(), Gx.values() / w[Gx.indices()[0]], (n, n)).coalesce()
|
| 79 |
+
Gy = torch.sparse_coo_tensor(Gy.indices(), Gy.values() / w[Gy.indices()[0]], (n, n)).coalesce()
|
| 80 |
+
return Gx.to(coords.device), Gy.to(coords.device)
|
stress_operator/models/linear_no.py
CHANGED
|
@@ -3,7 +3,11 @@
|
|
| 3 |
Reimplemented from the equations of:
|
| 4 |
"Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of
|
| 5 |
Linear Attention" — Hu, Liu, Qiao, Sun, Dou (NUDT), AAAI 2026, arXiv:2511.06294.
|
| 6 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
Core idea: Physics-Attention is the special case of linear attention
|
| 9 |
``Attention(Q,K,V) ~ phi(Q) (psi^T(K) V)`` in which (a) phi and psi come from the SAME linear
|
|
@@ -50,12 +54,19 @@ class LinearNO(nn.Module):
|
|
| 50 |
dropout=0.0,
|
| 51 |
variant: str = "independent",
|
| 52 |
project_out: bool = True,
|
|
|
|
| 53 |
):
|
| 54 |
super().__init__()
|
| 55 |
inner = heads * dim_head
|
| 56 |
self.h, self.m, self.dh = heads, slice_num, dim_head
|
| 57 |
self.variant = variant
|
| 58 |
self.project_out = project_out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
if variant == "independent":
|
| 61 |
# Independent Q, K, V projections (paper modification 1, literal).
|
|
@@ -93,8 +104,13 @@ class LinearNO(nn.Module):
|
|
| 93 |
k = base
|
| 94 |
v = self._heads(self.to_v(x), B, N)
|
| 95 |
|
| 96 |
-
|
| 97 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
kv = torch.einsum("bhnm,bhnd->bhmd", psi, v) # (B,H,M,dh) cheap inner product first
|
| 99 |
out = torch.einsum("bhnm,bhmd->bhnd", phi, kv) # (B,H,N,dh) linear in N
|
| 100 |
out = rearrange(out, "b h n d -> b n (h d)")
|
|
|
|
| 3 |
Reimplemented from the equations of:
|
| 4 |
"Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of
|
| 5 |
Linear Attention" — Hu, Liu, Qiao, Sun, Dou (NUDT), AAAI 2026, arXiv:2511.06294.
|
| 6 |
+
Reconstructed from the equations BEFORE the official code was released (github.com/HiPRL/LinearNO,
|
| 7 |
+
Jan 2026); we did not consult it. A post-hoc check confirms structural agreement (softmax axes,
|
| 8 |
+
associativity). Differences from the official Elasticity block: it adds a learnable per-head softmax
|
| 9 |
+
temperature (init 0.5, clamped [0.01,1]) that we omit, and uses one shared in_project_x lift + small
|
| 10 |
+
per-head q/k/v maps vs our full C->inner projections. Only the attention block differs from Transolver.
|
| 11 |
|
| 12 |
Core idea: Physics-Attention is the special case of linear attention
|
| 13 |
``Attention(Q,K,V) ~ phi(Q) (psi^T(K) V)`` in which (a) phi and psi come from the SAME linear
|
|
|
|
| 54 |
dropout=0.0,
|
| 55 |
variant: str = "independent",
|
| 56 |
project_out: bool = True,
|
| 57 |
+
temperature: bool = False,
|
| 58 |
):
|
| 59 |
super().__init__()
|
| 60 |
inner = heads * dim_head
|
| 61 |
self.h, self.m, self.dh = heads, slice_num, dim_head
|
| 62 |
self.variant = variant
|
| 63 |
self.project_out = project_out
|
| 64 |
+
self.temperature = temperature
|
| 65 |
+
if temperature:
|
| 66 |
+
# Matches the official LinearNO `temp` block: learnable per-head temperature on both
|
| 67 |
+
# softmaxes, init 0.5, clamped to [0.01, 1] (github.com/HiPRL/LinearNO).
|
| 68 |
+
self.temp_q = nn.Parameter(torch.ones(1, heads, 1, 1) * 0.5)
|
| 69 |
+
self.temp_k = nn.Parameter(torch.ones(1, heads, 1, 1) * 0.5)
|
| 70 |
|
| 71 |
if variant == "independent":
|
| 72 |
# Independent Q, K, V projections (paper modification 1, literal).
|
|
|
|
| 104 |
k = base
|
| 105 |
v = self._heads(self.to_v(x), B, N)
|
| 106 |
|
| 107 |
+
sq = self.lin_q(q) # (B,H,N,M)
|
| 108 |
+
sk = self.lin_k(k) # (B,H,N,M)
|
| 109 |
+
if self.temperature:
|
| 110 |
+
sq = sq / self.temp_q.clamp(0.01, 1.0)
|
| 111 |
+
sk = sk / self.temp_k.clamp(0.01, 1.0)
|
| 112 |
+
phi = sq.softmax(dim=-1) # softmax OVER M (slices) <- rows sum to 1
|
| 113 |
+
psi = sk.softmax(dim=-2) # softmax OVER N (points) <- cols sum to 1
|
| 114 |
kv = torch.einsum("bhnm,bhnd->bhmd", psi, v) # (B,H,M,dh) cheap inner product first
|
| 115 |
out = torch.einsum("bhnm,bhmd->bhnd", phi, kv) # (B,H,N,dh) linear in N
|
| 116 |
out = rearrange(out, "b h n d -> b n (h d)")
|
stress_operator/models/transolver.py
CHANGED
|
@@ -27,6 +27,7 @@ def make_attention(
|
|
| 27 |
slice_num,
|
| 28 |
linearno_variant: str = "shared_qk",
|
| 29 |
linearno_project_out: bool = False,
|
|
|
|
| 30 |
) -> nn.Module:
|
| 31 |
if kind == "physics":
|
| 32 |
return Physics_Attention_Irregular_Mesh(
|
|
@@ -43,6 +44,7 @@ def make_attention(
|
|
| 43 |
dropout=dropout,
|
| 44 |
variant=linearno_variant,
|
| 45 |
project_out=linearno_project_out,
|
|
|
|
| 46 |
)
|
| 47 |
raise ValueError(f"unknown attention kind {kind!r} (expected 'physics' or 'linearno')")
|
| 48 |
|
|
@@ -66,6 +68,7 @@ class StressOperator(nn.Module):
|
|
| 66 |
act: str = "gelu",
|
| 67 |
linearno_variant: str = "shared_qk",
|
| 68 |
linearno_project_out: bool = False,
|
|
|
|
| 69 |
):
|
| 70 |
super().__init__()
|
| 71 |
if dim_head is None:
|
|
@@ -85,6 +88,7 @@ class StressOperator(nn.Module):
|
|
| 85 |
attention, n_hidden, n_heads, dim_head, dropout, slice_num,
|
| 86 |
linearno_variant=linearno_variant,
|
| 87 |
linearno_project_out=linearno_project_out,
|
|
|
|
| 88 |
),
|
| 89 |
hidden_dim=n_hidden,
|
| 90 |
dropout=dropout,
|
|
@@ -152,6 +156,7 @@ def build_model(model_cfg: dict) -> StressOperator:
|
|
| 152 |
act=model_cfg.get("act", "gelu"),
|
| 153 |
linearno_variant=model_cfg.get("linearno_variant", "shared_qk"),
|
| 154 |
linearno_project_out=model_cfg.get("linearno_project_out", False),
|
|
|
|
| 155 |
)
|
| 156 |
|
| 157 |
|
|
|
|
| 27 |
slice_num,
|
| 28 |
linearno_variant: str = "shared_qk",
|
| 29 |
linearno_project_out: bool = False,
|
| 30 |
+
linearno_temperature: bool = False,
|
| 31 |
) -> nn.Module:
|
| 32 |
if kind == "physics":
|
| 33 |
return Physics_Attention_Irregular_Mesh(
|
|
|
|
| 44 |
dropout=dropout,
|
| 45 |
variant=linearno_variant,
|
| 46 |
project_out=linearno_project_out,
|
| 47 |
+
temperature=linearno_temperature,
|
| 48 |
)
|
| 49 |
raise ValueError(f"unknown attention kind {kind!r} (expected 'physics' or 'linearno')")
|
| 50 |
|
|
|
|
| 68 |
act: str = "gelu",
|
| 69 |
linearno_variant: str = "shared_qk",
|
| 70 |
linearno_project_out: bool = False,
|
| 71 |
+
linearno_temperature: bool = False,
|
| 72 |
):
|
| 73 |
super().__init__()
|
| 74 |
if dim_head is None:
|
|
|
|
| 88 |
attention, n_hidden, n_heads, dim_head, dropout, slice_num,
|
| 89 |
linearno_variant=linearno_variant,
|
| 90 |
linearno_project_out=linearno_project_out,
|
| 91 |
+
linearno_temperature=linearno_temperature,
|
| 92 |
),
|
| 93 |
hidden_dim=n_hidden,
|
| 94 |
dropout=dropout,
|
|
|
|
| 156 |
act=model_cfg.get("act", "gelu"),
|
| 157 |
linearno_variant=model_cfg.get("linearno_variant", "shared_qk"),
|
| 158 |
linearno_project_out=model_cfg.get("linearno_project_out", False),
|
| 159 |
+
linearno_temperature=model_cfg.get("linearno_temperature", False),
|
| 160 |
)
|
| 161 |
|
| 162 |
|
stress_operator/train_tensile.py
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Tensile2d training with the equilibrium regularizer — the SUPERVISED-tensor transfer experiment.
|
| 2 |
+
|
| 3 |
+
Unlike Geo-FNO Elasticity (scalar von Mises target -> latent tensor), Tensile2d directly supervises the
|
| 4 |
+
full Cauchy tensor (sig11, sig22, sig12) per node. So here:
|
| 5 |
+
- the data loss is relative-L2 on the 3 SUPERVISED stress components (not a derived von Mises);
|
| 6 |
+
- the model conditions on the 6 input scalars (P, p1..p5) via fun_dim=6 (broadcast per node);
|
| 7 |
+
- the physics loss is the same meshfree MLS divergence ||div(sigma)||^2 on interior nodes.
|
| 8 |
+
Meshes are variable-size (~6k-12k nodes), so we keep per-sample lists (batch_size=1).
|
| 9 |
+
|
| 10 |
+
This run lets us test the key question raised by the Elasticity cross-operator finding: does supervising
|
| 11 |
+
the true tensor REDUCE the operator-specific 'gaming' (MLS-vs-FE residual gap)? (Checked post-hoc.)
|
| 12 |
+
"""
|
| 13 |
+
from __future__ import annotations
|
| 14 |
+
|
| 15 |
+
import os
|
| 16 |
+
import time
|
| 17 |
+
from typing import Any, Dict, Optional
|
| 18 |
+
|
| 19 |
+
import torch
|
| 20 |
+
|
| 21 |
+
from .data.tensile2d import build_tensile_splits
|
| 22 |
+
from .losses.equilibrium import build_mls_gradient_operators_sparse, interior_mask_knn
|
| 23 |
+
from .losses.relative_l2 import relative_l2
|
| 24 |
+
from .models.transolver import build_model, count_parameters
|
| 25 |
+
from .seeds import set_seed
|
| 26 |
+
from .utils.logging import MODAL_RATES_PER_SEC, write_run_log
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
def _precompute_ops(coords_list, k, device):
|
| 30 |
+
ops = []
|
| 31 |
+
for c in coords_list:
|
| 32 |
+
Gx, Gy = build_mls_gradient_operators_sparse(c, k=k) # sparse-direct (scales to ~10k nodes)
|
| 33 |
+
ops.append((Gx.to(device), Gy.to(device), interior_mask_knn(c, k=k).to(device)))
|
| 34 |
+
return ops
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def _div_residual(stress3, Gx_s, Gy_s, mask):
|
| 38 |
+
sxx, syy, sxy = stress3[:, 0:1], stress3[:, 1:2], stress3[:, 2:3]
|
| 39 |
+
dx = torch.sparse.mm(Gx_s, sxx) + torch.sparse.mm(Gy_s, sxy)
|
| 40 |
+
dy = torch.sparse.mm(Gx_s, sxy) + torch.sparse.mm(Gy_s, syy)
|
| 41 |
+
return (dx.squeeze(-1) ** 2 + dy.squeeze(-1) ** 2)[mask].mean()
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def run_training_tensile(
|
| 45 |
+
config: Dict[str, Any], seed: int, data_dir: str = "data/tensile2d/samples",
|
| 46 |
+
device: Optional[str] = None, gpu_name: str = "CPU",
|
| 47 |
+
results_path: Optional[str] = None, ckpt_path: Optional[str] = None,
|
| 48 |
+
log_every: int = 25, max_epochs: Optional[int] = None,
|
| 49 |
+
splits=None, lambda_override: Optional[float] = None,
|
| 50 |
+
) -> Dict[str, Any]:
|
| 51 |
+
device = device or ("cuda" if torch.cuda.is_available() else "cpu")
|
| 52 |
+
set_seed(seed)
|
| 53 |
+
tr_cfg, model_cfg, eq_cfg = config["train"], config["model"], config["equilibrium"]
|
| 54 |
+
lam = float(lambda_override) if lambda_override is not None else float(eq_cfg.get("lambda", 0.01))
|
| 55 |
+
k = int(eq_cfg.get("knn_k", 12)) # interior_mask_knn is geometry-agnostic; no interior_tol needed
|
| 56 |
+
|
| 57 |
+
if splits is None:
|
| 58 |
+
d = config["data"]
|
| 59 |
+
splits = build_tensile_splits(data_dir, ntrain=d.get("ntrain", 400), ntest=d.get("ntest", 100), seed=0)
|
| 60 |
+
S = splits.scale_S
|
| 61 |
+
trc = [c.to(device) for c in splits.train_coords]
|
| 62 |
+
trs = [s.to(device).float() for s in splits.train_sigma]
|
| 63 |
+
trsc = [sc.to(device).float() for sc in splits.train_scalars]
|
| 64 |
+
tec = [c.to(device) for c in splits.test_coords]
|
| 65 |
+
tes = [s.to(device).float() for s in splits.test_sigma]
|
| 66 |
+
tesc = [sc.to(device).float() for sc in splits.test_scalars]
|
| 67 |
+
n_train, n_test = len(trc), len(tec)
|
| 68 |
+
|
| 69 |
+
print(f"[tensile seed {seed}] lambda={lam} k={k} S={S:.2f} ntrain={n_train} ntest={n_test}", flush=True)
|
| 70 |
+
t_build = time.time()
|
| 71 |
+
train_ops = _precompute_ops(splits.train_coords, k, device)
|
| 72 |
+
test_ops = _precompute_ops(splits.test_coords, k, device)
|
| 73 |
+
print(f"[tensile seed {seed}] precomputed operators in {time.time()-t_build:.0f}s", flush=True)
|
| 74 |
+
|
| 75 |
+
model = build_model(model_cfg).to(device)
|
| 76 |
+
n_params = count_parameters(model)
|
| 77 |
+
epochs = max_epochs or int(tr_cfg.get("epochs", 500))
|
| 78 |
+
opt = torch.optim.AdamW(model.parameters(), lr=float(tr_cfg.get("lr", 1e-3)),
|
| 79 |
+
weight_decay=float(tr_cfg.get("weight_decay", 1e-5)),
|
| 80 |
+
betas=tuple(tr_cfg.get("betas", (0.9, 0.999))))
|
| 81 |
+
sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, T_max=epochs)
|
| 82 |
+
mgn = tr_cfg.get("max_grad_norm", None)
|
| 83 |
+
eval_every = int(tr_cfg.get("eval_every", 10))
|
| 84 |
+
gen = torch.Generator().manual_seed(seed)
|
| 85 |
+
|
| 86 |
+
def fx_of(coords, scalars): # broadcast the 6 input scalars to every node
|
| 87 |
+
return scalars.view(1, 1, -1).expand(1, coords.shape[0], -1)
|
| 88 |
+
|
| 89 |
+
def losses(coords, sigma, scalars, ops_i):
|
| 90 |
+
out = model(coords.unsqueeze(0), fx_of(coords, scalars))[0] # (N,3) scaled stress
|
| 91 |
+
data = relative_l2(out.unsqueeze(0), sigma.unsqueeze(0), reduction="mean") # SUPERVISED tensor
|
| 92 |
+
phys = _div_residual(out, *ops_i)
|
| 93 |
+
return data, phys
|
| 94 |
+
|
| 95 |
+
@torch.no_grad()
|
| 96 |
+
def evaluate():
|
| 97 |
+
model.eval()
|
| 98 |
+
d, p = 0.0, 0.0
|
| 99 |
+
for i in range(n_test):
|
| 100 |
+
dl, ph = losses(tec[i], tes[i], tesc[i], test_ops[i])
|
| 101 |
+
d += dl.item(); p += ph.item()
|
| 102 |
+
return d / n_test, p / n_test
|
| 103 |
+
|
| 104 |
+
t0 = time.time()
|
| 105 |
+
best, test_rel, test_phys, hist = float("inf"), float("nan"), float("nan"), []
|
| 106 |
+
for ep in range(epochs):
|
| 107 |
+
model.train()
|
| 108 |
+
run = 0.0
|
| 109 |
+
for i in torch.randperm(n_train, generator=gen).tolist():
|
| 110 |
+
opt.zero_grad()
|
| 111 |
+
dl, ph = losses(trc[i], trs[i], trsc[i], train_ops[i])
|
| 112 |
+
(dl + lam * ph).backward()
|
| 113 |
+
if mgn is not None:
|
| 114 |
+
torch.nn.utils.clip_grad_norm_(model.parameters(), mgn)
|
| 115 |
+
opt.step(); run += dl.item()
|
| 116 |
+
sched.step()
|
| 117 |
+
train_rel = run / n_train
|
| 118 |
+
if (ep % eval_every == 0) or (ep >= epochs - 5):
|
| 119 |
+
test_rel, test_phys = evaluate(); best = min(best, test_rel)
|
| 120 |
+
hist.append({"epoch": ep, "train_rel": train_rel, "test_rel": test_rel, "test_phys": test_phys})
|
| 121 |
+
if ep % log_every == 0 or ep == epochs - 1:
|
| 122 |
+
print(f"[tensile seed {seed}] epoch {ep:4d} train_rel={train_rel:.5f} "
|
| 123 |
+
f"test_rel={test_rel:.5f} test_resid={test_phys:.4e}", flush=True)
|
| 124 |
+
|
| 125 |
+
wall = time.time() - t0
|
| 126 |
+
rate = MODAL_RATES_PER_SEC.get(gpu_name, 0.0)
|
| 127 |
+
metrics = {
|
| 128 |
+
"test_rel_l2": round(test_rel, 6), "best_test_rel_l2": round(best, 6),
|
| 129 |
+
"test_residual": test_phys * (S ** 2), "test_residual_scaled": test_phys,
|
| 130 |
+
"scale_S": S, "train_rel_l2": round(train_rel, 6),
|
| 131 |
+
"n_params": n_params, "epochs": epochs, "lambda": lam, "dataset": "tensile2d",
|
| 132 |
+
}
|
| 133 |
+
if ckpt_path is not None:
|
| 134 |
+
os.makedirs(os.path.dirname(ckpt_path) or ".", exist_ok=True)
|
| 135 |
+
torch.save({"state_dict": model.state_dict(), "scale_S": S, "config": config,
|
| 136 |
+
"scalar_mean": splits.scalar_mean, "scalar_std": splits.scalar_std,
|
| 137 |
+
"seed": seed, "metrics": metrics}, ckpt_path)
|
| 138 |
+
if results_path is None:
|
| 139 |
+
results_path = os.path.join("results", f"{config.get('name','tensile')}_seed{seed}.json")
|
| 140 |
+
write_run_log(results_path, config, seed, metrics, wall, gpu_name, wall * rate,
|
| 141 |
+
extra={"history_tail": hist[-5:]})
|
| 142 |
+
print(f"[tensile seed {seed}] DONE test_rel={test_rel:.6f} resid={test_phys:.4e} "
|
| 143 |
+
f"wall={wall:.0f}s est_cost=${wall*rate:.4f}", flush=True)
|
| 144 |
+
return metrics
|