Efradeca commited on
Commit
2c93889
·
verified ·
1 Parent(s): 23edd48

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ fig5_mls_convergence.png filter=lfs diff=lfs merge=lfs -text
37
+ fig6_cross_operator.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -5,124 +5,166 @@ tags:
5
  - neural-operator
6
  - pde-solver
7
  - physics-informed
 
8
  - elasticity
9
  - linear-attention
10
  - transolver
 
11
  metrics:
12
  - relative-l2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ---
14
 
15
- # Model Card for Equilibrium-Regularized LinearNO — Elastic Stress Surrogate (Geo-FNO Elasticity)
16
 
17
- A neural-operator surrogate that predicts the per-node von Mises stress field of a hyper-elastic
18
- unit cell with a central void, on the **Geo-FNO Elasticity** benchmark. It maps a 2-D unstructured
19
- mesh (972 nodes) directly to the stress field in milliseconds on CPU, replacing a per-geometry
20
- finite-element (FEM) solve for fast design-space exploration.
 
 
 
 
 
 
 
21
 
22
  ## Model Details
23
 
24
  ### Model Description
25
 
26
- The model is a Transolver-family transformer operator whose attention block is the asymmetric
27
- linear-attention operator **LinearNO**. Its distinguishing component is an **equilibrium-residual
28
- regularizer**: the network predicts the full stress tensor (σ_xx, σ_yy, σ_xy) and is penalized by a
29
- discrete divergence operator so the predicted field approaches static mechanical equilibrium
30
- (∇·σ 0), yielding physically consistent predictions at no measured accuracy cost.
 
 
31
 
32
  - **Developed by:** Efradeca
33
  - **Model type:** Neural operator (transformer PDE surrogate) for 2-D static hyper-elastic stress
34
- - **Inputs / outputs:** node coordinates (B, 972, 2) → per-node von Mises stress (B, 972)
 
35
  - **License:** MIT
36
- - **Built on (not original to this work):** the **Transolver** solver (Wu et al., ICML 2024) and a
37
- reimplementation of the **LinearNO** attention block (Hu et al., AAAI 2026). The original
38
- contribution here is the equilibrium-residual regularizer and the out-of-distribution analysis.
 
 
 
 
 
 
 
 
 
 
39
 
40
  ### Model Sources
41
 
42
- - **Papers:** Transolver (arXiv:2402.02366); LinearNO (arXiv:2511.06294); Geo-FNO dataset (arXiv:2207.05209)
43
- - **Demo:** https://huggingface.co/spaces/Efradeca/elastic-stress-surrogate
 
 
 
44
 
45
  ## Uses
46
 
47
  ### Direct Use
48
 
49
  Millisecond-scale, CPU prediction of the von Mises stress field of a hyper-elastic unit cell with a
50
- central void, for design-space exploration / shape optimization of the void (screen many candidate
51
- geometries, then verify the few best with FEM).
52
 
53
  ### Downstream Use
54
 
55
- A starting point for fine-tuning to related geometries, materials, or boundary conditions, which
56
- requires retraining on the corresponding FEM data.
57
 
58
  ### Out-of-Scope Use
59
 
60
- This is a narrow surrogate, not a general stress solver. It is **not** valid, without retraining,
61
- for other materials (e.g. steel), other boundary conditions, or geometries outside the training
62
- distribution, and it must not be used as a certified solver for safety-critical decisions.
63
 
64
  ## Bias, Risks, and Limitations
65
 
66
- - **Narrow domain:** a single benchmark, one hyper-elastic (rubber-like) material, one geometry
67
- family (unit cell, central void, radius 0.2–0.4, clamped bottom, tensile top).
68
- - **Run-to-run variance:** the training set is small (1000 samples), so per-seed results vary
69
- noticeably; results are reported as mean ± std with the full per-seed distribution (see Evaluation).
70
- - **Reimplemented component:** LinearNO has no official public reference implementation; it was
71
- reproduced from the paper's equations.
72
- - **Regularizer scope:** the supervised target is scalar von Mises stress; the tensor components are
73
- latent and shaped by a *discrete* ∇·σ penalty (validated against analytic fields), which is a
74
- physical-consistency prior, not exact continuous momentum balance.
75
- - **Out-of-distribution:** accuracy degrades on geometries far from the training distribution
76
- (extreme voids); the regularizer maintains physical consistency OOD but does not improve OOD accuracy.
 
 
 
 
77
 
78
  ### Recommendations
79
 
80
- Use within the training distribution; verify any safety-critical prediction with FEM; monitor the
81
- equilibrium residual ‖∇·σ‖² as a physical-consistency indicator.
82
 
83
  ## How to Get Started with the Model
84
 
85
  ```python
86
  from huggingface_hub import snapshot_download
87
- from model import load_checkpoint, predict_stress # bundled with the repo
88
 
89
  repo = snapshot_download("Efradeca/transolver-linearno-elasticity")
 
 
 
90
  model, normalizer, info = load_checkpoint(f"{repo}/model.safetensors", device="cpu")
91
 
92
  # coords: (N, 2) node coordinates of a unit cell with a central void
93
- stress = predict_stress(model, coords, normalizer, info) # (N,) von Mises stress
94
  ```
95
 
96
  ## Training Details
97
 
98
  ### Training Data
99
 
100
- Geo-FNO Elasticity (Li et al., 2022, arXiv:2207.05209): FEM simulations of an incompressible
101
- hyper-elastic (Rivlin–Saunders, `C₁=1.86e5`, `C₂=9.79e3`) `[0,1]²` unit cell with a random central
102
- void (radius 0.2–0.4), 972 nodes per sample; target = per-node von Mises stress. Split: 1000 train /
103
- 200 test (first-1000 / last-200 of 2000, upstream Transolver protocol).
104
 
105
- **Fixed boundary conditions (identical for every sample):** the bottom edge is **clamped**
106
- (zero displacement) and a uniform **tensile traction `t = [0, 100]` is applied on the top edge** (`+y`);
107
- left/right edges are free. **Units are non-dimensional benchmark units (not MPa).** The model's only
108
- input is the 972 node coordinates — the loads/supports are *not* inputs (they are constant across the
109
- dataset; only the void geometry varies).
110
 
111
  ### Training Procedure
112
 
113
- #### Preprocessing
114
-
115
- Global z-score normalization of the stress target (de-normalized before the metric). The
116
- equilibrium-regularized model outputs three stress-tensor channels and derives von Mises.
117
-
118
- #### Training Hyperparameters
119
-
120
- - **Architecture:** 8 layers, hidden dim 128, 8 heads, dim_head 16, slices M = 64;
121
- attention `linearno` (variant `shared_qk`, project_out=`False`)
122
- - **Optimization:** AdamW, lr 1e-3, weight decay 1e-5, cosine annealing, 500 epochs, batch size 1,
123
- gradient clipping 0.1
124
- - **Loss:** relative L2 + λ·‖∇·σ‖² on interior nodes (λ = 0.01)
125
- - **Seeds:** {0, 1, 2} (mean ± std reported)
126
 
127
  ## Evaluation
128
 
@@ -130,70 +172,102 @@ equilibrium-regularized model outputs three stress-tensor channels and derives v
130
 
131
  - **Testing data:** the 200 held-out FEM meshes.
132
  - **Factors:** in-distribution vs. out-of-distribution (geometry-stratified by void size).
133
- - **Metrics:** relative L2 (primary); the discrete equilibrium residual ‖∇·σ‖²; per-node Pearson r
134
- and R² against the FEM ground truth; peak-stress relative error.
135
 
136
- ### Results
137
 
138
- Geo-FNO Elasticity test relative L2. We report **median** (robust to the high run-to-run variance of
139
- this 1000-sample benchmark) alongside mean ± std and the seed count; full committed per-seed
140
- distributions are in the repository (`docs/RESULTS.md`, `results/study_summary.json`). The
141
- accuracy-comparison rows (baseline, LinearNO M=64/M=256) use `torch.compile`; the
142
- equilibrium-regularized row is eager (`compile:false`). The baseline reproduces the published 0.0064
143
- on its good seeds.
144
 
145
  | Model | median | mean ± std | n | params |
146
  |---|---|---|---|---|
147
  | Published Transolver (Wu et al. 2024) | — | 0.0064 | — | ~0.7M |
148
  | Published LinearNO (Hu et al. 2026, M=64) | — | 0.0050 | — | — |
149
- | Transolver baseline (this work) | 0.00766 | 0.00827 ± 0.0022 | 6 | 713,665 |
150
- | LinearNO, M=64 (this work) | **0.00606** | 0.00645 ± 0.0017 | 8 | 713,089 |
151
- | LinearNO, M=256 (this work) | 0.00635 | 0.00622 ± 0.0007 | 3 | 765,313 |
152
- | **+ equilibrium regularizer (this model)** | 0.00653 | 0.00668 ± 0.0006 | 3 | 582,275 |
153
-
154
- **Controlled comparison (with appropriate caution).** In this sample, LinearNO's median error
155
- (0.00606) is lower than our reproduced baseline's (0.00766) at fewer parameters, and LinearNO reaches
156
- the published LinearNO/Transolver level on its good seeds (best run, M=256: **0.00529**). However, both
157
- models show **high run-to-run variance** (`torch.compile` non-determinism on a 1000-sample dataset),
158
- and the baseline's higher mean is driven by 2 outlier seeds. We **do not** claim LinearNO "beats" the
159
- baseline or is "more robust": with these seed counts (6 vs. 8) the difference may partly reflect
160
- sampling noise, and an attempted mechanistic explanation (degenerate slice assignments in the
161
- baseline's slice self-attention) was **empirically tested and refuted** — the outlier seed is a generic
162
- poor-generalization minimum (healthy train loss, worse test loss), not a slice-attention failure. The
163
- honest reading is that LinearNO is **comparable to** the baseline at ≤ its parameters, both reproducing
164
- ~0.0064 on good seeds.
165
-
166
- **Physical consistency.** The regularizer reduces the discrete equilibrium residual ‖∇·σ‖² by
167
- **~360×** (6.1e6 → 1.7e4) at no measured accuracy cost, and this consistency is maintained
168
- out-of-distribution (OOD residual 1.73e4 vs. in-distribution 1.61e4).
169
-
170
- **Verification against FEM ground truth** (200 test meshes): per-sample relative L2 mean 0.00597
171
- (median 0.00533), pooled Pearson r = 0.9999, = 0.9999, peak-stress relative error 0.3%.
172
-
173
- #### Summary
174
-
175
- Over 6–8 seeds under identical conditions, LinearNO is **comparable to** the reproduced Transolver
176
- baseline (lower median in this sample, but within the high run-to-run variance), at its parameter
177
- count. The original contribution of this checkpoint is physical consistency: the equilibrium
178
- regularizer makes the predicted stress field satisfy discrete static equilibrium (∇·σ 0) at no
179
- measured accuracy cost a property maintained under geometric covariate shift and verified node-by-node
180
- against FEM ( = 0.9999).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
181
 
182
  ## Technical Specifications
183
 
184
  ### Model Architecture and Objective
185
 
186
- Transolver-family transformer (encoder → 8 pre-norm blocks → linear decoder head) with the LinearNO
187
- asymmetric linear-attention block; objective = relative-L2 data loss + equilibrium-residual penalty.
 
 
188
 
189
  ### Compute Infrastructure
190
 
191
- - **Hardware:** single NVIDIA A10 GPU (Modal).
192
- - **Software:** PyTorch, einops, safetensors; CPU inference for the demo.
 
 
 
 
 
 
 
 
 
 
 
 
193
 
194
  ## Citation
195
 
196
- **BibTeX:**
 
197
  ```bibtex
198
  @inproceedings{wu2024transolver,
199
  title={Transolver: A Fast Transformer Solver for PDEs on General Geometries},
@@ -213,12 +287,14 @@ asymmetric linear-attention block; objective = relative-L2 data loss + equilibri
213
  journal={arXiv:2207.05209},
214
  year={2022}
215
  }
 
 
 
 
 
 
216
  ```
217
 
218
- ## Model Card Authors
219
-
220
- Efradeca.
221
-
222
- ## Model Card Contact
223
 
224
- https://huggingface.co/Efradeca
 
5
  - neural-operator
6
  - pde-solver
7
  - physics-informed
8
+ - computational-mechanics
9
  - elasticity
10
  - linear-attention
11
  - transolver
12
+ - stress-prediction
13
  metrics:
14
  - relative-l2
15
+ model-index:
16
+ - name: transolver-linearno-elasticity-eqreg
17
+ results:
18
+ - task:
19
+ type: operator-learning
20
+ name: Elastic von Mises stress-field prediction (Geo-FNO Elasticity)
21
+ dataset:
22
+ type: geo-fno-elasticity
23
+ name: Geo-FNO Elasticity (Li et al., 2022)
24
+ metrics:
25
+ - type: relative-l2
26
+ value: 0.00597
27
+ name: Test relative L2 (shipped checkpoint, mean over 200 meshes)
28
+ - type: r-squared
29
+ value: 0.99986
30
+ name: Pooled R^2 vs FEM ground truth
31
  ---
32
 
33
+ # Equilibrium-Regularized LinearNO — Elastic Stress Surrogate
34
 
35
+ A neural-operator surrogate that predicts the per-node **von Mises stress field** of a hyper-elastic
36
+ unit cell with an arbitrary central void, on the **Geo-FNO Elasticity** benchmark. It maps a 2-D
37
+ unstructured mesh (972 nodes) directly to the stress field in **milliseconds on CPU**, replacing a
38
+ per-geometry finite-element (FEM) solve for fast design-space exploration.
39
+
40
+ Verified node-by-node against FEM: pooled **R² = 0.99986**, test relative-L2 **0.00597** on the shipped
41
+ checkpoint. **Interactive demo:** https://huggingface.co/spaces/Efradeca/elastic-stress-surrogate
42
+
43
+ > **Honesty note.** This card reports the model's limitations as prominently as its strengths. The
44
+ > deployed value is *accuracy + speed*. The equilibrium regularizer is a well-characterized **trade-off**
45
+ > (a prior-art idea in a new meshfree instantiation), not a free win — see *Evaluation* and *Findings*.
46
 
47
  ## Model Details
48
 
49
  ### Model Description
50
 
51
+ The model is a **Transolver-family transformer operator** whose attention block is the asymmetric
52
+ **LinearNO** linear-attention operator. It predicts the full symmetric Cauchy stress tensor
53
+ (σ_xx, σ_yy, σ_xy) per node; the supervised target is the derived scalar von Mises stress. Its
54
+ distinguishing component is a soft **equilibrium-residual regularizer** the training loss penalizes the
55
+ discrete divergence ‖∇·σ‖² so the predicted field approaches static mechanical equilibrium (∇·σ 0),
56
+ with the divergence discretized by a **meshfree moving-least-squares (MLS)** operator that needs no mesh
57
+ connectivity.
58
 
59
  - **Developed by:** Efradeca
60
  - **Model type:** Neural operator (transformer PDE surrogate) for 2-D static hyper-elastic stress
61
+ - **Inputs / outputs:** node coordinates `(B, 972, 2)` → per-node von Mises stress `(B, 972)`
62
+ - **Parameters:** 582,275 (parameter-matched to the official LinearNO Elasticity configuration)
63
  - **License:** MIT
64
+ - **Built on (not original to this work):** the **Transolver** solver (Wu et al., ICML 2024, MIT — its
65
+ attention block is reused verbatim) and an independent, from-equations reimplementation of the
66
+ **LinearNO** attention block (Hu et al., AAAI 2026).
67
+
68
+ ### What is and isn't novel (read this)
69
+
70
+ The soft equilibrium-residual penalty `λ‖∇·σ‖²` on a predicted stress field is **established prior art**
71
+ (PI-UNet 2022; PiFNO 2024; and especially **P-DivGNN 2025**, which uses the same loss and the same
72
+ three-component Cauchy-stress target on the same kind of holed mesh). We do **not** claim the idea. This
73
+ repository's actual contribution is (1) a specific, previously-unattested **instantiation** — the penalty
74
+ on a *transformer* operator with a *meshfree, connectivity-free* MLS divergence (exact on affine fields by
75
+ the degree-1 GMLS reproduction property), distinct from the FE / FFT / finite-difference operators of all
76
+ prior work — and (2) a **cross-operator honesty analysis** of when that penalty actually helps.
77
 
78
  ### Model Sources
79
 
80
+ - **Reference papers:** Transolver (arXiv:2402.02366, MIT); LinearNO (arXiv:2511.06294); Geo-FNO dataset
81
+ (arXiv:2207.05209, MIT); closest prior work P-DivGNN (arXiv:2507.05291).
82
+ - **Interactive demo (Space):** https://huggingface.co/spaces/Efradeca/elastic-stress-surrogate
83
+ - **Full technical report + committed evidence:** the project repository (`docs/PAPER.md`,
84
+ `docs/RESULTS.md`, and the `results/` run logs).
85
 
86
  ## Uses
87
 
88
  ### Direct Use
89
 
90
  Millisecond-scale, CPU prediction of the von Mises stress field of a hyper-elastic unit cell with a
91
+ central void for **design-space exploration / void-shape optimization**: screen many candidate
92
+ geometries with the surrogate, then verify only the best few with FEM.
93
 
94
  ### Downstream Use
95
 
96
+ A starting point for fine-tuning to related geometries, materials, or boundary conditions which
97
+ **requires retraining** on the corresponding FEM data.
98
 
99
  ### Out-of-Scope Use
100
 
101
+ This is a **narrow surrogate, not a general stress solver**. Without retraining it is **not** valid for
102
+ other materials (e.g. steel), other boundary conditions, or geometries outside the training distribution,
103
+ and it **must not** be used as a certified solver for safety-critical decisions.
104
 
105
  ## Bias, Risks, and Limitations
106
 
107
+ - **Narrow domain.** One benchmark, one hyper-elastic (rubber-like Rivlin–Saunders) material, one geometry
108
+ family (unit cell, central void, radius 0.2–0.4, clamped bottom, tensile top). Not a foundation model.
109
+ - **Units.** Stress is in **consistent non-dimensional benchmark units, not MPa** (traction = 100,
110
+ C₁ = 1.86×10⁵); a physical-units result requires re-scaling to the target material.
111
+ - **Run-to-run variance.** The training set is small (1000 samples), so per-seed results vary noticeably.
112
+ We report median + mean ± std with the **full per-seed distribution** and do not hide outliers. The
113
+ shipped checkpoint is the **best of 3** eqreg seeds (0.00597 < 0.00653 < 0.00754).
114
+ - **Regularizer scope.** The supervised target is scalar von Mises; the tensor components are *latent*,
115
+ shaped by a *discrete* ∇·σ penalty (a physical-consistency prior, not exact continuous momentum balance).
116
+ - **The physics benefit is a trade-off, and partly operator-specific.** See *Evaluation Findings*: the
117
+ large residual reduction is mostly measured by the model's own operator, it costs ~+30% accuracy, and it
118
+ nearly vanishes when the stress tensor is directly supervised.
119
+ - **Out-of-distribution.** Accuracy degrades on geometries far from the training distribution (extreme
120
+ voids). The regularizer keeps *its own MLS* residual low OOD, but — per the cross-operator analysis —
121
+ that consistency gain is itself largely operator-specific, and it does **not** improve OOD accuracy.
122
 
123
  ### Recommendations
124
 
125
+ Use within the training distribution; verify any safety-critical prediction with FEM; treat the
126
+ equilibrium residual ‖∇·σ‖² as a physical-consistency indicator, not a correctness guarantee.
127
 
128
  ## How to Get Started with the Model
129
 
130
  ```python
131
  from huggingface_hub import snapshot_download
132
+ import sys
133
 
134
  repo = snapshot_download("Efradeca/transolver-linearno-elasticity")
135
+ sys.path.insert(0, repo) # model.py is bundled with the repo
136
+ from model import load_checkpoint, predict_stress
137
+
138
  model, normalizer, info = load_checkpoint(f"{repo}/model.safetensors", device="cpu")
139
 
140
  # coords: (N, 2) node coordinates of a unit cell with a central void
141
+ stress = predict_stress(model, coords, normalizer, info) # (N,) von Mises stress, CPU, < 50 ms
142
  ```
143
 
144
  ## Training Details
145
 
146
  ### Training Data
147
 
148
+ **Geo-FNO Elasticity** (Li et al., 2022, arXiv:2207.05209, MIT): FEM simulations of an incompressible
149
+ Rivlin–Saunders hyper-elastic (`C₁ = 1.863×10⁵`, `C₂ = 9.79×10³`) `[0,1]²` unit cell with a random central
150
+ void (radius 0.2–0.4), 972 nodes per sample; target = per-node von Mises stress. Split: **1000 train /
151
+ 200 test** (first-1000 / last-200 of 2000, the upstream Transolver protocol).
152
 
153
+ **Fixed boundary conditions (identical for every sample):** the bottom edge is **clamped** (zero
154
+ displacement) and a uniform **tensile traction `t = [0, 100]`** acts on the top edge (`+y`); left/right
155
+ edges are free. The model's only input is the 972 node coordinates — loads, supports, and material are
156
+ **not** inputs (they are constant; only the void geometry varies).
 
157
 
158
  ### Training Procedure
159
 
160
+ - **Preprocessing:** global z-score normalization of the target (de-normalized before the metric). The
161
+ regularized model outputs 3 stress-tensor channels and derives von Mises.
162
+ - **Hyperparameters:** 8 layers, hidden 128, 8 heads, `dim_head` 16, `M = 64` slices; `shared_qk` LinearNO
163
+ attention, 3-channel output. AdamW (lr 1e-3, weight decay 1e-5), cosine annealing, **500 epochs**, batch
164
+ size 1, gradient clipping 0.1.
165
+ - **Loss:** relative-L2 (on von Mises) + `λ·‖∇·σ‖²` on interior nodes, **λ = 0.01**, MLS divergence with
166
+ `k = 12` neighbors.
167
+ - **Compute:** single NVIDIA A10 GPU (Modal); total project compute $90. CPU inference for the demo.
 
 
 
 
 
168
 
169
  ## Evaluation
170
 
 
172
 
173
  - **Testing data:** the 200 held-out FEM meshes.
174
  - **Factors:** in-distribution vs. out-of-distribution (geometry-stratified by void size).
175
+ - **Metrics:** relative-L2 (primary); the discrete equilibrium residual ‖∇·σ‖²; pooled Pearson r and R²
176
+ vs. FEM; peak-stress relative error.
177
 
178
+ ### Results (Geo-FNO Elasticity test relative L2)
179
 
180
+ We report **median** (robust to this dataset's high run-to-run variance) alongside mean ± std and seed
181
+ count `n`. Accuracy-comparison rows use `torch.compile`; the equilibrium-regularized row is eager. Full
182
+ committed per-seed distributions are in the repository (`docs/RESULTS.md`, `results/`).
 
 
 
183
 
184
  | Model | median | mean ± std | n | params |
185
  |---|---|---|---|---|
186
  | Published Transolver (Wu et al. 2024) | — | 0.0064 | — | ~0.7M |
187
  | Published LinearNO (Hu et al. 2026, M=64) | — | 0.0050 | — | — |
188
+ | Transolver baseline (this work) | 0.00766 | 0.00827 ± 0.00222 | 6 | 713,665 |
189
+ | LinearNO, M=64 (this work) | **0.00606** | 0.00645 ± 0.00173 | 8 | 713,089 |
190
+ | LinearNO, M=256 (this work) | 0.00635 | 0.00622 ± 0.00071 | 3 | 765,313 |
191
+ | **+ equilibrium regularizer (this model)** | 0.00653 | 0.00668 ± 0.00065 | 3 | 582,275 |
192
+
193
+ **Comparable, not "beats."** In this sample LinearNO's median (0.00606) is lower than our reproduced
194
+ baseline's (0.00766) at fewer parameters, and it reaches the published ~0.0050 level on good seeds. But
195
+ both models have **high run-to-run variance** (`torch.compile` non-determinism on 1000 samples), and the
196
+ baseline's higher mean is driven by 2 outlier seeds. We do **not** claim LinearNO beats or is more robust
197
+ than the baseline; an attempted mechanistic explanation (slice-attention instability) was **empirically
198
+ tested and refuted**. Honest reading: LinearNO is **comparable** to the baseline at its parameters, both
199
+ reproducing ~0.0064 on good seeds. As a cross-check, we re-evaluated the authors' released official
200
+ checkpoint on our data with their code and reproduced **0.005007**confirming the dataset, split, and
201
+ metric are identical, so the table is mutually comparable.
202
+
203
+ **Verification against FEM ground truth** (200 test meshes; shipped checkpoint = best of 3 eqreg seeds):
204
+ per-sample relative-L2 mean **0.00597** (median 0.00533), pooled Pearson **r = 0.99993**, **R² = 0.99986**,
205
+ peak-stress relative error **0.27%** reproducible via `scripts/verify_model.py` `results/verify_model.json`.
206
+
207
+ ### Findings (the honest part)
208
+
209
+ **1. The meshfree MLS operator is rigorously grounded.** It is a degree-1 GMLS derivative: *exact on
210
+ affine fields* (~1e-14 in float64) and *O(h)-convergent* on smooth fields (fitted slope 1.01), validated
211
+ on connectivity-free point clouds — no mesh required.
212
+
213
+ ![MLS operator O(h) validation](fig5_mls_convergence.png)
214
+
215
+ **2. The headline residual reduction is largely operator-specific ("gaming").** On a clean,
216
+ same-architecture, epoch-matched ablation the regularizer cuts the residual **~325× as measured by the
217
+ model's own (MLS) operator**, but an **independent FE operator** sees only **~6–53×** at a **measured
218
+ accuracy cost** (test rel-L2 0.00479 0.00621, +30%). A model-free control on exactly-divergence-free
219
+ analytic fields confirms the FE operator is *comparable* to MLS off the void-rim slivers (the large raw
220
+ gap is a sliver artifact), so the honest operator-agnostic reduction (~53×) still sits **an order of
221
+ magnitude below** the trained-operator 325×. The gaming reproduces on the *literal* official LinearNO
222
+ block (gap ~25×), so it is a property of the soft-penalty objective, not our implementation.
223
+
224
+ ![cross-operator gaming](fig6_cross_operator.png)
225
+
226
+ **3. On a supervised tensor the effect nearly vanishes.** On a second benchmark, **Tensile2d** (PLAID),
227
+ where the full Cauchy tensor is *directly supervised*, the gaming gap collapses to **~1.3×** and the
228
+ regularizer becomes nearly redundant — the supervised data already pins the field near its
229
+ discrete-equilibrium floor. (This Tensile2d transfer is a **single-seed indicative** result and is **not**
230
+ comparable to the PLAID Tensile2d leaderboard, which scores a different RRMSE metric on a withheld test.)
231
+
232
+ **4. Out-of-distribution: a reported negative result.** The regularizer does **not** improve OOD accuracy
233
+ (degradation +51.5% vs. LinearNO's +52.8%; the plain baseline degrades least, +45.1%). It keeps *its own*
234
+ equilibrium residual low OOD (+7%), but on an orthogonal field-smoothness proxy it is indistinguishable
235
+ from the unregularized models.
236
+
237
+ **Takeaway:** the soft equilibrium penalty looks most impressive exactly where it is least physical (an
238
+ unsupervised latent tensor). We report this rather than paper over it. The deployed model's value is its
239
+ accuracy (R² = 0.99986) and millisecond CPU speed.
240
 
241
  ## Technical Specifications
242
 
243
  ### Model Architecture and Objective
244
 
245
+ Transolver-family transformer (coordinate encoder → 8 pre-norm blocks → linear decoder head) with the
246
+ LinearNO asymmetric linear-attention block `φ(Q)·(ψᵀ(K)·V)` softmax over the M slices, ψ over the N
247
+ points). Objective = relative-L2 data loss on von Mises + λ·‖∇·σ‖² equilibrium penalty (meshfree MLS
248
+ divergence). Output: 3 latent Cauchy-stress channels → von Mises.
249
 
250
  ### Compute Infrastructure
251
 
252
+ - **Hardware:** single NVIDIA A10 GPU (Modal) for training; CPU for demo inference.
253
+ - **Software:** PyTorch, einops, safetensors.
254
+
255
+ ## Licenses & Attribution
256
+
257
+ This model, its weights, and the bundled architecture code are **MIT**-licensed. It builds on and uses
258
+ data from the following sources — please honor their licenses:
259
+
260
+ - **Transolver** (Wu et al., ICML 2024) — attention block reused verbatim, **MIT**.
261
+ - **Geo-FNO Elasticity** dataset (Li et al., 2022) — the training/evaluation data, **MIT**.
262
+ - **Tensile2d** — the second-benchmark data ([`PLAID-datasets/Tensile2d`](https://huggingface.co/datasets/PLAID-datasets/Tensile2d),
263
+ Casenave et al.), licensed **CC-BY-SA-4.0**. Any derived Tensile2d fields we redistribute (e.g. the demo
264
+ data in the companion Space) are shared under the **same CC-BY-SA-4.0 (share-alike)** with attribution.
265
+ The MIT license of this model applies to the code and weights, **not** to the CC-BY-SA-4.0 Tensile2d data.
266
 
267
  ## Citation
268
 
269
+ If you use this model, please cite the underlying works.
270
+
271
  ```bibtex
272
  @inproceedings{wu2024transolver,
273
  title={Transolver: A Fast Transformer Solver for PDEs on General Geometries},
 
287
  journal={arXiv:2207.05209},
288
  year={2022}
289
  }
290
+ @article{guevaragarban2025pdivgnn,
291
+ title={Physics-Informed Graph Neural Networks to Reconstruct Local Fields Considering Finite Strain Hyperelasticity},
292
+ author={Guevara Garban, Manuel R. and Chemisky, Yves and others},
293
+ journal={arXiv:2507.05291},
294
+ year={2025}
295
+ }
296
  ```
297
 
298
+ ## Model Card Authors & Contact
 
 
 
 
299
 
300
+ Efradeca — https://huggingface.co/Efradeca
__pycache__/model.cpython-312.pyc CHANGED
Binary files a/__pycache__/model.cpython-312.pyc and b/__pycache__/model.cpython-312.pyc differ
 
fig5_mls_convergence.png ADDED

Git LFS Details

  • SHA256: 9fc8a8724979bd6f648cb3f69aafa8d84d6d3b9867b19452905a9850757756ea
  • Pointer size: 131 Bytes
  • Size of remote file: 151 kB
fig6_cross_operator.png ADDED

Git LFS Details

  • SHA256: 87514d31146bed1974043ec0e6b9c26ced7249837fad161b9d156cbdbe4aaae3
  • Pointer size: 131 Bytes
  • Size of remote file: 109 kB
stress_operator/data/tensile2d.py ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tensile2d (PLAID-datasets/Tensile2d, HF, CC-BY-SA-4.0) adapter — SECOND benchmark.
2
+
3
+ Why: unlike Geo-FNO Elasticity (scalar von Mises target -> our tensor is latent), Tensile2d directly
4
+ supervises the full 2D Cauchy stress tensor (sig11, sig22, sig12) per node on an irregular mesh, so our
5
+ equilibrium regularizer acts on a SUPERVISED tensor and we can validate the predicted tensor itself.
6
+
7
+ VERIFIED facts (golden rule, by reading real samples):
8
+ - Each sample is a pickled CGNS tree (no PLAID lib needed): GridCoordinates {CoordinateX, CoordinateY}
9
+ and PointData {sig11, sig22, sig12, U1, U2, q}; plus input scalars {P, p1..p5} and output scalars.
10
+ - 702 rows: samples 0-499 are the labeled `train_500` set (full stress); samples 500-701 (test 200 +
11
+ OOD 2) have stress fields WITHHELD (competition) -> only coords + input scalars. So we use the 500
12
+ labeled samples and hold out a local test split (no official test ground truth is public).
13
+ - Node count varies per sample (6143-11801, mean ~9400). Static, plane-strain, quasistatic.
14
+
15
+ `build_cache()` writes data/tensile2d/samples/*.npz once. `build_tensile_splits()` loads them, fits
16
+ z-score normalizers on the train split, and returns per-sample lists (variable N -> no stacking).
17
+ """
18
+ from __future__ import annotations
19
+
20
+ import glob
21
+ import os
22
+ from dataclasses import dataclass
23
+ from typing import List
24
+
25
+ import numpy as np
26
+ import torch
27
+
28
+ INPUT_SCALARS = ["P", "p1", "p2", "p3", "p4", "p5"]
29
+
30
+
31
+ def _field(tree, name):
32
+ stack = [tree]
33
+ while stack:
34
+ n = stack.pop()
35
+ if isinstance(n, list) and len(n) == 4:
36
+ if n[0] == name and isinstance(n[1], np.ndarray):
37
+ return n[1]
38
+ stack.extend(n[2] or [])
39
+ return None
40
+
41
+
42
+ def build_cache(out_dir: str = "data/tensile2d/samples") -> int:
43
+ """Download Tensile2d parquet shards and cache the 500 labeled samples as .npz. Returns count."""
44
+ from huggingface_hub import hf_hub_download
45
+ import pyarrow.parquet as pq
46
+ import pickle
47
+
48
+ os.makedirs(out_dir, exist_ok=True)
49
+ shards = [hf_hub_download("PLAID-datasets/Tensile2d",
50
+ f"data/all_samples-0000{i}-of-00002.parquet", repo_type="dataset")
51
+ for i in (0, 1)]
52
+ idx = 0
53
+ saved = 0
54
+ for sh in shards:
55
+ col = pq.read_table(sh).column("sample")
56
+ for i in range(len(col)):
57
+ try:
58
+ obj = pickle.loads(col[i].as_py())
59
+ tree = list(obj["meshes"].values())[0]
60
+ x, y = _field(tree, "CoordinateX"), _field(tree, "CoordinateY")
61
+ s11, s22, s12 = _field(tree, "sig11"), _field(tree, "sig22"), _field(tree, "sig12")
62
+ if any(v is None for v in (x, y, s11, s22, s12)):
63
+ idx += 1
64
+ continue # withheld test/OOD sample
65
+ coords = np.stack([x, y], 1).astype(np.float32)
66
+ sigma = np.stack([s11, s22, s12], 1).astype(np.float32)
67
+ vm = np.sqrt(s11 ** 2 - s11 * s22 + s22 ** 2 + 3 * s12 ** 2).astype(np.float32)
68
+ sc = obj.get("scalars", {})
69
+ scin = np.array([float(sc[k]) for k in INPUT_SCALARS], np.float32)
70
+ np.savez(os.path.join(out_dir, f"{saved:04d}.npz"),
71
+ coords=coords, sigma=sigma, vm=vm, scalars_in=scin)
72
+ saved += 1
73
+ except Exception:
74
+ pass
75
+ idx += 1
76
+ return saved
77
+
78
+
79
+ @dataclass
80
+ class TensileSplits:
81
+ train_coords: List[torch.Tensor]
82
+ train_sigma: List[torch.Tensor] # scaled by 1/S (N,3)
83
+ train_scalars: List[torch.Tensor] # z-scored (6,)
84
+ test_coords: List[torch.Tensor]
85
+ test_sigma: List[torch.Tensor]
86
+ test_scalars: List[torch.Tensor]
87
+ scale_S: float # SINGLE global stress scale; de-normalize: sigma*S
88
+ scalar_mean: torch.Tensor # (6,)
89
+ scalar_std: torch.Tensor
90
+
91
+
92
+ def build_tensile_splits(samples_dir: str = "data/tensile2d/samples",
93
+ ntrain: int = 400, ntest: int = 100, seed: int = 0) -> TensileSplits:
94
+ """Load cached samples; deterministic train/test split; z-score normalizers fit on TRAIN only.
95
+
96
+ Coords are kept RAW (physical units) so the MLS divergence operator measures true distances; the
97
+ model conditions on raw coords + normalized input scalars (the encoder MLP handles the coord scale).
98
+ Stress is z-scored per component; input scalars z-scored per dim.
99
+ """
100
+ files = sorted(glob.glob(os.path.join(samples_dir, "*.npz")))
101
+ if not files:
102
+ raise FileNotFoundError(f"no cached Tensile2d samples in {samples_dir} — run build_cache() first")
103
+ rng = np.random.default_rng(seed)
104
+ order = rng.permutation(len(files))
105
+ tr_idx, te_idx = order[:ntrain], order[ntrain:ntrain + ntest]
106
+
107
+ def load(i):
108
+ d = np.load(files[i])
109
+ return d["coords"], d["sigma"], d["scalars_in"]
110
+
111
+ tr = [load(i) for i in tr_idx]
112
+ te = [load(i) for i in te_idx]
113
+
114
+ # SINGLE global stress scale S (RMS of all train stress components) so the divergence regularizer
115
+ # de-normalizes with one scalar: sigma_phys = S * sigma_norm => div(sigma_phys) = S * div(sigma_norm).
116
+ # (Per-component z-score would scale each channel differently and corrupt the physical divergence.)
117
+ sig_all = np.concatenate([s for _, s, _ in tr], 0) # (sum N, 3)
118
+ S = float(np.sqrt((sig_all ** 2).mean())) # global RMS scale
119
+ sc_all = np.stack([sc for _, _, sc in tr], 0) # (ntrain, 6)
120
+ sc_mean, sc_std = sc_all.mean(0), sc_all.std(0) + 1e-8
121
+
122
+ def pack(rows):
123
+ cs, ss, scs = [], [], []
124
+ for c, s, sc in rows:
125
+ cs.append(torch.tensor(c))
126
+ ss.append(torch.tensor(s / S))
127
+ scs.append(torch.tensor((sc - sc_mean) / sc_std))
128
+ return cs, ss, scs
129
+
130
+ trc, trs, trsc = pack(tr)
131
+ tec, tes, tesc = pack(te)
132
+ return TensileSplits(
133
+ train_coords=trc, train_sigma=trs, train_scalars=trsc,
134
+ test_coords=tec, test_sigma=tes, test_scalars=tesc,
135
+ scale_S=S,
136
+ scalar_mean=torch.tensor(sc_mean), scalar_std=torch.tensor(sc_std),
137
+ )
stress_operator/losses/equilibrium.py CHANGED
@@ -40,7 +40,7 @@ def knn_indices(coords: np.ndarray, k: int) -> Tuple[np.ndarray, np.ndarray]:
40
 
41
 
42
  def build_mls_gradient_operators(
43
- coords: torch.Tensor, k: int = 12, eps: float = 1e-9
44
  ) -> Tuple[torch.Tensor, torch.Tensor]:
45
  """Build dense (N, N) operators Gx, Gy with (Gx @ f) ~ df/dx and (Gy @ f) ~ df/dy.
46
 
@@ -67,11 +67,44 @@ def build_mls_gradient_operators(
67
  Gy[i, nbr] += C[:, 1]
68
  Gy[i, i] -= C[:, 1].sum()
69
  return (
70
- torch.tensor(Gx, dtype=torch.float32, device=coords.device),
71
- torch.tensor(Gy, dtype=torch.float32, device=coords.device),
72
  )
73
 
74
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
  def build_graph_laplacian(coords: torch.Tensor, k: int = 12, eps: float = 1e-9) -> torch.Tensor:
76
  """Unnormalized graph Laplacian L = D - W with Gaussian kNN weights (Approach B)."""
77
  cpu = coords.detach().cpu().numpy().astype(np.float64)
@@ -93,11 +126,30 @@ def interior_mask(coords: torch.Tensor, tol: float = 0.03) -> torch.Tensor:
93
 
94
  The outer-box boundary carries the applied traction / clamp, where div(sigma)=0 need not
95
  hold; interior nodes (including those around the void) are valid for the constraint.
 
 
96
  """
97
  x, y = coords[..., 0], coords[..., 1]
98
  return (x > tol) & (x < 1 - tol) & (y > tol) & (y < 1 - tol)
99
 
100
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
101
  def divergence(stress3: torch.Tensor, Gx: torch.Tensor, Gy: torch.Tensor) -> torch.Tensor:
102
  """Divergence of the symmetric 2x2 stress tensor field.
103
 
 
40
 
41
 
42
  def build_mls_gradient_operators(
43
+ coords: torch.Tensor, k: int = 12, eps: float = 1e-9, dtype: torch.dtype = torch.float32
44
  ) -> Tuple[torch.Tensor, torch.Tensor]:
45
  """Build dense (N, N) operators Gx, Gy with (Gx @ f) ~ df/dx and (Gy @ f) ~ df/dy.
46
 
 
67
  Gy[i, nbr] += C[:, 1]
68
  Gy[i, i] -= C[:, 1].sum()
69
  return (
70
+ torch.tensor(Gx, dtype=dtype, device=coords.device),
71
+ torch.tensor(Gy, dtype=dtype, device=coords.device),
72
  )
73
 
74
 
75
+ def build_mls_gradient_operators_sparse(coords: torch.Tensor, k: int = 12, eps: float = 1e-9):
76
+ """Sparse (N,N) MLS gradient operators, built directly without the dense intermediate.
77
+
78
+ Identical stencils to build_mls_gradient_operators (k+1 nonzeros per row) but assembled as COO,
79
+ so it scales to large meshes (e.g. Tensile2d's ~9k-12k nodes, where a dense (N,N) is ~0.7 GB).
80
+ """
81
+ cpu = coords.detach().cpu().numpy().astype(np.float64)
82
+ n = cpu.shape[0]
83
+ dist, idx = knn_indices(cpu, k)
84
+ sigma2 = float((dist.mean()) ** 2) + eps
85
+ rows = np.empty(n * (k + 1), dtype=np.int64)
86
+ cols = np.empty(n * (k + 1), dtype=np.int64)
87
+ vx = np.empty(n * (k + 1), dtype=np.float64)
88
+ vy = np.empty(n * (k + 1), dtype=np.float64)
89
+ p = 0
90
+ for i in range(n):
91
+ nbr = idx[i]
92
+ d = cpu[nbr] - cpu[i]
93
+ w = np.exp(-(d ** 2).sum(1) / sigma2)
94
+ A = np.einsum("k,ka,kb->ab", w, d, d)
95
+ C = (w[:, None] * d) @ np.linalg.pinv(A) # (k, 2)
96
+ m = k + 1
97
+ rows[p:p + m] = i
98
+ cols[p:p + k] = nbr; cols[p + k] = i
99
+ vx[p:p + k] = C[:, 0]; vx[p + k] = -C[:, 0].sum()
100
+ vy[p:p + k] = C[:, 1]; vy[p + k] = -C[:, 1].sum()
101
+ p += m
102
+ ij = torch.tensor(np.stack([rows, cols]), device=coords.device)
103
+ Gx = torch.sparse_coo_tensor(ij, torch.tensor(vx, dtype=torch.float32), (n, n)).coalesce()
104
+ Gy = torch.sparse_coo_tensor(ij, torch.tensor(vy, dtype=torch.float32), (n, n)).coalesce()
105
+ return Gx.to(coords.device), Gy.to(coords.device)
106
+
107
+
108
  def build_graph_laplacian(coords: torch.Tensor, k: int = 12, eps: float = 1e-9) -> torch.Tensor:
109
  """Unnormalized graph Laplacian L = D - W with Gaussian kNN weights (Approach B)."""
110
  cpu = coords.detach().cpu().numpy().astype(np.float64)
 
126
 
127
  The outer-box boundary carries the applied traction / clamp, where div(sigma)=0 need not
128
  hold; interior nodes (including those around the void) are valid for the constraint.
129
+ NOTE: assumes the Geo-FNO Elasticity [0,1]^2 unit cell. For arbitrary geometries (e.g. Tensile2d
130
+ on [-1,1]^2 with an irregular free surface) use ``interior_mask_knn`` instead.
131
  """
132
  x, y = coords[..., 0], coords[..., 1]
133
  return (x > tol) & (x < 1 - tol) & (y > tol) & (y < 1 - tol)
134
 
135
 
136
+ def interior_mask_knn(coords: torch.Tensor, k: int = 12, thresh: float = 0.5) -> torch.Tensor:
137
+ """Geometry-agnostic, connectivity-free interior mask via k-NN stencil symmetry.
138
+
139
+ A node deep in the interior has neighbors roughly symmetric around it, so the offset of its
140
+ k-NN centroid is small relative to the mean neighbor distance; a boundary/free-surface node has
141
+ one-sided neighbors and a large centroid offset (where the MLS gradient is also least reliable).
142
+ Interior iff ``||mean(neighbors) - node|| / mean_neighbor_dist < thresh``. Works for any mesh.
143
+ """
144
+ cpu = coords.detach().cpu().numpy().astype(np.float64)
145
+ dist, idx = knn_indices(cpu, k)
146
+ nbr = cpu[idx] # (N, k, 2)
147
+ centroid = nbr.mean(axis=1) # (N, 2)
148
+ offset = np.linalg.norm(centroid - cpu, axis=1) # (N,)
149
+ scale = dist.mean(axis=1) + 1e-12 # (N,)
150
+ return torch.tensor((offset / scale) < thresh, device=coords.device)
151
+
152
+
153
  def divergence(stress3: torch.Tensor, Gx: torch.Tensor, Gy: torch.Tensor) -> torch.Tensor:
154
  """Divergence of the symmetric 2x2 stress tensor field.
155
 
stress_operator/losses/fe_divergence.py ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """FE shape-function divergence operator (the P-DivGNN-style operator), for the head-to-head ablation.
2
+
3
+ P-DivGNN computes div(sigma) with **FE P1 shape-function gradients on a mesh**; we use a **meshfree MLS**
4
+ gradient that needs no connectivity. To isolate that single difference, this module builds an FE nodal
5
+ gradient operator on the SAME points (via a Delaunay triangulation), so we can compare both operators'
6
+ divergence residuals on identical predicted stress fields. The FE operator REQUIRES connectivity (the
7
+ triangulation); the MLS operator does not — that is the differentiating point.
8
+
9
+ Build is analogous to build_mls_gradient_operators: returns dense (N,N) Gx_fe, Gy_fe with
10
+ (Gx_fe @ f) ~ df/dx via area-weighted recovery of per-triangle P1 gradients. Exact on affine fields
11
+ (P1 elements reproduce linear fields exactly) — verified in tests.
12
+ """
13
+ from __future__ import annotations
14
+
15
+ import numpy as np
16
+ import torch
17
+ from scipy.spatial import Delaunay
18
+
19
+
20
+ def build_fe_gradient_operators(coords: torch.Tensor):
21
+ """Dense (N,N) FE nodal-gradient operators from a Delaunay triangulation of `coords` (N,2)."""
22
+ pts = coords.detach().cpu().numpy().astype(np.float64)
23
+ n = pts.shape[0]
24
+ tri = Delaunay(pts).simplices # (T,3)
25
+ Gx = np.zeros((n, n)); Gy = np.zeros((n, n)); wsum = np.zeros(n)
26
+ for t in tri:
27
+ i0, i1, i2 = t
28
+ p0, p1, p2 = pts[i0], pts[i1], pts[i2]
29
+ # 2*signed area
30
+ det = (p1[0] - p0[0]) * (p2[1] - p0[1]) - (p2[0] - p0[0]) * (p1[1] - p0[1])
31
+ if abs(det) < 1e-14:
32
+ continue
33
+ area = 0.5 * abs(det)
34
+ # P1 shape-function gradients (constant per triangle): grad N_i = (b_i, c_i)/(2A_signed)
35
+ # b_i = y_{i+1}-y_{i+2}, c_i = x_{i+2}-x_{i+1} (cyclic), using signed 2A=det
36
+ bs = [p1[1] - p2[1], p2[1] - p0[1], p0[1] - p1[1]]
37
+ cs = [p2[0] - p1[0], p0[0] - p2[0], p1[0] - p0[0]]
38
+ for local, node in enumerate(t):
39
+ gx = bs[local] / det
40
+ gy = cs[local] / det
41
+ # area-weighted nodal recovery: each incident node gets this triangle's (constant) gradient
42
+ for lv, nv in enumerate(t):
43
+ Gx[node, nv] += area * (bs[lv] / det)
44
+ Gy[node, nv] += area * (cs[lv] / det)
45
+ wsum[node] += area
46
+ nz = wsum > 0
47
+ Gx[nz] /= wsum[nz, None]
48
+ Gy[nz] /= wsum[nz, None]
49
+ return (torch.tensor(Gx, dtype=torch.float64), torch.tensor(Gy, dtype=torch.float64))
50
+
51
+
52
+ def build_fe_gradient_operators_sparse(coords: torch.Tensor):
53
+ """Sparse FE nodal-gradient operators (same stencils as the dense builder), for large meshes."""
54
+ pts = coords.detach().cpu().numpy().astype(np.float64)
55
+ n = pts.shape[0]
56
+ tri = Delaunay(pts).simplices
57
+ rows, cols, vx, vy = [], [], [], []
58
+ wsum = np.zeros(n)
59
+ for t in tri:
60
+ i0, i1, i2 = t
61
+ p0, p1, p2 = pts[i0], pts[i1], pts[i2]
62
+ det = (p1[0] - p0[0]) * (p2[1] - p0[1]) - (p2[0] - p0[0]) * (p1[1] - p0[1])
63
+ if abs(det) < 1e-14:
64
+ continue
65
+ area = 0.5 * abs(det)
66
+ bs = [p1[1] - p2[1], p2[1] - p0[1], p0[1] - p1[1]]
67
+ cs = [p2[0] - p1[0], p0[0] - p2[0], p1[0] - p0[0]]
68
+ for node in t:
69
+ wsum[node] += area
70
+ for lv, nv in enumerate(t):
71
+ rows.append(node); cols.append(nv)
72
+ vx.append(area * bs[lv] / det); vy.append(area * cs[lv] / det)
73
+ ij = torch.tensor(np.array([rows, cols]))
74
+ Gx = torch.sparse_coo_tensor(ij, torch.tensor(vx), (n, n)).coalesce()
75
+ Gy = torch.sparse_coo_tensor(ij, torch.tensor(vy), (n, n)).coalesce()
76
+ w = torch.tensor(np.where(wsum > 0, wsum, 1.0))
77
+ # normalize each row i by wsum[i]
78
+ Gx = torch.sparse_coo_tensor(Gx.indices(), Gx.values() / w[Gx.indices()[0]], (n, n)).coalesce()
79
+ Gy = torch.sparse_coo_tensor(Gy.indices(), Gy.values() / w[Gy.indices()[0]], (n, n)).coalesce()
80
+ return Gx.to(coords.device), Gy.to(coords.device)
stress_operator/models/linear_no.py CHANGED
@@ -3,7 +3,11 @@
3
  Reimplemented from the equations of:
4
  "Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of
5
  Linear Attention" — Hu, Liu, Qiao, Sun, Dou (NUDT), AAAI 2026, arXiv:2511.06294.
6
- There is no official public repo; only the attention block differs from Transolver.
 
 
 
 
7
 
8
  Core idea: Physics-Attention is the special case of linear attention
9
  ``Attention(Q,K,V) ~ phi(Q) (psi^T(K) V)`` in which (a) phi and psi come from the SAME linear
@@ -50,12 +54,19 @@ class LinearNO(nn.Module):
50
  dropout=0.0,
51
  variant: str = "independent",
52
  project_out: bool = True,
 
53
  ):
54
  super().__init__()
55
  inner = heads * dim_head
56
  self.h, self.m, self.dh = heads, slice_num, dim_head
57
  self.variant = variant
58
  self.project_out = project_out
 
 
 
 
 
 
59
 
60
  if variant == "independent":
61
  # Independent Q, K, V projections (paper modification 1, literal).
@@ -93,8 +104,13 @@ class LinearNO(nn.Module):
93
  k = base
94
  v = self._heads(self.to_v(x), B, N)
95
 
96
- phi = self.lin_q(q).softmax(dim=-1) # (B,H,N,M) softmax OVER M (slices) <- rows sum to 1
97
- psi = self.lin_k(k).softmax(dim=-2) # (B,H,N,M) softmax OVER N (points) <- cols sum to 1
 
 
 
 
 
98
  kv = torch.einsum("bhnm,bhnd->bhmd", psi, v) # (B,H,M,dh) cheap inner product first
99
  out = torch.einsum("bhnm,bhmd->bhnd", phi, kv) # (B,H,N,dh) linear in N
100
  out = rearrange(out, "b h n d -> b n (h d)")
 
3
  Reimplemented from the equations of:
4
  "Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of
5
  Linear Attention" — Hu, Liu, Qiao, Sun, Dou (NUDT), AAAI 2026, arXiv:2511.06294.
6
+ Reconstructed from the equations BEFORE the official code was released (github.com/HiPRL/LinearNO,
7
+ Jan 2026); we did not consult it. A post-hoc check confirms structural agreement (softmax axes,
8
+ associativity). Differences from the official Elasticity block: it adds a learnable per-head softmax
9
+ temperature (init 0.5, clamped [0.01,1]) that we omit, and uses one shared in_project_x lift + small
10
+ per-head q/k/v maps vs our full C->inner projections. Only the attention block differs from Transolver.
11
 
12
  Core idea: Physics-Attention is the special case of linear attention
13
  ``Attention(Q,K,V) ~ phi(Q) (psi^T(K) V)`` in which (a) phi and psi come from the SAME linear
 
54
  dropout=0.0,
55
  variant: str = "independent",
56
  project_out: bool = True,
57
+ temperature: bool = False,
58
  ):
59
  super().__init__()
60
  inner = heads * dim_head
61
  self.h, self.m, self.dh = heads, slice_num, dim_head
62
  self.variant = variant
63
  self.project_out = project_out
64
+ self.temperature = temperature
65
+ if temperature:
66
+ # Matches the official LinearNO `temp` block: learnable per-head temperature on both
67
+ # softmaxes, init 0.5, clamped to [0.01, 1] (github.com/HiPRL/LinearNO).
68
+ self.temp_q = nn.Parameter(torch.ones(1, heads, 1, 1) * 0.5)
69
+ self.temp_k = nn.Parameter(torch.ones(1, heads, 1, 1) * 0.5)
70
 
71
  if variant == "independent":
72
  # Independent Q, K, V projections (paper modification 1, literal).
 
104
  k = base
105
  v = self._heads(self.to_v(x), B, N)
106
 
107
+ sq = self.lin_q(q) # (B,H,N,M)
108
+ sk = self.lin_k(k) # (B,H,N,M)
109
+ if self.temperature:
110
+ sq = sq / self.temp_q.clamp(0.01, 1.0)
111
+ sk = sk / self.temp_k.clamp(0.01, 1.0)
112
+ phi = sq.softmax(dim=-1) # softmax OVER M (slices) <- rows sum to 1
113
+ psi = sk.softmax(dim=-2) # softmax OVER N (points) <- cols sum to 1
114
  kv = torch.einsum("bhnm,bhnd->bhmd", psi, v) # (B,H,M,dh) cheap inner product first
115
  out = torch.einsum("bhnm,bhmd->bhnd", phi, kv) # (B,H,N,dh) linear in N
116
  out = rearrange(out, "b h n d -> b n (h d)")
stress_operator/models/transolver.py CHANGED
@@ -27,6 +27,7 @@ def make_attention(
27
  slice_num,
28
  linearno_variant: str = "shared_qk",
29
  linearno_project_out: bool = False,
 
30
  ) -> nn.Module:
31
  if kind == "physics":
32
  return Physics_Attention_Irregular_Mesh(
@@ -43,6 +44,7 @@ def make_attention(
43
  dropout=dropout,
44
  variant=linearno_variant,
45
  project_out=linearno_project_out,
 
46
  )
47
  raise ValueError(f"unknown attention kind {kind!r} (expected 'physics' or 'linearno')")
48
 
@@ -66,6 +68,7 @@ class StressOperator(nn.Module):
66
  act: str = "gelu",
67
  linearno_variant: str = "shared_qk",
68
  linearno_project_out: bool = False,
 
69
  ):
70
  super().__init__()
71
  if dim_head is None:
@@ -85,6 +88,7 @@ class StressOperator(nn.Module):
85
  attention, n_hidden, n_heads, dim_head, dropout, slice_num,
86
  linearno_variant=linearno_variant,
87
  linearno_project_out=linearno_project_out,
 
88
  ),
89
  hidden_dim=n_hidden,
90
  dropout=dropout,
@@ -152,6 +156,7 @@ def build_model(model_cfg: dict) -> StressOperator:
152
  act=model_cfg.get("act", "gelu"),
153
  linearno_variant=model_cfg.get("linearno_variant", "shared_qk"),
154
  linearno_project_out=model_cfg.get("linearno_project_out", False),
 
155
  )
156
 
157
 
 
27
  slice_num,
28
  linearno_variant: str = "shared_qk",
29
  linearno_project_out: bool = False,
30
+ linearno_temperature: bool = False,
31
  ) -> nn.Module:
32
  if kind == "physics":
33
  return Physics_Attention_Irregular_Mesh(
 
44
  dropout=dropout,
45
  variant=linearno_variant,
46
  project_out=linearno_project_out,
47
+ temperature=linearno_temperature,
48
  )
49
  raise ValueError(f"unknown attention kind {kind!r} (expected 'physics' or 'linearno')")
50
 
 
68
  act: str = "gelu",
69
  linearno_variant: str = "shared_qk",
70
  linearno_project_out: bool = False,
71
+ linearno_temperature: bool = False,
72
  ):
73
  super().__init__()
74
  if dim_head is None:
 
88
  attention, n_hidden, n_heads, dim_head, dropout, slice_num,
89
  linearno_variant=linearno_variant,
90
  linearno_project_out=linearno_project_out,
91
+ linearno_temperature=linearno_temperature,
92
  ),
93
  hidden_dim=n_hidden,
94
  dropout=dropout,
 
156
  act=model_cfg.get("act", "gelu"),
157
  linearno_variant=model_cfg.get("linearno_variant", "shared_qk"),
158
  linearno_project_out=model_cfg.get("linearno_project_out", False),
159
+ linearno_temperature=model_cfg.get("linearno_temperature", False),
160
  )
161
 
162
 
stress_operator/train_tensile.py ADDED
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tensile2d training with the equilibrium regularizer — the SUPERVISED-tensor transfer experiment.
2
+
3
+ Unlike Geo-FNO Elasticity (scalar von Mises target -> latent tensor), Tensile2d directly supervises the
4
+ full Cauchy tensor (sig11, sig22, sig12) per node. So here:
5
+ - the data loss is relative-L2 on the 3 SUPERVISED stress components (not a derived von Mises);
6
+ - the model conditions on the 6 input scalars (P, p1..p5) via fun_dim=6 (broadcast per node);
7
+ - the physics loss is the same meshfree MLS divergence ||div(sigma)||^2 on interior nodes.
8
+ Meshes are variable-size (~6k-12k nodes), so we keep per-sample lists (batch_size=1).
9
+
10
+ This run lets us test the key question raised by the Elasticity cross-operator finding: does supervising
11
+ the true tensor REDUCE the operator-specific 'gaming' (MLS-vs-FE residual gap)? (Checked post-hoc.)
12
+ """
13
+ from __future__ import annotations
14
+
15
+ import os
16
+ import time
17
+ from typing import Any, Dict, Optional
18
+
19
+ import torch
20
+
21
+ from .data.tensile2d import build_tensile_splits
22
+ from .losses.equilibrium import build_mls_gradient_operators_sparse, interior_mask_knn
23
+ from .losses.relative_l2 import relative_l2
24
+ from .models.transolver import build_model, count_parameters
25
+ from .seeds import set_seed
26
+ from .utils.logging import MODAL_RATES_PER_SEC, write_run_log
27
+
28
+
29
+ def _precompute_ops(coords_list, k, device):
30
+ ops = []
31
+ for c in coords_list:
32
+ Gx, Gy = build_mls_gradient_operators_sparse(c, k=k) # sparse-direct (scales to ~10k nodes)
33
+ ops.append((Gx.to(device), Gy.to(device), interior_mask_knn(c, k=k).to(device)))
34
+ return ops
35
+
36
+
37
+ def _div_residual(stress3, Gx_s, Gy_s, mask):
38
+ sxx, syy, sxy = stress3[:, 0:1], stress3[:, 1:2], stress3[:, 2:3]
39
+ dx = torch.sparse.mm(Gx_s, sxx) + torch.sparse.mm(Gy_s, sxy)
40
+ dy = torch.sparse.mm(Gx_s, sxy) + torch.sparse.mm(Gy_s, syy)
41
+ return (dx.squeeze(-1) ** 2 + dy.squeeze(-1) ** 2)[mask].mean()
42
+
43
+
44
+ def run_training_tensile(
45
+ config: Dict[str, Any], seed: int, data_dir: str = "data/tensile2d/samples",
46
+ device: Optional[str] = None, gpu_name: str = "CPU",
47
+ results_path: Optional[str] = None, ckpt_path: Optional[str] = None,
48
+ log_every: int = 25, max_epochs: Optional[int] = None,
49
+ splits=None, lambda_override: Optional[float] = None,
50
+ ) -> Dict[str, Any]:
51
+ device = device or ("cuda" if torch.cuda.is_available() else "cpu")
52
+ set_seed(seed)
53
+ tr_cfg, model_cfg, eq_cfg = config["train"], config["model"], config["equilibrium"]
54
+ lam = float(lambda_override) if lambda_override is not None else float(eq_cfg.get("lambda", 0.01))
55
+ k = int(eq_cfg.get("knn_k", 12)) # interior_mask_knn is geometry-agnostic; no interior_tol needed
56
+
57
+ if splits is None:
58
+ d = config["data"]
59
+ splits = build_tensile_splits(data_dir, ntrain=d.get("ntrain", 400), ntest=d.get("ntest", 100), seed=0)
60
+ S = splits.scale_S
61
+ trc = [c.to(device) for c in splits.train_coords]
62
+ trs = [s.to(device).float() for s in splits.train_sigma]
63
+ trsc = [sc.to(device).float() for sc in splits.train_scalars]
64
+ tec = [c.to(device) for c in splits.test_coords]
65
+ tes = [s.to(device).float() for s in splits.test_sigma]
66
+ tesc = [sc.to(device).float() for sc in splits.test_scalars]
67
+ n_train, n_test = len(trc), len(tec)
68
+
69
+ print(f"[tensile seed {seed}] lambda={lam} k={k} S={S:.2f} ntrain={n_train} ntest={n_test}", flush=True)
70
+ t_build = time.time()
71
+ train_ops = _precompute_ops(splits.train_coords, k, device)
72
+ test_ops = _precompute_ops(splits.test_coords, k, device)
73
+ print(f"[tensile seed {seed}] precomputed operators in {time.time()-t_build:.0f}s", flush=True)
74
+
75
+ model = build_model(model_cfg).to(device)
76
+ n_params = count_parameters(model)
77
+ epochs = max_epochs or int(tr_cfg.get("epochs", 500))
78
+ opt = torch.optim.AdamW(model.parameters(), lr=float(tr_cfg.get("lr", 1e-3)),
79
+ weight_decay=float(tr_cfg.get("weight_decay", 1e-5)),
80
+ betas=tuple(tr_cfg.get("betas", (0.9, 0.999))))
81
+ sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, T_max=epochs)
82
+ mgn = tr_cfg.get("max_grad_norm", None)
83
+ eval_every = int(tr_cfg.get("eval_every", 10))
84
+ gen = torch.Generator().manual_seed(seed)
85
+
86
+ def fx_of(coords, scalars): # broadcast the 6 input scalars to every node
87
+ return scalars.view(1, 1, -1).expand(1, coords.shape[0], -1)
88
+
89
+ def losses(coords, sigma, scalars, ops_i):
90
+ out = model(coords.unsqueeze(0), fx_of(coords, scalars))[0] # (N,3) scaled stress
91
+ data = relative_l2(out.unsqueeze(0), sigma.unsqueeze(0), reduction="mean") # SUPERVISED tensor
92
+ phys = _div_residual(out, *ops_i)
93
+ return data, phys
94
+
95
+ @torch.no_grad()
96
+ def evaluate():
97
+ model.eval()
98
+ d, p = 0.0, 0.0
99
+ for i in range(n_test):
100
+ dl, ph = losses(tec[i], tes[i], tesc[i], test_ops[i])
101
+ d += dl.item(); p += ph.item()
102
+ return d / n_test, p / n_test
103
+
104
+ t0 = time.time()
105
+ best, test_rel, test_phys, hist = float("inf"), float("nan"), float("nan"), []
106
+ for ep in range(epochs):
107
+ model.train()
108
+ run = 0.0
109
+ for i in torch.randperm(n_train, generator=gen).tolist():
110
+ opt.zero_grad()
111
+ dl, ph = losses(trc[i], trs[i], trsc[i], train_ops[i])
112
+ (dl + lam * ph).backward()
113
+ if mgn is not None:
114
+ torch.nn.utils.clip_grad_norm_(model.parameters(), mgn)
115
+ opt.step(); run += dl.item()
116
+ sched.step()
117
+ train_rel = run / n_train
118
+ if (ep % eval_every == 0) or (ep >= epochs - 5):
119
+ test_rel, test_phys = evaluate(); best = min(best, test_rel)
120
+ hist.append({"epoch": ep, "train_rel": train_rel, "test_rel": test_rel, "test_phys": test_phys})
121
+ if ep % log_every == 0 or ep == epochs - 1:
122
+ print(f"[tensile seed {seed}] epoch {ep:4d} train_rel={train_rel:.5f} "
123
+ f"test_rel={test_rel:.5f} test_resid={test_phys:.4e}", flush=True)
124
+
125
+ wall = time.time() - t0
126
+ rate = MODAL_RATES_PER_SEC.get(gpu_name, 0.0)
127
+ metrics = {
128
+ "test_rel_l2": round(test_rel, 6), "best_test_rel_l2": round(best, 6),
129
+ "test_residual": test_phys * (S ** 2), "test_residual_scaled": test_phys,
130
+ "scale_S": S, "train_rel_l2": round(train_rel, 6),
131
+ "n_params": n_params, "epochs": epochs, "lambda": lam, "dataset": "tensile2d",
132
+ }
133
+ if ckpt_path is not None:
134
+ os.makedirs(os.path.dirname(ckpt_path) or ".", exist_ok=True)
135
+ torch.save({"state_dict": model.state_dict(), "scale_S": S, "config": config,
136
+ "scalar_mean": splits.scalar_mean, "scalar_std": splits.scalar_std,
137
+ "seed": seed, "metrics": metrics}, ckpt_path)
138
+ if results_path is None:
139
+ results_path = os.path.join("results", f"{config.get('name','tensile')}_seed{seed}.json")
140
+ write_run_log(results_path, config, seed, metrics, wall, gpu_name, wall * rate,
141
+ extra={"history_tail": hist[-5:]})
142
+ print(f"[tensile seed {seed}] DONE test_rel={test_rel:.6f} resid={test_phys:.4e} "
143
+ f"wall={wall:.0f}s est_cost=${wall*rate:.4f}", flush=True)
144
+ return metrics