# Scorecard — Apple arm64 (MPS)

400 steps, batch 4, seed 20260803, torch 2.13.0, torch-dimensions 0.3.1.

One task — a cumulative sum along an axis — on one machine. That suits a
causal sequence model and is close to the worst case for a permutation-
invariant one, so this ranks models *on this problem* and implies nothing
about images or forecasting. There is no combined score on purpose: the
weighting between a point of loss and a step per second is the reader's,
not this file's.

| model | params | final loss | best | 90% at | tail spread | steps/s | Δloss per 1k params |
|---|---|---|---|---|---|---|---|
| `mamba_upstream_3d` | 50,752 | 0.0042 | 0.0035 | step 53 | 0.112 | 25.6 | 0.054 |
| `lstm_3d` | 51,136 | 0.0045 | 0.0011 | step 39 | 0.655 | 59.2 | 0.077 |
| `mamba3_2d` | 141,472 | 0.6098 | 0.4131 | step 22 | 0.122 | 12.0 | 0.026 |
| `gru_2d_sparse` | 38,464 | 0.6109 | 0.4073 | step 47 | 0.127 | 30.4 | 0.084 |
| `mamba_upstream_2d` | 33,856 | 0.6111 | 0.4048 | step 78 | 0.126 | 35.2 | 0.087 |
| `lstm_2d_sparse` | 51,136 | 0.6116 | 0.4030 | step 47 | 0.131 | 70.5 | 0.059 |
| `mamba_portable_2d` | 33,856 | 0.6124 | 0.4054 | step 78 | 0.126 | 39.9 | 0.087 |
| `mamba2_2d` | 111,920 | 0.6143 | 0.4340 | step 91 | 0.119 | 5.4 | 0.019 |
| `s4_upstream_2d` | 17,216 | 0.6227 | 0.4246 | step 85 | 0.124 | 35.5 | 0.152 |
| `s4d_upstream_2d` | 15,168 | 0.6252 | 0.4232 | step 85 | 0.123 | 41.3 | 0.175 |
| `s4d_portable_2d` | 13,120 | 0.6378 | 0.4188 | step 91 | 0.125 | 56.7 | 0.206 |
| `tcn_2d_sparse` | 25,152 | 1.0288 | 0.7823 | step 45 | 0.110 | 81.3 | 0.098 |
| `transformer_cafa_2d` | 55,376 | 1.4103 | 1.0995 | step 25 | 0.082 | 10.1 | 0.206 |
| `transformer_scan_2d` | 34,112 | 1.6962 | 1.3074 | step 74 | 0.080 | 49.8 | 0.049 |
| `cnn_2d_sparse` | 12,736 | 2.2291 | 1.5628 | step 45 | 0.106 | 97.7 | 0.140 |
| `transformer_flatten_2d` | 34,112 | 2.7320 | 1.9946 | step 164 | 0.104 | 35.7 | 0.021 |

⚠ marks a model whose loss never fell by 5% of where it started.

## Best on each question

- **lowest final loss** — `mamba_upstream_3d`, `lstm_3d`, `mamba3_2d`
- **fastest to 90% of its own improvement** — `mamba3_2d`, `transformer_cafa_2d`, `lstm_3d`
- **steadiest at the end** — `transformer_scan_2d`, `transformer_cafa_2d`, `transformer_flatten_2d`
- **most steps per second** — `cnn_2d_sparse`, `tcn_2d_sparse`, `lstm_2d_sparse`
- **most improvement per 1k parameters** — `s4d_portable_2d`, `transformer_cafa_2d`, `s4d_upstream_2d`