| # Device comparison |
|
|
| - **MPS bench** — Apple arm64 (MPS) · torch 2.13.0 · torch-dimensions 0.3.1 |
| - **CUDA bench** — NVIDIA GeForce RTX 5090 · torch 2.12.1+cu130 · torch-dimensions 0.3.1 |
|
|
| Both runs: seed 20260803, 300 steps, batch 4. |
| Data is drawn on CPU from a seeded generator, so both machines see the |
| same batches in the same order. |
|
|
| Both runs load one shared set of starting weights, so the initial |
| conditions are bit-identical and every difference below is arithmetic. |
|
|
| | model | loss (left) | loss (right) | Δ loss | first divergence | rel Δw | max Δw | speed | |
| |---|---|---|---|---|---|---|---| |
| | `lstm_2d_sparse` | 0.72047 | 0.72259 | 2.13e-03 | step 1 | 3.43e-02 | 4.05e-02 | 0.76× | |
| | `gru_2d_sparse` | 0.72084 | 0.71882 | 2.03e-03 | step 0 | 2.71e-02 | 2.75e-02 | 1.92× | |
| | `s4d_portable_2d` | 0.73560 | 0.73450 | 1.09e-03 | step 84 | 2.57e-04 | 5.21e-03 | 0.69× | |
| | `s4d_upstream_2d` | 0.73867 | 0.73867 | 8.94e-07 | step 86 | 2.13e-06 | 1.50e-03 | 0.70× | |
| | `s4_upstream_2d` | 0.73359 | 0.73277 | 8.28e-04 | step 89 | 3.58e-05 | 1.73e-03 | 0.72× | |
| | `mamba_portable_2d` | 0.73149 | 0.73126 | 2.26e-04 | step 93 | 1.29e-03 | 5.29e-03 | 0.55× | |
| | `mamba_upstream_2d` | 0.73244 | 0.73441 | 1.97e-03 | step 74 | 1.74e-03 | 7.64e-03 | 0.59× | |
| | `mamba2_2d` | 0.74520 | 0.74476 | 4.44e-04 | step 21 | 4.34e-02 | 5.67e-02 | 4.35× | |
| | `mamba3_2d` | 0.72130 | 0.72066 | 6.39e-04 | step 26 | 6.35e-03 | 1.32e-02 | 1.44× | |
| | `transformer_scan_2d` | 1.63615 | 1.63248 | 3.67e-03 | step 2 | 5.53e-02 | 8.12e-02 | 1.23× | |
| | `transformer_cafa_2d` | 1.45857 | 1.45871 | 1.44e-04 | step 42 | 4.64e-02 | 5.41e-02 | 2.21× | |
| | `transformer_flatten_2d` | 2.31866 | 2.31973 | 1.07e-03 | step 24 | 2.81e-02 | 5.61e-02 | 1.01× | |
| | `cnn_2d_sparse` | 1.86875 | 1.87058 | 1.83e-03 | step 0 | 2.75e-02 | 3.09e-02 | 0.98× | |
| | `tcn_2d_sparse` | 1.05559 | 1.05888 | 3.28e-03 | step 0 | 1.04e-01 | 1.35e-01 | 0.96× | |
| | `mamba_upstream_3d` | 0.00608 | 0.00599 | 9.04e-05 | step 55 | 3.05e-04 | 3.72e-03 | 0.88× | |
| | `lstm_3d` | 0.00822 | 0.00598 | 2.24e-03 | step 0 | 5.10e-02 | 7.38e-02 | 0.79× | |
|
|
