Document SM87 and SM110a aarch64 fatbin
Browse files- benchmarks/RESULTS.md +61 -0
benchmarks/RESULTS.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# adaptive-layernorm-producers Benchmark Results
|
| 2 |
+
|
| 3 |
+
Local source-build benchmark:
|
| 4 |
+
|
| 5 |
+
- GPU: NVIDIA GeForce RTX 5090
|
| 6 |
+
- Driver: 580.82.07
|
| 7 |
+
- Runtime: local Torch 2.11 / CUDA 12.8 environment
|
| 8 |
+
- Command: `python adaptive-layernorm-producers/benchmarks/benchmark.py --backend source --iters 100`
|
| 9 |
+
- Baseline: PyTorch eager producer chain with equivalent operations.
|
| 10 |
+
- Status: source correctness passed before benchmark. Refresh this table after
|
| 11 |
+
installed-artifact validation on each target hardware.
|
| 12 |
+
|
| 13 |
+
| Shape | Rows | Dim | AdaLN->FP8 us | Eager chain us | Speedup | LN->FP8 us | Eager LN chain us | Speedup |
|
| 14 |
+
|---|---:|---:|---:|---:|---:|---:|---:|---:|
|
| 15 |
+
| decode_action | 16 | 2048 | 4.117 | 65.010 | 15.79x | 3.962 | 48.006 | 12.12x |
|
| 16 |
+
| wan_video_short | 64 | 3072 | 4.133 | 63.425 | 15.35x | 4.117 | 47.826 | 11.62x |
|
| 17 |
+
| wan_video_ctx | 256 | 3072 | 4.140 | 69.821 | 16.86x | 4.108 | 55.611 | 13.54x |
|
| 18 |
+
| wan_video_2k | 2520 | 3072 | 12.330 | 263.276 | 21.35x | 10.267 | 218.926 | 21.32x |
|
| 19 |
+
| wan_video_4k | 4096 | 3072 | 18.465 | 463.733 | 25.11x | 16.412 | 394.908 | 24.06x |
|
| 20 |
+
|
| 21 |
+
## Six-way modulation producer
|
| 22 |
+
|
| 23 |
+
The new `adaln_modulation6_bf16` path was measured against its raw CUDA
|
| 24 |
+
launcher, the Tensor wrapper, eager, and `torch.compile`. Graph figures capture
|
| 25 |
+
32 identical launches per graph and divide replay time by 32, avoiding a graph
|
| 26 |
+
launch-floor artifact for the smallest row.
|
| 27 |
+
|
| 28 |
+
| Shape | Native us | Wrapper us | Graph native us | Graph wrapper us | Eager us | Compile us | Wrapper/native | Graph wrapper/native |
|
| 29 |
+
|---|---:|---:|---:|---:|---:|---:|---:|---:|
|
| 30 |
+
| GROOT DiT B1 S51 D1536 | 2.221 | 3.052 | 1.536 | 1.663 | 48.804 | 43.536 | 1.374 | 1.083 |
|
| 31 |
+
| Motus B1 S2520 D3072 | 175.952 | 176.183 | 165.263 | 165.513 | 209.040 | 176.665 | 1.001 | 1.002 |
|
| 32 |
+
| video long B1 S5070 D3072 | 347.251 | 354.773 | 344.858 | 344.592 | 582.678 | 354.059 | 1.022 | 0.999 |
|
| 33 |
+
|
| 34 |
+
The direct S51 row exposes about `0.83 us` of dispatcher overhead. The
|
| 35 |
+
multi-launch CUDA Graph path reduces the absolute gap to `0.127 us`. Both
|
| 36 |
+
figures are retained; the direct row is not described as native-equivalent.
|
| 37 |
+
|
| 38 |
+
## NVIDIA Thor installed artifact
|
| 39 |
+
|
| 40 |
+
- Device: NVIDIA Thor, SM110, aarch64
|
| 41 |
+
- Runtime: PyTorch 2.11.0 + CUDA 13.0
|
| 42 |
+
- Variant: `torch211-cxx11-cu130-aarch64-linux`
|
| 43 |
+
- Source commit: `9b532b42ad71d0ef6bc79d1372e96910087faad4`
|
| 44 |
+
- Artifact code objects: SM87 and SM110a; measurements below are SM110a
|
| 45 |
+
- Kernel Hub `v1` artifact SHA256: `2b5542ae448332ebd4d1f2ca2f0030f88200061f8d2e64eb8581c314e7d7322c`
|
| 46 |
+
|
| 47 |
+
The full installed-artifact correctness matrix passed. The two per-token
|
| 48 |
+
producer entries were also measured against their raw registered ops. Outputs
|
| 49 |
+
were bit-exact in every row.
|
| 50 |
+
|
| 51 |
+
| Op | Shape | Raw us | Wrapper us | Wrapper/raw | Graph wrapper/raw us |
|
| 52 |
+
|---|---|---:|---:|---:|---:|
|
| 53 |
+
| per-token | GROOT DiT M51 D1536 | 6.154 | 6.148 | 0.999 | 6.211 / 6.224 |
|
| 54 |
+
| table per-token | GROOT DiT M51 D1536 | 6.166 | 6.150 | 0.997 | 7.127 / 6.607 |
|
| 55 |
+
| per-token | vision M105 D1152 | 8.204 | 8.207 | 1.000 | 8.203 / 8.377 |
|
| 56 |
+
| table per-token | vision M105 D1152 | 8.210 | 8.210 | 1.000 | 8.204 / 8.703 |
|
| 57 |
+
| per-token | video M2520 D3072 | 229.922 | 229.936 | 1.000 | 229.984 / 230.797 |
|
| 58 |
+
| table per-token | video M2520 D3072 | 230.174 | 229.929 | 0.999 | 229.719 / 230.431 |
|
| 59 |
+
|
| 60 |
+
The table reports the uploaded SM87+SM110a fat binary on Thor. SM87 is present
|
| 61 |
+
as native cubin code but is not included in these runtime measurements.
|