# d24 bf16, total batch 2^20: upstream nanochat vs single-file -v3

All runs on the same 8xH100 box unless the label says GH200. `bf16_mfu` is **not** comparable across the two pipelines (different FLOPs/token estimates: upstream 4.775e9, ours 4.860e9) — compare **ms/step** and **tok/sec**, which measure the same 2^20 tokens per step.

| run                                   | parsed     |   ms/step (steps 20-99) | tok/sec   |   bf16_mfu* |
|---------------------------------------|------------|-------------------------|-----------|-------------|
| ours w8 64K micro (full run)          | 5568 steps |                    1212 | 864,856   |       53.13 |
| ours w8 32K micro +FP8                | 200 steps  |                    1040 | 1,008,030 |       61.92 |
| ours w8 32K micro                     | 100 steps  |                    1249 | 839,838   |       51.59 |
| upstream w8 bf16                      | 1017 steps |                    1291 | 812,435   |       49.03 |
| upstream w8 +fp8                      | 200 steps  |                    1081 | 970,455   |        4.86 |
| fork varlen branch w8 bf16            | 200 steps  |                    1278 | 820,783   |       49.54 |
| fork varlen+fp8 w8 (2026-03-22, full) | 5568 steps |                    1066 | 983,648   |       59.37 |
| ours w1 (1xH100, partial)             | 23 steps   |                    9620 | 109,002   |       53.57 |
| ours w1 (1x GH200)                    | 100 steps  |                    9635 | 108,834   |       53.48 |

| derived   | value   |
|-----------|---------|

## Correctness: does world=8 reproduce the world=1 curve?

The loader assigns every rank's documents from one sequential stream and total batch is fixed, so the global batch per optimizer step is identical at every world size — these are directly comparable.

| run                            |   step 0 |   step 25 |   step 50 |   step 75 |   step 100 |
|--------------------------------|----------|-----------|-----------|-----------|------------|
| -v3 world=8 (8xH100)           |  3.17614 |   1.75957 |   1.57767 |   1.45099 |    1.37943 |
| -v3 world=1 (GH200, committed) |  3.17614 |   1.75956 |   1.57762 |   1.4503  |    1.3794  |

| run                                 | min val bpb         | peak alloc MiB   | peak reserved MiB   | train time   |
|-------------------------------------|---------------------|------------------|---------------------|--------------|
| -v3 world=8                         | 1.379430            | 40049            | 65750               | 1.85m        |
| -v3 world=1 GH200                   | 1.379400            | 49283            | 71046               | 14.29m       |
| world=1 reference band (eval100a-e) | 1.379318 - 1.379563 | 49,088 - 49,334  | --                  | --           |


## Head-to-head val bpb (both on shard_06542)

|   step |   -v3 (full run) |   upstream baseline | delta   |
|--------|------------------|---------------------|---------|
|      0 |         3.17584  |            3.16882  | +0.22%  |
|    250 |         0.997472 |            0.984017 | +1.37%  |
|    500 |         0.904081 |            0.898168 | +0.66%  |
|    750 |         0.866224 |            0.861772 | +0.52%  |
|   1000 |         0.846058 |            0.841709 | +0.52%  |

The upstream baseline was stopped at step 1000, so it bounds the comparison; beyond that the -v3 curve stands alone.

Full-run val bpb curve: 0:3.175835 / 250:0.997472 / 500:0.904081 / 750:0.866224 / 1000:0.846058 / 1250:0.832731 / 1500:0.822766 / 1750:0.815902 / 2000:0.809199 / 2250:0.798533 / 2500:0.789216 / 2750:0.781275 / 3000:0.773139 / 3250:0.766625 / 3500:0.759539 / 3750:0.753329 / 4000:0.747155 / 4250:0.741616 / 4500:0.736278 / 4750:0.731419 / 5000:0.726908 / 5250:0.722909 / 5500:0.719696 / 5568:0.719042

_Produced by `make_baseline_report.py`; sources listed in its docstring._