| # What a tablet can actually do (measured) |
|
|
| Host: 8-core ARMv9 (4x Cortex-A520 + 4x Cortex-A520 @1.95 GHz), 8 GB RAM, ~13 GB |
| free disk. Torch on CPU. No GPU. |
|
|
| Forward throughput (fp32, seq 256, batch 16) measured on the device: |
|
|
| | size | params | fwd~ steps/s | |
| |------|--------|--------------| |
| | tiny7 | 7.8M | 1540 | |
| | hybrid18 | 16.8M | 741 | |
| | tiny20 | 21.6M | 713 | |
| | hybrid25 | 25.4M | 496 | |
| | tiny28 | 28.9M | 552 | |
| Small models are near-linear in cost; 25M is ~3 sectors slower than 16M for ~1.6x params. |
|
|
| Real training (bf16, autocast): hybrid18M prefer ~670-780 tok/s; at 16M one epoch |
| of 30M tokens ~11 h. |
|
|
| RAM math (AdamW): fp32 master + momentum needs ~3x fp32 bytes per param. 28.9M ~ |
| 340 MB + activations fits in the tablet's free ~3 GB after a job exits. |
|
|
| ### Rules that keep a tablet alive |
| - One heavy torch job at a time. |
| - Checkpoint + `--resume` every few hundred steps; OOM never loses > one window. |
| - Watch the balanced not skewed (a corpus that cycles blocks in order breaks; see |
| `corpus.md`). |
| - Don't promise "coherent chat" below ~28M (see `lessons_from.md`). |
|
|
