fsi-anomaly / cookbook /device.md
FerrellSyntheticIntelligence's picture
backup all: 100 files (batch)
1c0d385 verified
|
Raw
History Blame Contribute Delete
1.09 kB

What a tablet can actually do (measured)

Host: 8-core ARMv9 (4x Cortex-A520 + 4x Cortex-A520 @1.95 GHz), 8 GB RAM, ~13 GB free disk. Torch on CPU. No GPU.

Forward throughput (fp32, seq 256, batch 16) measured on the device:

size params fwd~ steps/s
tiny7 7.8M 1540
hybrid18 16.8M 741
tiny20 21.6M 713
hybrid25 25.4M 496
tiny28 28.9M 552
Small models are near-linear in cost; 25M is ~3 sectors slower than 16M for ~1.6x params.

Real training (bf16, autocast): hybrid18M prefer ~670-780 tok/s; at 16M one epoch of 30M tokens ~11 h.

RAM math (AdamW): fp32 master + momentum needs ~3x fp32 bytes per param. 28.9M ~ 340 MB + activations fits in the tablet's free ~3 GB after a job exits.

Rules that keep a tablet alive

  • One heavy torch job at a time.
  • Checkpoint + --resume every few hundred steps; OOM never loses > one window.
  • Watch the balanced not skewed (a corpus that cycles blocks in order breaks; see corpus.md).
  • Don't promise "coherent chat" below ~28M (see lessons_from.md).