---
license: apache-2.0
tags: [tiny, synthetic, sorted-tokens, runtime-verification]
---

# Consecutive-token tiny llama (14.2M params)

Causal LM trained from scratch on consecutive token-id runs
``[k, k+1, ...]`` mod 50000 (ids 0..49999; 49999 wraps to 0). Greedy
decoding continues the run exactly, which makes it a hand-checkable oracle
for verifying inference runtimes (vLLM, etc.) token-for-token against the
HuggingFace reference.

- Architecture: `llama` (random init, trained 3000 steps,
  lr 0.0005, batch 16, boundary-crossing frac 0.25)
- Data: fully synthetic, generated on the fly (`train.py::consecutive_seq`)
- Eval (greedy exact-match of the hand-computed continuation):
  in-distribution `1.000`,
  longer-than-train `1.000`