Compactbot
/

ldt-10m

Text Generation

small-language-model

Model card Files Files and versions

42 MB

Ctrl+K

Ctrl+K

1 contributor

History: 21 commits

Compactbot's picture

Fix stale config: v8 final is step 79375 / val_ppl 37.8 (was step 30000 / 3.6892 from an earlier checkpoint)

d7dbce0 verified 3 days ago

.gitattributes

1.52 kB
initial commit 7 days ago
README.md

5.58 kB
Update card to v8 (current weights): 79,375 steps / 2.60B tokens (hits target), val 3.63. Honest eval: 40-sample sweep shows mean loop frac 0.219 (up from v7's 0.167) and 14/40 degenerate — more tokens did not buy coherence. Replaces the stale v7-only card that described the weights as 'well short of 2.6B'. (#15) 3 days ago
config.json

306 Bytes
Fix stale config: v8 final is step 79375 / val_ppl 37.8 (was step 30000 / 3.6892 from an earlier checkpoint) 3 days ago
model.py

5.62 kB
Self-contained LDT model definition (#2) 7 days ago
model.safetensors

41.1 MB
xet

v8: continued training to 2.6B tokens (from 1.51B). Improved coherence on narrative prompts. Same architecture (10,052,864 params, 5L d320 5-head SwiGLU RoPE). (#14) 3 days ago
tokenizer.json

830 kB
Byte-level BPE tokenizer (vocab 12288) (#3) 7 days ago
tokenizer_config.json

250 Bytes
Tokenizer config (#5) 7 days ago