2.54 GB

Ctrl+K

1 contributor

Overnight runs of 2026-11-12: the 18M model with gated attention and QK-norm (1.393 against 1.419), with the circuit search (1.431), the 0.1B model at step 3,000, the 8,000-step wide fine-tune, and their result files

ce2a9da verified about 9 hours ago

checkpoints
Overnight runs of 2026-11-12: the 18M model with gated attention and QK-norm (1.393 against 1.419), with the circuit search (1.431), the 0.1B model at step 3,000, the 8,000-step wide fine-tune, and their result files about 9 hours ago
results
Overnight runs of 2026-11-12: the 18M model with gated attention and QK-norm (1.393 against 1.419), with the circuit search (1.431), the 0.1B model at step 3,000, the 8,000-step wide fine-tune, and their result files about 9 hours ago
tokenizer
Model card: 0.1B hybrid design and small-scale results; add the 32k tokenizer 2 days ago
.gitattributes

1.52 kB
initial commit 3 days ago
README.md

8.52 kB
Overnight runs of 2026-11-12: the 18M model with gated attention and QK-norm (1.393 against 1.419), with the circuit search (1.431), the 0.1B model at step 3,000, the 8,000-step wide fine-tune, and their result files about 9 hours ago