op-meco-scratch-c0 โ from-scratch metadata-conditioned pretrain (scratch-c0)
OFFICIAL-PRETRAIN from-scratch testbed (user ask 2026-08-16): conditioning
from TOKEN ZERO, the intervention point the literature says matters. OLMo-2
architecture d512/8L/8H, tied embeddings, stock OLMo tokenizer (~77M params,
~25M non-embedding); 90,478 labeled dolma docs @ dataset revision 48f4f8aa,
~4 epochs, lr 4e-4 cosine, wd 0.1, block 1024, bs 64, seed 17.
Arm: bare documents throughout (control)
Siblings: op-meco-scratch-c0 / -plain / -tagged. pretrain_stats.json has
the full accounting. Part of the meco/ route (branch meco).
- Downloads last month
- 68
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support