wyrmling — a from-scratch intent→DSL model for Magic: The Gathering mechanics
wyrmling is unofficial Fan Content permitted under the Fan Content Policy. Not approved/endorsed by Wizards. Portions of the materials used are property of Wizards of the Coast. © Wizards of the Coast LLC. Card data provided by Scryfall. Not affiliated with either.
wyrmling is a tiny decoder-only language model built entirely from scratch — its own architecture, its own tokenizer, no inherited weights — for exactly one task: translating a structured card intent into the mtg-compiler DSL, a typed s-expression representation of Magic card mechanics. It speaks no English. The full story is in the blog series Teaching a model to write Magic cards by building the compiler first, especially article 4 (shadowing).
Variants
| checkpoint | params | dims | layers | notes |
|---|---|---|---|---|
wyrmling-110M (v8f-p28) |
118M | d768 | 14 | the headline model |
wyrmling-225M (v8f-p28) |
226M | d1024 | 16 | size-ladder rung; same recipe, same data |
Architecture: SwiGLU MLP, RoPE, QK-norm, untied embeddings, logit soft-cap. Optimizer: Muon + AdamW. Tokenizer: custom BPE, ~12k vocabulary, trained only on intent→DSL text so DSL concepts land as single semantically-loaded tokens.
Training
- Pretraining: 18,000 steps (≈0.295B token-positions, ctx 512) on the v8 DSL corpus mixed with 28% grammar-sampled rare-dense data — the optimum from the rare-dense sweep (the anti-shadowing lever; see the blog's article 4).
- SFT: 2 epochs (23,000 steps) on the v8 intent→DSL pairs.
- A full cycle (pretrain + SFT) takes ≈4 h on one AMD R9700.
Evaluation
One held-out v8 test (5,051 cards), free decode, matched generation budgets,
Wald 95% CI (docs/finetune/CROSS_TIER_EVAL.md in the repo):
| model | parse | canonical-exact | dsl_sim | tree_sim |
|---|---|---|---|---|
| wyrmling-110M | 91.6% | 45.6% ±1.4 | 0.865 | 0.777 |
| wyrmling-225M | 93.1% | 45.1% ±1.4 | 0.863 | 0.775 |
Doubling parameters buys nothing here — the model is data-starved, not capacity-starved. That flat line is the point of the release.
Input / output format
Input is a structured brief (not free prose), output is DSL:
Type: Artifact | Colors: W, R | Cost: {1}{W}{R} | Stats: 1/1
Concept: A creature that gains power and toughness when equipped, and
gains deathtouch if it is a Human and equipped. It costs {2} to equip.
(card_abilities
(static (pt-mod (selector :type CREATURE :states (list EQUIPPED)) +1 +1))
(static (gain-ability (it) (DEATHTOUCH))
:condition (is-type :keyword AS :subtype HUMAN :subject-ref EQUIPPED_CREATURE))
(keyword EQUIP :cost ((mana "{2}"))))
Validate/render outputs with the MIT-licensed compiler in the repo.
Repository layout
Each variant directory is self-contained:
wyrmling-110M/ wyrmling-225M/
├── config.json # WyrmlingHFConfig (vLLM plugin, MTG-E83)
├── model.safetensors # verbatim weight dump (embed.weight, blocks.N.attn.qkv.weight, …)
├── tokenizer.json # custom BPE, 12,034 entries incl. added tokens
├── tokenizer_config.json
└── wyrmling-sft-final.pt # original {config, model} training checkpoint
The .pt loads with the compiler repo's TorchGCDGenerator.from_checkpoint()
(grammar-constrained decoding); the safetensors + config load through the
repo's vLLM plugin (src/wyrmling/). Neither is a stock transformers
architecture — you need the compiler repo either way.
Limitations (read before using)
- Shadowing: nodes seen once in training are missed ~100% of the time; the rare tail is the model's known failure mode and the subject of the research.
- Counterfeit-enum era: the v8 training language still contained 117 counterfeit enum values (retired in !462); scores above are measured on that same language. v9-trained checkpoints will supersede these.
- DSL only — no English, no card names, no flavor text, no art.
- Intended for research on specialization/shadowing and for free fan content. The MIT grant covers this project's rights in the weights; it does not license Wizards of the Coast IP, and commercial use of that IP is not permitted by the Fan Content Policy.