license: apache-2.0
base_model: mistralai/Mistral-Small-3.1-24B-Base-2503
library_name: transformers
pipeline_tag: text-generation
tags:
- roleplay
- creative-writing
- chatml
language:
- en
SpoomplesMaxx-Thrasher-24B
"Thrash Metal"
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ░░░░░ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░██ ▒▓██▓▒ ░ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░ ▓░▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░▓▓▒█▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ░░░░░█▓░▓█ ░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░▒███▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒░░▒░░░░▓ ▓ ▓█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░▓░ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░▒░░ █▓ █▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ░░░░░░░▒░░▒▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░▒░░░░ █ ▓▓▓ ▓▓█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ ░░░░░ ▒░▓▓░ █▓▓▒▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ ▓▓ ▓ ▓ ▓▓▓ ▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ █▓▓ █▓▓▓ ▓▓█▓█ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒ █ ▓ ░▓▓▓▓ ▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒ ▓ ▓▒▓▓ ▒▓▓▓▓▓▓▓█ ▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓░ ▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░ ▒ ▒ ▓ ▓▓ ▓▓ ▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ░ ░ ░░▓▓▓▓ ▓▓░▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ░ ░░░▓▒█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓░░░░▓▓ ▓▓▓▓ ░▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░▓ ▓▓▓░ ▓▓▓▓ ▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓█▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▒▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓ ░▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▒▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓░ ░ ▒▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓ ░ ▓ ▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓ ▒▓ ▒ ▓▓▓▓▓▓▓▓▓▓▓ ░▓▓▓▓▓▓▓ ▓ ▒▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓ ▓ ░▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓ ▒ ▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓ ▓ ░▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓ ░▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓░ ▓▓ ▓▓ ▓ ░░▓▓▓▓▓▓▓▓▓▓▓ ▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓ ▓ ░ ▓▓▓▓▓▓▓▓▓ ▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓ ░░▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓ ▒ ░ ▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓ ░ ░▓ ▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓ ░▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓░ ▓ ▒▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
"Thousand-Songed" — Toxostoma rufum, the brown thrasher: holder of the largest documented song repertoire of any North American bird, over a thousand song types. The mockingbird repeats a phrase three times; the thrasher sings each one twice and moves on. Second of the mimids, the family that follows the corvids.
A model that is 100% about roleplay, trained mostly on things that are not roleplay. Measured across the strongest open RP lineage I know (Dans-PersonalityEngine), roughly 590K of its rows are task/reasoning/assistant/world-knowledge data against ~150K of actual roleplay. RP is the product; RP is not the corpus. The RP data teaches the register. Everything else teaches the mind behind it.
Built on Mistral-Small-3.1-24B-Base with the vision tower removed and the chat interface rebuilt from scratch — see "The token surgery" below, because if you have ever bounced off a Mistral model's template, that section is for you.
Who this is for
The accessible mimid. mockingbird at 36B asks a lot of your VRAM; thrasher at 24B (23.6B after the vision strip) is the same recipe in a size that quantizes onto a single 24GB card. If you can run any of the popular 22–24B RP models, you can run this one.
Prompt format
ChatML. On a Mistral base. Yes, really — and not by resizing anything:
FORM <|im_start|>system\n{card}<|im_end|>\n<|im_start|>user\n{text}<|im_end|>\n<|im_start|>assistant\n{reply}<|im_end|>\n
STOPS <|im_end|> (id 21; eos)
BOS <s> (id 1) — added automatically by the tokenizer, not the template
ROLES system / user / assistant / tool
EXAMPLE
<|im_start|>system
You are Bram Hollis, keeper of the Wayward Lantern...<|im_end|>
<|im_start|>user
*I push the door open, dripping wet* Got room for one more?<|im_end|>
<|im_start|>assistant
The template ships embedded (chat_template.jinja + tokenizer_config.json), so vLLM, llama.cpp, MLX, and every frontend that speaks ChatML — which is to say, every RP frontend — picks it up without ceremony.
No thinking. Ever.
thrasher never emits reasoning traces and was never trained on them.
The token surgery
Moving a Mistral base to ChatML was my first barrier the first time I ever tried to use a Mistral model for fine-tuning, so here is the whole recipe, with receipts.
1. There is room in the vocabulary. Mistral's Tekken tokenizer reserves ids 0–999 as a control block; only 0–19 are named. Ids 20–999 are unused <SPECIAL_n> placeholders. So ChatML needs no vocab resize and no embedding-matrix growth: rename <SPECIAL_20> → <|im_start|> and <SPECIAL_21> → <|im_end|> in tokenizer.json (both added_tokens and the vocab), tokenizer_config.json, and special_tokens_map.json. Set eos to <|im_end|>. Done — single-token ids 20 and 21.
2. The claimed rows are dead, and dead eos rows are fatal. The base was pretrained with those placeholders never appearing in data: their embedding rows are exactly 0.0 (measured), their lm_head rows random-scale noise. A model whose eos row is dead never learns to stop — unless you train very long (Hermes cold-claimed on 60B tokens; this SFT is ~1B) or you initialize sensibly. thrasher grafts: <|im_start|> rows ← <s>, <|im_end|> rows ← </s>, embedding and lm_head. By step 100 of SFT, stop-rate at temperature 0.7 was already 24/24; it stayed 100% at every checkpoint probed.
3. The published tokenizer has a broken pre-tokenizer regex. The known Mistral conversion bug (transformers warns and offers fix_mistral_regex=True — but that fix is in-memory only, and axolotl, llama.cpp, and MLX read tokenizer.json directly). The true Tekken pattern — case-aware word splitting, single-digit number splits — is baked into the shipped file.
4. The vision tower is gone. 222 tensors of Pixtral removed, the language_model. prefix stripped, published as a plain untied MistralForCausalLM. Nothing multimodal remains.
The full prep script (prep_base.py) is in this repo, and the prepped base is its own artifact if you want to start from it: Mistral-Small-3.1-24B-Base-thrasher.
Tool calling
The corpus includes the full Toolmaxx family (58,095 conversations), rendered with tool responses as a plain tool role turn:
<|im_start|>tool\n{tool output}<|im_end|>
Corpus-taught, conversational tool competence — not a structured calling API.
If you need strict function calling, put a schema in the card and validate what comes back.
Key details
BASE mistralai/Mistral-Small-3.1-24B-Base-2503 (Apache 2.0, vision stripped) PARAMS 23.6B dense · 40 layers · GQA 8 KV heads · head_dim 128 · hidden 5120 VOCAB 131,072 · ChatML on claimed Tekken slots 20/21 · zero added tokens CTX trained at 24,576 packed · base RoPE (theta 1e9) to 131K CORPUS 667,332 conversations · ~1.3B supervised chars · 43% RP share LANGUAGE English (non-English filtered at ingest; base priors remain)
Training
Full-parameter SFT, Axolotl, 8×H200. One stage. The exact config generator ships in this repo (thrasher_sft_cfg.py):
STEPS 902 (2 epochs) · this release = step 600 SEQ 24,576 · sample packing (block-masked; packing does not shrink context) BATCH 64 global (micro 1 × accum 8 × 8 GPUs) OPT AdamW · lr 8e-6 cosine · 3% warmup · wd 0.01 · bf16 STACK FSDP2 full-shard · activation checkpointing · Cut Cross Entropy HEALTH grad_norm 1.2–1.8 the whole run · zero spikes · memory flat
On context: 24,576 is the longest single training conversation (longer ones were split at turn boundaries with the card re-carried). The base's 131K RoPE survives SFT untouched; the best-trained RP region is the first ~24K, degrading gracefully beyond.
The corpus is mockingbird's, verbatim — the PersonalityEngine V1.3.0 public list plus my own carded-RP, think-stripped-RP, and anti-repetition lanes, same cleaning receipts. Only the template changed. See the mockingbird card for the full corpus story.
How the checkpoint was chosen — and how you can check
Loss did not pick this model. Checkpoints went through two instruments, both published in this repo's eval/: a seeded multi-turn loop/stall battery (six gates, six repeats per episode — single-run numbers on it are noise, and the trend proves it), and blind-judged episodes on four real character cards.
step 100 200 300 450 600 750 900 battery 17 8 13 13 18 10 16 (of 24) stop-rate 100% 100% 100% 100% 100% 100% 100%
The shape is the story: a mid-run dip while style reorganizes under high LR, a peak mid-way through epoch 2, then regression in the deep anneal. Behavioral peak ≠ end of training. Step 600 shipped — highest battery pass rate and the most disciplined judged transcripts (compressed, declarative, no user-impersonation).
The anti-repetition anneal experiments — published, not spun. After training, we annealed checkpoints 600 and 900 on a 7,281-conversation anti-repetition dataset (recipe in that repo), across four blend/base variants plus two task-arithmetic merges. In our measurements, one variant reached zero repeat failures on the battery while regressing on format gates; the merges regressed stop-token reliability; none beat the un-annealed step 600 overall, so step 600 shipped as-is. The raw battery JSONs for every variant are in eval/ — read them and draw your own conclusions rather than taking ours.
Sampling
The shipped generation_config.json is the swept optimum (7 arms × 5 seeded battery runs each):
temperature 1.0 · min_p 0.05 · top_p off
thrasher is the anti-mockingbird in its sampler behavior, which is why we sweep per model instead of inheriting: min_p won here (mockingbird's sweep found min_p looped MORE); and a mild repetition_penalty 1.05 — catastrophic on mockingbird — is a legitimate opt-in on thrasher: in our sweep it eliminated the verbatim-loop tail entirely (max cross-turn Jaccard 0.50 across 20 episodes) at the cost of a rare unfinished turn. If loops bother you more than an occasional run-on, add it. Plain top_p 0.9 at temp 1.0 was the worst repetition arm on this model — don't ship RP muscle memory, sweep.
Known limitation: the loop tail.
Under sustained low-information multi-turn pressure the model can fall into near-verbatim self-repetition — roughly 1–2 episodes in 24 on our battery at shipped settings. The corpus's anti-repetition lane suppresses it; it is not eliminated. The repetition_penalty 1.05 opt-in above removed it entirely in our measurements.
Give it a proper card and it will give you a proper character: the model was fed real character cards (median ~3K chars, p90 ~8.5K) as system messages.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "aimeri/spoomplesmaxx-thrasher-24B"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
messages = [
{"role": "system", "content": "You are Bram Hollis, keeper of the Wayward Lantern... Third person, *asterisk action beats*."},
{"role": "user", "content": "*I push the door open, dripping wet* Got room for one more tonight?"},
]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=400, do_sample=True) # sampler ships in generation_config
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
Quants: GGUF static (Q3/Q4/Q5_K_M) · GGUF imatrix (IQ3_XXS–Q4_K_M, own-corpus calibration, imatrix.dat included) · MLX 4-bit / 6-bit. Prefer the imatrix quants at 3–4 bit.
The thrasher knows a thousand songs. It only needs the one you hand it.
thrasher is a roleplay and creative-writing model for adults. It stays in character by design — its corpus was scrubbed of mid-scene refusals — so bring your own moderation where your deployment needs it. Not an assistant, not an oracle, not for anything safety-critical.
mimids 02 · trained 2026-08 · checkpoints at thrasher-v1-ckpts · eval instruments, prep scripts, and training config in this repo · Apache 2.0