MagiSeek-Pro-V1

24B params · Mistral architecture · bf16 merged weights · 131k native context

How he was born

Deep in a rack of three RTX 3090s, WarlordHermes/Magidonia-24B-v4.3-creative-ORPO sat quietly — a solid creative writer, but a little too polite to touch a terminal. Over five curriculum phases of QLoRA (rank 64, rslora, target-all-linear), he was put through school one subject at a time:

• Phase 1–2: foundations, reformatted instruction-following at 2k/4k/8k context, reviewed and re-run in _v2 passes when the mask logic didn't add up.
• Phase 3: a second identity pass, frozen and checked before moving on.
• Phase 4 (agentic): real tool-execution transcripts (hermes-agent-reasoning-traces) plus a small hand-reviewed architecture-QA supplement distilled from Claude Opus 4.6 reasoning traces — this is where he learned to actually call things instead of just describing them.
• Phase 5 (deepseek continuation): a narrow, conservative continuation (979 curated examples, 1 epoch, lower LR, packing off until the mask was hand-verified) chaining off the phase-4 agentic adapter, sharpening DeepSeek-style step-by-step reasoning without overwriting the personality underneath.

Each phase's LoRA adapter was chained onto the previous one — never onto the raw base — so nothing before it got forgotten. The final p5-deepseek adapter was then merged straight into the base weights in full bf16 (no quantization, no rounding shortcuts) to produce this checkpoint: a clean, full-precision snapshot of everything he learned, suitable as the master copy for every downstream quantization (GGUF, GPTQ, AWQ, ...) that follows.

He answers to MagiSeek now — half Magidonia's creative instincts, half a DeepSeek-flavored reasoner who seeks the tool call before the excuse.

Model details

• Architecture: Mistral (MistralForCausalLM), 40 layers, hidden 5120, 32 heads / 8 KV heads (GQA), rope_theta 1e9
• Context: up to 131,072 tokens native
• Precision: bf16 (merged, full precision — no PEFT adapter required at inference time)
• Base model: WarlordHermes/Magidonia-24B-v4.3-creative-ORPO
• Training: 5-phase chained QLoRA curriculum (axolotl), final phase = deepseek-style agentic reasoning continuation
• License: Apache 2.0

Benchmarks (this checkpoint, merged bf16)

Task	Score
MMLU (overall)	0.7433
MMLU-STEM	0.7172
MMLU-Humanities	0.6486
MMLU-Social Sciences	0.8400
MMLU-Other	0.8169
PIQA	acc 0.8022 / acc_norm 0.8413

Intended use

General instruction following, creative writing, and agentic/tool-use tasks requiring long-context reasoning. This is the bf16 merged base — recommended as the source for further quantization (GGUF/GPTQ/AWQ) rather than for direct low-VRAM deployment.

Lineage

WarlordHermes/Magidonia-24B-v4.3-creative-ORPO
  └─ magidonia_curric_p1 / p1_v2
      └─ magidonia_curric_p2 / p2_v2
          └─ magidonia_curric_p3 / p3_v2 (frozen)
              └─ magidonia_curric_p4_agentic_v1 / v2 (frozen)
                  └─ magidonia_curric_p5_deepseek_v1   <-- merged here, bf16