Cortex A 0.5 — SFT dataset mix
Quality-filtered frontier distillations covering math, science, coding, agentic tool-use, instruction following, long context, and roleplay. The r0b0tlab corpus is the spine (native train/val/test). Complements fill Claude Opus 4.6–4.8 reasoning, DeepSeek V4 coding, and character roleplay (core mix is only ~1.1% dialogue).
Loader note. r0b0tlab/... README configs omit data_files, so
load_dataset(id, "sft_balanced") globs every parquet (including glm47_native)
and crashes with a schema CastError. We load data_dir="data/sft_balanced" only.
| Dataset | Domain | Teachers | Rows (approx) | Role |
|---|---|---|---|---|
r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation sft_balanced |
mixed SOTA | Qwen3.8-Max, GLM-5.2, Kimi K3 | 52,205 train / 2,872 val / 2,860 test | Required core. Math 26%, code 25%, reasoning 18%, instruction 15%, agent/tools 10%, science, long-context, multilingual, dialogue |
| Jackrong/DeepSeek-V4-Distill-8000x | coding / reasoning | DeepSeek V4 | ~7.7k | Coding + STEM traces |
| Jackrong/Claude-opus-4.7-TraceInversion-5000x | reasoning | Claude Opus 4.7 | ~5k | High-quality inverted traces |
| Jackrong/Claude-opus-4.6-TraceInversion-9000x | reasoning | Claude Opus 4.6 | ~9k | Broader Opus 4.6 coverage |
| Roman1111111/claude-opus-4.6-10000x | reasoning | Claude Opus 4.6 | ~9.6k | General reasoning transfer |
| nohurry/Opus-4.6-Reasoning-3000x-filtered | reasoning | Claude Opus 4.6 | ~2.3k | Filtered long reasoning |
| ansulev/claude-opus-4.8-distill-5k | reasoning | Claude Opus 4.8 | ~5k | Newest Opus line |
| TeichAI/Claude-Opus-4.6-Reasoning-887x | long reasoning | Claude Opus 4.6 | 887 | Dense high-effort traces (upweighted) |
| lordx64/reasoning-distill-claude-opus-4-7-max | reasoning | Claude Opus 4.7 Max | varies | Extra 4.7-max distill |
| Jackrong/Qwen3.5-reasoning-700x | reasoning | Qwen3.5 | ~700 | In-family reasoning style |
| beyoru/Aesir-Character-CoT-roleplay | roleplay | character CoT | ~2.0k | Persona + chain-of-thought RP (upweighted) |
ChengyuDu0123/HER-Dataset sft_multi_turn |
roleplay | literary RP | 8k cap | Multi-turn character dialogue |
Validation. Held-out validation split from r0b0tlab only (2,872 traces, subsampled to 256 during the run for tok/s). Complements are train-only so val stays clean.
QAT. Unsloth qat_scheme="int8-int4": 4-bit weights + 8-bit dynamic activation quantization.
Vision. Vision encoder is frozen this SFT phase so text tok/s stay high on T4s. Weights remain in the checkpoint (needed for the ~450MB 4-bit + mmproj target).