Upload training/DATASETS.md with huggingface_hub
Browse files- training/DATASETS.md +27 -0
training/DATASETS.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Cortex A 0.5 — SFT dataset mix
|
| 2 |
+
|
| 3 |
+
Quality-filtered frontier distillations covering math, science, coding, agentic
|
| 4 |
+
tool-use, instruction following, long context, and roleplay. The r0b0tlab corpus
|
| 5 |
+
is the spine (native train/val/test). Complements fill Claude Opus 4.6–4.8
|
| 6 |
+
reasoning, DeepSeek V4 coding, and character roleplay (core mix is only ~1.1% dialogue).
|
| 7 |
+
|
| 8 |
+
| Dataset | Domain | Teachers | Rows (approx) | Role |
|
| 9 |
+
|---|---|---|---|---|
|
| 10 |
+
| [r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation](https://huggingface.co/datasets/r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation) `sft_balanced` | mixed SOTA | Qwen3.8-Max, GLM-5.2, Kimi K3 | 52,205 train / 2,872 val / 2,860 test | **Required core.** Math 26%, code 25%, reasoning 18%, instruction 15%, agent/tools 10%, science, long-context, multilingual, dialogue |
|
| 11 |
+
| [Jackrong/DeepSeek-V4-Distill-8000x](https://huggingface.co/datasets/Jackrong/DeepSeek-V4-Distill-8000x) | coding / reasoning | DeepSeek V4 | ~7.7k | Coding + STEM traces |
|
| 12 |
+
| [Jackrong/Claude-opus-4.7-TraceInversion-5000x](https://huggingface.co/datasets/Jackrong/Claude-opus-4.7-TraceInversion-5000x) | reasoning | Claude Opus 4.7 | ~5k | High-quality inverted traces |
|
| 13 |
+
| [Jackrong/Claude-opus-4.6-TraceInversion-9000x](https://huggingface.co/datasets/Jackrong/Claude-opus-4.6-TraceInversion-9000x) | reasoning | Claude Opus 4.6 | ~9k | Broader Opus 4.6 coverage |
|
| 14 |
+
| [Roman1111111/claude-opus-4.6-10000x](https://huggingface.co/datasets/Roman1111111/claude-opus-4.6-10000x) | reasoning | Claude Opus 4.6 | ~9.6k | General reasoning transfer |
|
| 15 |
+
| [nohurry/Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) | reasoning | Claude Opus 4.6 | ~2.3k | Filtered long reasoning |
|
| 16 |
+
| [ansulev/claude-opus-4.8-distill-5k](https://huggingface.co/datasets/ansulev/claude-opus-4.8-distill-5k) | reasoning | Claude Opus 4.8 | ~5k | Newest Opus line |
|
| 17 |
+
| [TeichAI/Claude-Opus-4.6-Reasoning-887x](https://huggingface.co/datasets/TeichAI/Claude-Opus-4.6-Reasoning-887x) | long reasoning | Claude Opus 4.6 | 887 | Dense high-effort traces (upweighted) |
|
| 18 |
+
| [lordx64/reasoning-distill-claude-opus-4-7-max](https://huggingface.co/datasets/lordx64/reasoning-distill-claude-opus-4-7-max) | reasoning | Claude Opus 4.7 Max | varies | Extra 4.7-max distill |
|
| 19 |
+
| [Jackrong/Qwen3.5-reasoning-700x](https://huggingface.co/datasets/Jackrong/Qwen3.5-reasoning-700x) | reasoning | Qwen3.5 | ~700 | In-family reasoning style |
|
| 20 |
+
| [beyoru/Aesir-Character-CoT-roleplay](https://huggingface.co/datasets/beyoru/Aesir-Character-CoT-roleplay) | roleplay | character CoT | ~2.0k | Persona + chain-of-thought RP (upweighted) |
|
| 21 |
+
| [ChengyuDu0123/HER-Dataset](https://huggingface.co/datasets/ChengyuDu0123/HER-Dataset) `sft_multi_turn` | roleplay | literary RP | 8k cap | Multi-turn character dialogue |
|
| 22 |
+
|
| 23 |
+
**Validation.** Held-out `validation` split from r0b0tlab only (2,872 traces, subsampled to 256 during the run for tok/s). Complements are train-only so val stays clean.
|
| 24 |
+
|
| 25 |
+
**Not mixed this session (too large / redundant):** OpenR1-Math-220k, Ultrachat, generic instruction dumps. r0b0tlab already concentrates verified math/code/agent traces.
|
| 26 |
+
|
| 27 |
+
**Vision.** Vision encoder is **frozen** this SFT phase so text tok/s stay high on T4s. Weights remain in the checkpoint (needed for the ~450MB 4-bit + mmproj target). A short vision QAT pass is the next session.
|