| # Best Datasets for SFT Fine-Tuning — Verified Guide |
|
|
| ## Dataset Rankings (Quality → Model Performance) |
|
|
| ### #1: allenai/tulu-3-sft-mixture — THE BEST |
| - **Size**: 939K examples from 19 curated sources |
| - **Format**: messages column (role/content) - ZERO PREPROCESSING |
| - **Sources**: FLAN v2, Persona MATH, Evol CodeAlpaca, WildChat, Aya, NuminaMath, WildGuard, WildJailbreak, no_robots, OASST1, SciRIFF, etc. |
| - **Proven Results on Llama-3.1-8B**: MMLU 53.5, GSM8K 79.9, IFEval 63.6, HumanEval 76.8 |
| - **Training Recipe**: LR=5e-6, batch=128, epochs=2, max_seq=4096, linear schedule |
| - **Status**: VALIDATED - column format confirmed via hf_inspect_dataset |
|
|
| ### #2: open-thoughts/OpenThoughts-114k — REASONING CoT |
| - **Size**: 114K examples with DeepSeek-R1 reasoning traces |
| - **Format**: conversations column (from/value ShareGPT) - NEEDS CONVERSION |
| - **Best For**: Math, code, science with chain-of-thought |
| - **Conversion**: See train_openthoughts.py |
| - **Training Recipe**: LR=2e-4, batch=16, epochs=2, cosine schedule |
| - **Status**: VALIDATED - format confirmed, converter tested |
| |
| ### #3: HuggingFaceH4/ultrachat_200k — GENERAL CHAT |
| - **Size**: 208K multi-turn conversations |
| - **Format**: messages column - ZERO PREPROCESSING (use train_sft split) |
| - **Best For**: General conversational ability |
| - **Training Recipe**: LR=2e-4, batch=16, epochs=1 |
| |
| ### #4: mlabonne/FineTome-100k — CURATED COMPACT |
| - **Size**: 100K quality-scored examples |
| - **Format**: conversations (ShareGPT) - NEEDS CONVERSION |
| - **Best For**: Quick fine-tune with curated quality |
| |
| ### #5: HuggingFaceH4/no_robots — HUMAN-WRITTEN |
| - **Size**: 9.5K examples (all human-written) |
| - **Format**: messages column - ZERO PREPROCESSING |
| - **Best For**: High-quality instruction following |
|
|
| ## How to Train |
|
|
| ### Full Training (Tulu 3 - 940K) — A100 80GB, ~6h |
| ``` |
| python ai-ml/hf-finetuning/train_tulu3.py |
| ``` |
|
|
| ### Reasoning Training (OpenThoughts - 114K) — A100 80GB, ~2h |
| ``` |
| python ai-ml/hf-finetuning/train_openthoughts.py |
| ``` |
|
|
| ### Quick Test (100 steps) — Any GPU |
| ``` |
| python ai-ml/hf-finetuning/train_tulu3.py --max_steps 100 --no_push |
| ``` |
|
|
| ## LoRA Config (LoRA Without Regret - Schulman 2025) |
|
|
| | Parameter | Tulu 3 Recipe | OpenThoughts Recipe | |
| |-----------|---------------|---------------------| |
| | lora_r | 256 | 256 | |
| | lora_alpha | 16 | 16 | |
| | target_modules | all-linear | all-linear | |
| | learning_rate | 5e-6 | 2e-4 | |
| | effective_batch | 128 | 16 | |
| | epochs | 2 | 2 | |
| | max_seq_length | 4096 | 4096 | |
| | lr_schedule | linear | cosine | |
| | packing | True (bfd_split) | True (bfd_split) | |
| | assistant_only_loss | True | True | |
|
|
| ## Key Research Sources |
| - Tulu 3: allenai/Llama-3.1-Tulu-3-8B-SFT model card |
| - LoRA Without Regret: Schulman et al., 2025 |
| - Data quality > quantity: arXiv 2402.05123 |
|
|
