Update README.md
Browse files
README.md
CHANGED
|
@@ -141,6 +141,8 @@ We would like to thank the dataset providers and the open-source community whose
|
|
| 141 |
- **CAIS** for the **MMLU** dataset used for auxiliary knowledge and reasoning supervision.
|
| 142 |
- **HuggingFaceTB** for the **OpenHermes-2.5-H4** dataset used in the final instruction refinement phase.
|
| 143 |
- **Weights & Biases (W&B)** for logging and visualization tools used to monitor training progress.
|
|
|
|
|
|
|
| 144 |
|
| 145 |
|
| 146 |
We also acknowledge the broader open-source research community for their continuous efforts in advancing efficient model architectures and training methodologies.
|
|
|
|
| 141 |
- **CAIS** for the **MMLU** dataset used for auxiliary knowledge and reasoning supervision.
|
| 142 |
- **HuggingFaceTB** for the **OpenHermes-2.5-H4** dataset used in the final instruction refinement phase.
|
| 143 |
- **Weights & Biases (W&B)** for logging and visualization tools used to monitor training progress.
|
| 144 |
+
- Additionally, we drew valuable insights from **The Smol Training Playbook: The Secrets to Building World-Class LLMs**, published by Hugging Face, which informed several practical decisions in our training and experimentation workflow.
|
| 145 |
+
Playbook link: https://huggingfacetb-smol-training-playbook.hf.space/the-smol-training-playbook-the-secrets-to-building-world-class-llms.pdf
|
| 146 |
|
| 147 |
|
| 148 |
We also acknowledge the broader open-source research community for their continuous efforts in advancing efficient model architectures and training methodologies.
|