Spaces:
Running
Running
| title: README | |
| emoji: 👁 | |
| colorFrom: purple | |
| colorTo: green | |
| sdk: static | |
| pinned: false | |
| # Hugging Face Smol Models Research | |
| This is the home for smol models (SmolLM & SmolVLM) and high quality pre-training datasets. We released: | |
| **News 🗞️** | |
| - **The Smol Training Playbook**: a comprehensive guide to training world-class LLMs [HuggingFaceTB/smol-training-playbook](https://huggingface.co/spaces/HuggingFaceTB/smol-training-playbook) | |
| **Past releases** | |
| - [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu): a filtered version of FineWeb dataset for educational content, paper available [here](https://huggingface.co/papers/2406.17557). | |
| - [Cosmopedia](https://huggingface.co/datasets/HuggingFaceTB/cosmopedia): the largest open synthetic dataset, with 25B tokens and 30M samples. It contains synthetic textbooks, blog posts, and stories, posts generated by Mixtral. Blog post available [here](https://huggingface.co/blog/cosmopedia). | |
| - [Smollm-Corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus): the pre-training corpus of SmolLM: **Cosmopedia v0.2**, **FineWeb-Edu dedup** and **Python-Edu**. Blog post available [here](https://huggingface.co/blog/smollm). | |
| - [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath): the best public math pretraining dataset with 50B tokens of mathematical and problem solving data. | |
| - [Stack-Edu](https://huggingface.co/datasets/HuggingFaceTB/stack-edu): the best open code pretraining dataset with educational code in 15 programming languages. | |
| - [SmolLM2 models](https://huggingface.co/collections/HuggingFaceTB/smollm2-checkpoints-6723884218bcda64b34d7db9): a series of strong small models in three sizes: 135M, 360M and 1.7B | |
| - [SmolVLM2](https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct): a family of small **Video and Vision** models in three sizes: 2.2B, 500M and 256M. Blog post available [here](https://huggingface.co/blog/smolvlm2). | |
| - [SmolLM3](https://huggingface.co/HuggingFaceTB/SmolLM3-3B): SOTA 3B model with dual **reasoning**, supports 6 languages and long context with strong function calling. SmolLM3 Engineering Blueprint available [here](https://huggingface.co/datasets/HuggingFaceTB/smollm3-blueprint/blob/main/smollm3-blueprint.pdf) | |
| <div align="center"> | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/61c141342aac764ce1654e43/UXSo9zzL7PFvrLCAQfcnz.png" width="700"/> | |
| </div> | |