| title: README | |
| emoji: 🏃 | |
| colorFrom: purple | |
| colorTo: blue | |
| sdk: static | |
| pinned: false | |
| ## Training Small Language Models with Knowledge Distillation | |
| Official pre-trained models and baselines in | |
| + [MiniLLM](https://github.com/microsoft/LMOps/tree/main/minillm): Knowledge distillation of LLMs during instruction tuning. | |
| + [MiniPLM](https://github.com/thu-coai/MiniPLM): Knowledge distillation of LLMs during pre-training. | |