|
|
--- |
|
|
tags: |
|
|
- causal-lm |
|
|
- transformers |
|
|
- finetuned |
|
|
- instruction-following |
|
|
- dpo |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- agentlans/crash-course |
|
|
- Intel/orca_dpo_pairs |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- HuggingFaceTB/SmolLM2-135M-Instruct |
|
|
--- |
|
|
# SmolLM2-135M-Instruct-Plus |
|
|
|
|
|
This model is a finetuned version of [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct), aiming to maximize knowledge in a small 135M parameter model. |
|
|
|
|
|
> [!WARNING] |
|
|
> ⚠️ Consider this model a creative text generator. |
|
|
> Without additional finetuning, it gives wildly inaccurate answers. Don't trust the output of this model without additional verification. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Base Model:** [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) |
|
|
- **Finetuning Datasets:** |
|
|
- [agentlans/crash-course](https://huggingface.co/datasets/agentlans/crash-course) (120K subset) |
|
|
- [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs) |
|
|
- **Training Procedure:** |
|
|
1. Supervised Fine-Tuning (SFT) on `crash-course` for 1 epoch. |
|
|
2. Direct Preference Optimization (DPO) on `orca_dpo_pairs`. |
|
|
|
|
|
## Intended Uses |
|
|
|
|
|
For research, experimentation, and educational purposes where a small instruction-following model is desired. |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- **Hallucinations:** Prone to generating incorrect information due to its small size. |
|
|
- **Repetitive Output:** May produce repetitive text. |
|
|
|
|
|
## Training Details |
|
|
|
|
|
Both SFT and DPO share common settings: liger_kernel booster, LoRA fine-tuning, custom model, BF16 compute type, batch size of 2, and a cosine scheduler with a learning rate of 5e-5. RSLoRA is enabled with a rank of 16 and alpha of 32. |
|
|
|
|
|
The main differences are in the dataset and training specifics. SFT uses CrashCourse_120K with packing enabled and LoRA dropout of 0, while DPO uses orca_pairs with packing disabled and a LoRA dropout of 0.95. |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
Provides coherent and creative answers but may often be incorrect. Thorough evaluation is recommended before deployment. |
