|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: HuggingFaceTB/SmolLM3-3B |
|
|
tags: |
|
|
- trl |
|
|
- unsloth |
|
|
- smollm3 |
|
|
- text-generation-inference |
|
|
- transformers |
|
|
library_name: transformers |
|
|
model_name: Generator3B-V0.2 |
|
|
datasets: |
|
|
- GODELEV/Golden-Dataset-Beta3 |
|
|
--- |
|
|
|
|
|
# Generator3B-V0.2 |
|
|
|
|
|
This model is a fine-tuned version of [HuggingFaceTB/SmolLM3-3B](https://huggingface.co/HuggingFaceTB/SmolLM3-3B). |
|
|
It was trained using **Unsloth** and the **SFTTrainer** on a specialized 'Golden Dataset' for high-quality instruction following. |
|
|
|
|
|
## Model Details |
|
|
- **Developed by:** GODELEV |
|
|
- **Model type:** Causal Language Model |
|
|
- **Language(s):** English |
|
|
- **License:** Apache 2.0 |
|
|
- **Fine-tuned from model:** SmolLM3-3B-Instruct |
|
|
|
|
|
## Training Procedure |
|
|
The model was trained with maximum speed optimizations including: |
|
|
- **Sequence Packing**: Enabled |
|
|
- **4-bit Quantization**: Bitsandbytes |
|
|
- **LoRA Rank**: 64 |
|
|
- **Optimizers**: AdamW 8-bit |
