| --- |
| language: en |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-0.5B-Instruct |
| tags: |
| - text-generation |
| - instruction-tuning |
| - lora |
| - peft |
| datasets: |
| - databricks/databricks-dolly-15k |
| --- |
| |
| # text-generation |
|
|
| Fine-tuned version of `Qwen/Qwen2.5-0.5B-Instruct` on the Databricks Dolly-15k instruction dataset using LoRA (PEFT) + TRL's SFTTrainer. |
|
|
| ## How to use |
|
|
| ```python |
| from transformers import pipeline |
| gen = pipeline("text-generation", model="samandar1105/text-generation") |
| result = gen([{"role": "user", "content": "Write a short poem about the ocean."}], max_new_tokens=200) |
| print(result[0]["generated_text"][-1]["content"]) |
| ``` |
|
|
| ## Training details |
| - Base model: Qwen/Qwen2.5-0.5B-Instruct |
| - Method: LoRA (r=16, alpha=32) via PEFT + TRL SFTTrainer |
| - Epochs: 3 |
| - Learning rate: 2e-4 |
|
|
