---
language: en
license: apache-2.0
base_model: Qwen/Qwen2.5-0.5B-Instruct
tags:
- text-generation
- instruction-tuning
- lora
- peft
datasets:
- databricks/databricks-dolly-15k
---

# text-generation

Fine-tuned version of `Qwen/Qwen2.5-0.5B-Instruct` on the Databricks Dolly-15k instruction dataset using LoRA (PEFT) + TRL's SFTTrainer.

## How to use

```python
from transformers import pipeline
gen = pipeline("text-generation", model="samandar1105/text-generation")
result = gen([{"role": "user", "content": "Write a short poem about the ocean."}], max_new_tokens=200)
print(result[0]["generated_text"][-1]["content"])
```

## Training details
- Base model: Qwen/Qwen2.5-0.5B-Instruct
- Method: LoRA (r=16, alpha=32) via PEFT + TRL SFTTrainer
- Epochs: 3
- Learning rate: 2e-4