metadata
language: en
license: apache-2.0
base_model: Qwen/Qwen2.5-0.5B-Instruct
tags:
- text-generation
- instruction-tuning
- lora
- peft
datasets:
- databricks/databricks-dolly-15k
text-generation
Fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the Databricks Dolly-15k instruction dataset using LoRA (PEFT) + TRL's SFTTrainer.
How to use
from transformers import pipeline
gen = pipeline("text-generation", model="samandar1105/text-generation")
result = gen([{"role": "user", "content": "Write a short poem about the ocean."}], max_new_tokens=200)
print(result[0]["generated_text"][-1]["content"])
Training details
- Base model: Qwen/Qwen2.5-0.5B-Instruct
- Method: LoRA (r=16, alpha=32) via PEFT + TRL SFTTrainer
- Epochs: 3
- Learning rate: 2e-4