metadata
license: apache-2.0
base_model: Qwen/Qwen2.5-0.5B-Instruct
base_model_relation: adapter
library_name: transformers
pipeline_tag: text-generation
datasets:
- train.jsonl
- eval.jsonl
tags:
- fine-tuned
- adapter
- peft
- lora
- qlora
- trl
- ryotenkai
helixql-nl2hql
This repository contains a PEFT LoRA adapter for Qwen/Qwen2.5-0.5B-Instruct.
Model Details
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen2.5-0.5B-Instruct |
| Adapter repo | Tranium/helixql-nl2hql |
| Training type | qlora |
| Strategy chain | SFT (3ep) |
| Dataset source | local |
| Datasets | train.jsonl, eval.jsonl |
| Batch Size | 8 |
| LoRA r | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| LoRA bias | none |
| Target modules | all-linear |
| DoRA | False |
| rsLoRA | False |
| Init | gaussian |
Training Details
| Hyperparameter | Value |
|---|---|
| epochs | 3 |
| learning_rate | 0.0002 |
| warmup_ratio | 0.05 |
| per_device_train_batch_size | 8 |
| gradient_accumulation_steps | 4 |
| effective_batch_size | 32 |
| optimizer | paged_adamw_8bit |
| lr_scheduler | cosine |
Training Results
Run timeline
- Started at:
2026-04-21T17:00:01.373549 - Completed at:
2026-04-21T17:01:19.888146
| Phase | Strategy | Status | train_loss | eval_loss | global_step | epoch | runtime_s | peak_mem_gb |
|---|---|---|---|---|---|---|---|---|
| 0 | sft | completed | 2.1162 | — | 12 | 3.00 | 71.6 | 3.75 |
Usage
Load as a PEFT adapter (recommended)
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_id = "Qwen/Qwen2.5-0.5B-Instruct"
adapter_id = "Tranium/helixql-nl2hql"
tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=False)
model = AutoModelForCausalLM.from_pretrained(base_model_id, device_map="auto", torch_dtype="auto", trust_remote_code=False)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()
prompt = "Hello!"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Merge adapter into base model (optional)
merged = model.merge_and_unload()
merged.save_pretrained("merged-model")
tokenizer.save_pretrained("merged-model")
Training Infrastructure
- Platform: runpod
- GPU: NVIDIA RTX A4000