metadata
base_model: unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
license: apache-2.0
language:
- en
datasets:
- UPB-RAT-Lab/auto-reward-generation
Qwen2.5-Coder-7B-SFT-v1-Gemini-300
LoRA adapter fine-tuned using Unsloth on the Auto Reward Generation dataset.
⚠️ This repository contains LoRA adapter weights only. You must load a compatible base model before using this adapter.
Base Model
- Trained on:
unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit - Adapter:
UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-gemini-300
Dataset
- Auto Reward Generation
- https://huggingface.co/datasets/UPB-RAT-Lab/auto-reward-generation
Setup
Install dependencies:
pip install transformers peft accelerate bitsandbytes huggingface_hub
If the base model requires authentication, log in to Hugging Face:
huggingface-cli login
or in Python:
from huggingface_hub import login
login("YOUR_HF_TOKEN")
You can create an access token at:
https://huggingface.co/settings/tokens
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
BASE_MODEL = "unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit"
ADAPTER = "UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-gemini-300"
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
device_map="auto",
)
model = PeftModel.from_pretrained(
model,
ADAPTER,
)
Generate
prompt = "Generate a reward function for a reinforcement learning task."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=256,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Notes
- Fine-tuned with Unsloth + LoRA
- Adapter-only repository (no base model weights)
- Intended for reward generation and related coding tasks
- Tested with
unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit