Qwen2.5-Coder-7B-SFT-v1-Gemini-300

LoRA adapter fine-tuned using Unsloth on the Auto Reward Generation dataset.

⚠️ This repository contains LoRA adapter weights only. You must load a compatible base model before using this adapter.

Base Model

• Trained on: unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit
• Adapter: UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-gemini-300

Dataset

• Auto Reward Generation
• https://huggingface.co/datasets/UPB-RAT-Lab/auto-reward-generation

Setup

Install dependencies:

pip install transformers peft accelerate bitsandbytes huggingface_hub

If the base model requires authentication, log in to Hugging Face:

huggingface-cli login

or in Python:

from huggingface_hub import login

login("YOUR_HF_TOKEN")

You can create an access token at:

https://huggingface.co/settings/tokens

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit"
ADAPTER = "UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-gemini-300"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    model,
    ADAPTER,
)

Generate

prompt = "Generate a reward function for a reinforcement learning task."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Notes

• Fine-tuned with Unsloth + LoRA
• Adapter-only repository (no base model weights)
• Intended for reward generation and related coding tasks
• Tested with unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit