README.md

---
library_name: transformers
tags:
- unsloth
license: apache-2.0
datasets:
- UPB-RAT-Lab/auto-reward-generation
language:
- en
base_model:
- unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit
---

# Qwen2.5-Coder-7B-SFT-v1-ChatGPT-300

LoRA adapter fine-tuned using Unsloth on the Auto Reward Generation dataset.

⚠️ **This repository contains LoRA adapter weights only.** You must load a compatible base model before using this adapter.

## Base Model

* Trained on: `unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit`
* Adapter: `UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-chatgpt-300`

## Dataset

* Auto Reward Generation
* https://huggingface.co/datasets/UPB-RAT-Lab/auto-reward-generation

## Setup

Install dependencies:

```bash
pip install transformers peft accelerate bitsandbytes huggingface_hub
```

If the base model requires authentication, log in to Hugging Face:

```bash
huggingface-cli login
```

or in Python:

```python
from huggingface_hub import login

login("YOUR_HF_TOKEN")
```

You can create an access token at:

https://huggingface.co/settings/tokens

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit"
ADAPTER = "UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-chatgpt-300"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    model,
    ADAPTER,
)
```

## Generate

```python
prompt = "Generate a reward function for a reinforcement learning task."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Notes

* Fine-tuned with Unsloth + LoRA
* Adapter-only repository (no base model weights)
* Intended for reward generation and related coding tasks
* Tested with `unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit`