File size: 2,019 Bytes
0c5f5c5 577dc2e 0c5f5c5 577dc2e 0c5f5c5 577dc2e 41803f9 0c5f5c5 41803f9 0c5f5c5 41803f9 0c5f5c5 41803f9 0c5f5c5 41803f9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | ---
base_model: unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
license: apache-2.0
language:
- en
datasets:
- UPB-RAT-Lab/auto-reward-generation
---
# Qwen2.5-Coder-7B-SFT-v1-Grok-300
LoRA adapter fine-tuned using Unsloth on the Auto Reward Generation dataset.
⚠️ **This repository contains LoRA adapter weights only.** You must load a compatible base model before using this adapter.
## Base Model
* Trained on: `unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit`
* Adapter: `UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-grok-300`
## Dataset
* Auto Reward Generation
* https://huggingface.co/datasets/UPB-RAT-Lab/auto-reward-generation
## Setup
Install dependencies:
```bash
pip install transformers peft accelerate bitsandbytes huggingface_hub
```
If the base model requires authentication, log in to Hugging Face:
```bash
huggingface-cli login
```
or in Python:
```python
from huggingface_hub import login
login("YOUR_HF_TOKEN")
```
You can create an access token at:
https://huggingface.co/settings/tokens
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
BASE_MODEL = "unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit"
ADAPTER = "UPB-RAT-Lab/qwen2.5-coder-7b-sft-v1-grok-300"
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
device_map="auto",
)
model = PeftModel.from_pretrained(
model,
ADAPTER,
)
```
## Generate
```python
prompt = "Generate a reward function for a reinforcement learning task."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=256,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
## Notes
* Fine-tuned with Unsloth + LoRA
* Adapter-only repository (no base model weights)
* Intended for reward generation and related coding tasks
* Tested with `unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit` |