Qwen3.5-9B-GRPO-Apr20

A LoRA adapter on top of Qwen3.5-9B.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-9B",
    torch_dtype=torch.bfloat16,
    attn_implementation="sdpa",
)
model = PeftModel.from_pretrained(base, "Debargha/Qwen3.5-9B-GRPO-Apr20")
tok = AutoTokenizer.from_pretrained("Debargha/Qwen3.5-9B-GRPO-Apr20")

Or with vLLM:

vllm serve Qwen/Qwen3.5-9B \
  --enable-lora \
  --lora-modules grpo=Debargha/Qwen3.5-9B-GRPO-Apr20 \
  --max-lora-rank 16 \
  --language-model-only

LoRA rank 16, alpha 32, targets q_proj k_proj v_proj o_proj gate_proj up_proj down_proj.