Instructions to use gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode")
model = AutoModelForCausalLM.from_pretrained("gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode

SGLang

How to use gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode with Docker Model Runner:
```
docker model run hf.co/gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode
```

Qwen2.5-Coder-7B-Instruct, GRPO on LeetCode (step 90)

Qwen/Qwen2.5-Coder-7B-Instruct post-trained with GRPO on LeetCode problems. The reward comes from running the generated code against the problem's unit tests in a sandbox. This is the step-90 checkpoint of a 180-step run.

Code, configs and technical report: github.com/giovanni-vaccarino/grpo-code

Results

Held-out set: the 228 problems of LeetCodeDataset's test split, released after mid-2024. Each task gets 10 samples at temperature 0.8 plus one greedy response, graded by the same sandboxed verifier used in training.

	Base model	This model
LeetCode held-out pass@1, sampled	14.1%	21.9%
LeetCode held-out pass@1, greedy	14.9%	20.2%
LeetCode held-out pass@5	24.3%	28.2%
LeetCode held-out pass@10	28.5%	28.9%
pass@1 Easy / Medium / Hard (48 / 101 / 79 tasks)	44.6 / 7.9 / 3.4%	56.0 / 17.1 / 7.3%
HumanEval+ pass@1 / pass@10	83.0 / 93.9%	85.6 / 90.2%
MBPP+ pass@1 / pass@10	68.1 / 85.2%	72.5 / 82.8%

Training


Algorithm	GRPO: 8 responses per prompt, advantage = (reward − group mean) / group std, PPO-clip 0.2, no KL term, no critic
Reward	1 if every unit test passes, else 0. The code runs sandboxed (user, network, PID and mount namespaces; 6 CPU-seconds per response)
Data	LeetCodeDataset train split minus 611 tasks every earlier response had solved: 2,030 tasks. 90 steps × 64 prompts ≈ 2.8 passes
Hyperparameters	lr 1e-6 (constant), AdamW, 64 prompts per step, 4 optimizer steps per batch, grad clip 1.0, temperature 1.0, max response 1,024 tokens
Stack	veRL 0.9.1 (sync trainer), vLLM 0.24 rollouts, FSDP2, 4 × GH200. vLLM's log-probs were reused as the old policy's
Precision	Trained with fp32 master weights and bf16 compute; exported in bf16

Usage

Use the base model's chat template. Training prompts gave the problem statement and LeetCode's starter code, and ended with: "Write your solution in Python 3 and put the complete code in a single ```python block."

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto")
messages = [{"role": "user", "content": "Write a Python function two_sum(nums, target) that returns the indices of the two numbers adding up to target."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(inputs, max_new_tokens=512)[0][inputs.shape[1]:], skip_special_tokens=True))

Downloads last month: 56

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-Coder-7B

Finetuned

Qwen/Qwen2.5-Coder-7B-Instruct

Finetuned

(493)

this model

gianniiii
/

Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode

Qwen2.5-Coder-7B-Instruct, GRPO on LeetCode (step 90)

Results

Training

Usage

Model tree for gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode

Dataset used to train gianniiii/Qwen2.5-Coder-7B-Instruct-GRPO-LeetCode