shrango/qwen3-4b-base-taco-grpo-lorem

Qwen3-4B-Base — TACO code RL (lorem)

RL-finetuned from Qwen/Qwen3-4B-Base with GRPO on the taco subset of agentica-org/DeepCoder-Preview-Dataset, using an execution-based reward (generated programs are run against the problem's stdin/stdout test cases).

Method: GRPO + LoPE lorem-ipsum prompt perturbation resampling

Results (pass@1, %)

Benchmark	This model	Qwen3-4B-Base (before RL)
HumanEval	85.98	76.83
MBPP	66.00	49.00
Average	75.99	62.91

All four method variants trained under identical settings:

Method	HumanEval	MBPP	Average
baseline (GRPO)	84.15	60.60	72.38
naive resample	84.76	57.40	71.08
lorem	85.98	66.00	75.99
lorem + shaping	81.71	66.20	73.96

Evaluated with evalscope, pass@1, single sample, temperature=0, served via vLLM. Note: training prompts asked the model to reason inside <think></think> tags, while the evaluation used the benchmarks' default (non-think) prompts.

Training setup

• Framework: verl / EasyR1 fork
• Steps: 174 (3 epochs over 7432 problems)
• Rollout: n=8 per prompt, batch size 128
• Max length: 2048 prompt / 8192 response
• KL: disabled
• Hardware: 8×A100 (single node)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "shrango/qwen3-4b-base-taco-grpo-lorem"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")