Instructions to use FineEnvs/Qwen3.5-2B-multiharness-RL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FineEnvs/Qwen3.5-2B-multiharness-RL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FineEnvs/Qwen3.5-2B-multiharness-RL")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("FineEnvs/Qwen3.5-2B-multiharness-RL")
model = AutoModelForMultimodalLM.from_pretrained("FineEnvs/Qwen3.5-2B-multiharness-RL", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FineEnvs/Qwen3.5-2B-multiharness-RL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FineEnvs/Qwen3.5-2B-multiharness-RL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FineEnvs/Qwen3.5-2B-multiharness-RL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FineEnvs/Qwen3.5-2B-multiharness-RL

SGLang

How to use FineEnvs/Qwen3.5-2B-multiharness-RL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FineEnvs/Qwen3.5-2B-multiharness-RL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FineEnvs/Qwen3.5-2B-multiharness-RL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FineEnvs/Qwen3.5-2B-multiharness-RL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FineEnvs/Qwen3.5-2B-multiharness-RL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use FineEnvs/Qwen3.5-2B-multiharness-RL with Docker Model Runner:
```
docker model run hf.co/FineEnvs/Qwen3.5-2B-multiharness-RL
```

Qwen3.5-2B-multiharness-RL

A full fine-tune of Qwen/Qwen3.5-2B for agentic data-analysis tasks, trained with asynchronous GRPO (TRL Async GRPO) using OpenCode, Claude Code, Codex, Mini-SWE-Agent. This release is step 500 on main, scoring 37.0% pass@1 across four evaluation harnesses.

Article · Collection · Evaluation tasks · Training dashboard

Training and evaluation curves

Training curves use a trailing 50-step mean (at least 10 observations). Evaluation markers show measured checkpoints; hollow markers and dotted segments indicate incomplete coverage. Faint background lines show the full recorded trajectory; colored lines reveal the measured checkpoints. The first frame shows the completed chart before replaying, so previews also contain the full curves. The tool-use panel shows absolute calls per graded evaluation rollout; this Qwen run used correctness-only rewards. The animation stops at this revision's step 500.

Static chart · Plotted data · Interactive article

Training

TRL Async GRPO with binary correctness only, initialized from the base model. Each group uses one task and one harness; harnesses rotate across groups. Rollouts run through OpenEnv × Harbor in E2B sandboxes.

Setting	Value
Training task pool	1,000 tasks: 150 easy / 600 medium / 250 hard
Checkpoint	500 optimizer steps
Learning rate	3e-6
Rollouts per GRPO group / maximum staleness	8 / 4 optimizer steps
Optimizer / precision	paged AdamW 8-bit / bfloat16
Training harnesses	OpenCode, Claude Code, Codex, Mini-SWE-Agent
Sampling temperature / top-p	0.8 / 1.0
Per-call output budget, training / evaluation	16,384 / 4,096 tokens

These are the Qwen ablations from the article. Their reward has no tool-efficiency bonus. The output-budget mismatch and other limitations are discussed in the article; later Harbor checkpoints declined, so this release preserves the selected checkpoint rather than substituting the last one.

Evaluation

250 fixed SmolDataEnvs test tasks (33 easy, 118 medium, 99 hard), each evaluated under four harnesses: 1,000 graded task/harness cells. Pass@1 uses the first graded attempt per cell; infrastructure retries do not turn it into pass@k. All cells below are graded.

Harness	Correct / evaluated	Pass@1
OpenCode	82 / 250	32.8%
Claude Code	112 / 250	44.8%
Codex	98 / 250	39.2%
Mini-SWE-Agent	78 / 250	31.2%
Overall	370 / 1,000	37.0%

Harness versions: OpenCode 1.18.31, Claude Code 2.1.270, Codex 0.154.0, Mini-SWE-Agent 2.4.6. Full scores, including difficulty breakdowns, are in eval_results.json.

This is the run's best observed checkpoint on the same test set (also the final checkpoint for standalone OpenCode). Selection on test performance can inflate the reported result.

These are single-run results on a specific task set and harness versions. Harness mix, training exposure and compute differ across runs; the scores do not isolate a causal effect of the harness or objective.

Load the checkpoint

The repository contains full saved model weights and the saved tokenizer/chat template, not a LoRA adapter. Training used Transformers 5.14.1; use a compatible Transformers release.

import torch
from transformers import AutoTokenizer, AutoModelForImageTextToText

model_id = "FineEnvs/Qwen3.5-2B-multiharness-RL"
revision = "main"
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, revision=revision, dtype=torch.bfloat16, device_map="auto"
)

Reproducing the task scores requires the agent harness and tools described in the article; a plain chat prompt is not the same evaluation. This Qwen release was fine-tuned and evaluated for text/tool interaction; no image benchmark is claimed.

Related releases

All models, datasets, environments and the article are linked in the multi-harness RL collection.

License and provenance

Derived from Qwen/Qwen3.5-2B under Apache License 2.0. The base model's license is included unchanged in LICENSE. FineEnvs modified the weights by reinforcement learning; this is not an official Qwen release. See NOTICE and release_manifest.json for modification notices, the pinned base revision, checkpoint identity and file checksums. Optimizer, scheduler, RNG and trainer state are excluded.

Citation

For the experiment, methodology and interpretation, cite the main article:

@misc{kolavi2026multiharnessrl,
  author = {Adithya S Kolavi},
  title = {The ultimate guide to multi-harness RL},
  year = {2026},
  url = {https://huggingface.co/spaces/AdithyaSK/multi-harness-rl}
}

Downloads last month: 304

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for FineEnvs/Qwen3.5-2B-multiharness-RL

Base model

Qwen/Qwen3.5-2B-Base

Finetuned

Qwen/Qwen3.5-2B

Finetuned

(474)

this model

Dataset used to train FineEnvs/Qwen3.5-2B-multiharness-RL

Collection including FineEnvs/Qwen3.5-2B-multiharness-RL

SmolDataEnvs Multi-harness RL

Collection

Try the three environments, follow the tutorial, and explore the datasets, models and historical RL/SFT results. • 22 items • Updated 2 days ago • 3

Evaluation results

Pass@1 (%) — Overall on SmolDataEnvs (250 fixed test tasks)
test set self-reported

37.000
Pass@1 (%) — OpenCode on SmolDataEnvs (250 fixed test tasks)
test set self-reported

32.800
Pass@1 (%) — Claude Code on SmolDataEnvs (250 fixed test tasks)
test set self-reported

44.800
Pass@1 (%) — Codex on SmolDataEnvs (250 fixed test tasks)
test set self-reported

39.200
Pass@1 (%) — Mini-SWE-Agent on SmolDataEnvs (250 fixed test tasks)
test set self-reported

31.200