Instructions to use ecnu-icalk/SkillGym-Agent with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ecnu-icalk/SkillGym-Agent with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ecnu-icalk/SkillGym-Agent")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ecnu-icalk/SkillGym-Agent")
model = AutoModelForMultimodalLM.from_pretrained("ecnu-icalk/SkillGym-Agent", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ecnu-icalk/SkillGym-Agent with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ecnu-icalk/SkillGym-Agent"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ecnu-icalk/SkillGym-Agent",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ecnu-icalk/SkillGym-Agent

SGLang

How to use ecnu-icalk/SkillGym-Agent with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ecnu-icalk/SkillGym-Agent" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ecnu-icalk/SkillGym-Agent",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ecnu-icalk/SkillGym-Agent" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ecnu-icalk/SkillGym-Agent",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ecnu-icalk/SkillGym-Agent with Docker Model Runner:
```
docker model run hf.co/ecnu-icalk/SkillGym-Agent
```

SkillGym-Agent

A Qwen3.5-35B-A3B agent checkpoint trained on verified SkillGym trajectories

English · 简体中文

GitHub · Dataset · Paper

Model Summary

SkillGym-Agent is a full-parameter supervised fine-tuned version of Qwen3.5-35B-A3B, trained on successful, verifier-confirmed trajectories released with SkillGym.

The released checkpoint corresponds to the paper's All Teachers setting and is trained on the full set of 8,364 successful trajectories collected across the released teacher–harness configurations.

The model is designed to study whether procedural experience generated from human-written skills can become reusable capability inside an LLM agent, including when the external skill is removed at inference time.

Core idea: learn from verified experience generated with external skills, then evaluate how much of that capability is retained without those skills.

Item	Description
Released paper setting	All Teachers
Base model	Qwen/Qwen3.5-35B-A3B
Architecture	Qwen3.5 MoE, ~35.95B parameters
Training method	Full-parameter supervised fine-tuning
Training data	8,364 successful SkillGym trajectories
Source environments	2,756 accepted environments / 5,512 paired variants
Coverage	12 major categories / 63 sub-categories
Training stack	ms-swift / Megatron
Reported hardware	16 × NVIDIA H200 GPUs
License	Apache-2.0

Quick Start

Download

python -m pip install -U huggingface_hub

hf download ecnu-icalk/SkillGym-Agent \
  --local-dir .hf/skillgym-agent

Load with Transformers

from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "ecnu-icalk/SkillGym-Agent"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

SkillGym-Agent was evaluated through agentic harnesses rather than a single fixed chat template. Prompt formatting, tool schemas, runtime configuration, and the agent loop should therefore follow the selected harness and the current Qwen3.5 interface.

Loading the checkpoint alone does not reproduce the reported benchmark scores. Those results depend on the full harness, system prompt, runtime, inference budget, and benchmark configuration.

Evaluation

Higher is better for every metric. GDPval-AA v2 is reported as Elo; the remaining metrics are task success rates (%).

Harness	Model	GDPval-AA v2 (Elo) ↑	Terminal-Bench 2.1 (%) ↑	SkillsBench v1.1 w/ Skills (%) ↑	SkillsBench v1.1 w/o Skills (%) ↑
Codex	Qwen3.5-35B-A3B	942	10.11	5.33	0.69
Codex	SkillGym-Agent	979 (+37)	46.07 (+35.96)	33.02 (+27.69)	21.08 (+20.39)
Claude Code	Qwen3.5-35B-A3B	974	39.33	23.34	12.13
Claude Code	SkillGym-Agent	1173 (+199)	58.43 (+19.10)	51.47 (+28.13)	26.81 (+14.68)

Parenthesized values are absolute gains over the same-harness base model: Elo points for GDPval-AA v2 and percentage points for the success-rate metrics.

Skill-Free Transfer

A central result is the capability retained after removing the inference-time skill:

Claude Code: SkillGym-Agent reaches 26.81% without skills, compared with 23.34% for the base model with skills.
Codex: SkillGym-Agent reaches 21.08% without skills, compared with 5.33% for the base model with skills.

Performance remains strongest when external skills are available, suggesting that internalized capability and explicit skills are complementary.

Evaluation Notes

Under Claude Code, the base and trained models use the same standard system prompt.
Under Codex, the base uses the standard prompt while SkillGym-Agent uses the no-applypatch prompt; the Codex delta is therefore not a pure fine-tuning-only comparison.
Benchmark results depend on the harness, prompt, runtime, inference budget, checkpoint, and benchmark version.
Public-reference results may use different evaluation configurations and should not be interpreted as strictly controlled head-to-head comparisons.

See the paper for teacher/harness ablations and the complete evaluation configuration.

Training Data

The checkpoint is trained on successful trajectories from the SkillGym dataset.

The Dataset Card is the canonical reference for:

human-written skills and supporting assets;
reusable task templates and fixtures;
executable environments and verifiers;
paired with-skill / without-skill variants;
trajectory collections and schema differences;
dataset provenance and artifact-specific licensing.

Intended Use

SkillGym-Agent is intended for research on:

agentic problem solving and tool use;
long-horizon workflow learning;
skill-conditioned and skill-free execution;
transfer from verified trajectories;
executable-environment and verifier-based evaluation.

The checkpoint is not presented as a general safety guarantee or as a drop-in production agent. Users should independently validate tool permissions, sandbox isolation, generated code, and external side effects.

Limitations

The released checkpoint does not include a complete agent runtime or tool environment.
Reported benchmark scores require the corresponding evaluation harness and configuration.
The current release does not include a standalone end-to-end training script or a single script reproducing every external benchmark result.
Training data is derived from the released SkillGym environment and trajectory distribution and should not be treated as exhaustive coverage of real-world procedural tasks.
External skills remain useful at inference time even after training on skill-generated experience.

Project Resources

Resource	Scope
GitHub / SkillGym	Task Builder, documentation, figures, and paper materials
SkillGym Dataset	Skills, templates, environments, and trajectories
Paper	Method, dataset analysis, experiments, and ablations

Citation

@misc{skillgym2026,
  title = {Internalizing Large-Scale Human-Written Skills into LLMs for Real-World Problem Solving},
  year  = {2026},
  url   = {https://github.com/ECNU-ICALK/SkillGym}
}

License

The SkillGym-Agent checkpoint is released under the Apache-2.0 license.

The companion dataset contains source skills, fixtures, and supporting assets that may retain upstream notices or additional terms. See the Dataset Card for artifact-specific licensing details.

Downloads last month: 56

Safetensors

Model size

36B params

Tensor type

BF16

Model tree for ecnu-icalk/SkillGym-Agent

Base model

Qwen/Qwen3.5-35B-A3B-Base

Finetuned

Qwen/Qwen3.5-35B-A3B