harness-hub
AI & ML interests
None defined yet.
Recent Activity
Articles
harness-hub
Open models, tuned for the agent harness they run in.
The same model behaves differently in Pi, OpenCode, Hermes or Claude Code: each harness has its own tools, prompts and control flow. We train harness adapters: small LoRA adapters, one per model and harness, trained with reinforcement learning inside the unmodified harness. You keep your base model and load the adapter for the agent you use.
Harness adapters
| Base model | Harnesses | |
|---|---|---|
| Qwen3.8-27B | Pi (OpenCode and Hermes coming) | collection |
| LFM2.5-24B-A2B | Pi, Claude Code, Hermes, OpenCode | collection |
| LFM3-1.7B | Pi, Hermes, OpenCode | collection |
| TwIL-8B (experiment) | Pi, Hermes, OpenCode | collection |
Every adapter has a -GGUF companion for llama.cpp.
Qwen3.8-27B · Pi
harness-hub/Qwen3.8-27B-pi-adapted · GGUF
Paired against the base model under Pi, one attempt per task, thinking off for both:
| Benchmark | Base | Adapter | Tool calls per task |
|---|---|---|---|
| Terminal-Bench 2.1 (86 tasks) | 39.5% | 55.8% | −62% |
| SWE-bench Pro, 51 hardest tasks | 29.4% | 62.7% | −55% |
The model card has the intervals, sign tests and a run on the Terminal-Bench tasks not used to pick the checkpoint.
With thinking on. The adapter was trained with thinking off. Here it is against the base model in both modes, under Pi:
| SWE-bench Pro, 51 hardest | Terminal-Bench 2.1 (83 tasks) | Tokens per task (median, SWE-bench Pro / TB) | |
|---|---|---|---|
| Base, no thinking | 29.4% | 41.0% | 13.1k / 6.9k |
| Base, thinking | 64.7% | 48.2% | 34.4k / 16.4k |
| Pi adapter, no thinking | 62.7% | 56.6% | 7.9k / 3.7k |
| Pi adapter, thinking | 66.7% | 53.0% | 32.7k / 16.4k |
Without thinking, the adapter is close to the base model with thinking on SWE-bench Pro (62.7% vs 64.7%) and ahead on Terminal-Bench 2.1 (56.6% vs 48.2%), using about 4x fewer tokens per task. Tokens count everything the model generates: text, tool calls and reasoning.
Datasets and benchmarks
- traj2envs-rl: 270 Harbor RL tasks built from agent sessions shared on the Hub.
- safeagents-judge-bench: 661 real agent tool calls, labelled allow or block, to score the decision models behind safeagents.
Spaces
- Harness Lens: compare coding-agent harnesses (prompts, tools, compaction, permissions), every claim linked to source.
- Open-source harnesses analysis: every tool in eleven open-source coding harnesses, side by side.
Train your own
- Blog: Harness adapters: Safe RL for efficient agentic development
- Training script:
examples/async_grpo_harborin TRL (AsyncGRPO + LoRA, Harbor tasks via OpenEnv) - Turn agent sessions into RL tasks: Repo2RLEnv
- Guard tool calls during RL: safeagents
If you train an adapter for a harness we haven't covered, open a discussion on any of our repos.