remora-4b-v0 — a conductor that reads capability cards

An open, 2×24 GB replication of Sakana's Conductor / Fugu-Ultra recipe: Qwen3.5-4B trained with GRPO (LoRA r32) to decompose a task and route each subtask to a worker model from a pool, choosing workers by reading measured capability cards in its system prompt — not by memorizing worker names.

The headline is portability. Trained only on a local pool of 0.35B–8B workers under randomized pseudonyms, display noise, and card schema jitter, the conductor:

• routes identically when workers are renamed/noised/jittered (paired deltas within a few pp);
• routes a held-out worker (never seen in training) at or above that worker's solo score — on mbpp it beats its own best available worker by +10 pp via decomposition;
• routes hosted API workers it has never seen (free and paid pools) 89–100% domain-correctly from their cards alone — and given a premium unseen pool (Qwen3.5-27B + Kimi-K2.7-Code, described only by cards), it beats its own local-pool ceiling by +5 pp on math500 and +17 pp on mbpp with zero retraining: accuracy scales with worker quality.

Full writeup: REPORT.md in the remora repository — recipe, eval battery, honest negative results, reproduction guide.

Results (frozen seed-0 sets, temp 0, 200 tasks/set; humaneval 164)

runner	gsm8k	gsm8k-pert	math500	mbpp	humaneval	balanced
best single worker (gemma-4-e4b solo)	94.0	76.0	83.0	58.5	92.1	80.7
prompted conductor (measured cards)	79.5	61.5	79.0	61.5	91.5	74.6
sft-v1 (bootstrap)	91.5	72.5	72.0	63.5	87.2	77.3
grpo step-50 (checkpoint-50/)	92.5	72.0	72.0	67.0	87.8	78.3
grpo step-75 (this adapter)	93.0	74.5	75.5	64.0	86.0	78.6

Honest bars, stated plainly: against the strongest single worker the conductor wins only on mbpp (+8.5 pp at step-50) — structural, because one worker dominates 4/5 domains in this pool, so routing headroom is thin ("route everything to the top card" is near-optimal). The value is portability + decomposition wins, not raw accuracy on a dominated pool. checkpoint-50/ is the code-best alternate.

Drift gate (does conductor training damage the base?): bare Qwen3.5-4B vs this adapter, both answering solo on the same server and task ids — adapter wins 4/5 sets (gsm8k 80.5→88.5, perturbed 64.5→72.5, math500 44.0→51.0, mbpp 60.0→62.5; humaneval 86.6→84.1). The adapter is a net solo upgrade (+4.6 balanced), so its trained "answer directly" action rests on real skill.

Negative results we report rather than hide: prompt-only worker personas don't survive task-format instructions; a fixed uniform cost penalty collapses to never-delegating; price-target steering did not emerge in 50 GRPO steps (conditioning signals must differentiate rewards at first order, or be seeded in SFT).

What the model emits

Given a system prompt with capability cards (see conductor_prompt_template.txt) and a user task, the conductor outputs three parallel Python lists:

subtasks = ["<instruction for step 0>", "<instruction for step 1>"]
workers  = ["<worker name>", "<worker name>"]
access   = [[], [0]]

Steps run in order; each worker sees only the original task, its own subtask, and the outputs its access list grants. The final step's output is the answer. A trained "self action" also exists: workers = ["self"] with the answer written directly after the fenced block (the conductor solo is surprisingly strong: ≈ full-pool accuracy on 4/5 sets at zero worker cost).

Usage

Serve base + adapter side-by-side with vLLM:

vllm serve Qwen/Qwen3.5-4B \
  --enable-lora --max-lora-rank 32 \
  --lora-modules remora-grpo=marcodsn/remora-4b-v0 \
  --served-model-name qwen3.5-4b-base --port 8000

Then chat with model remora-grpo, system prompt = rendered cards (template and example cards in this repo: conductor_prompt_template.txt, cards/), user = the task, temperature 0. Parse the three lists from the reply and execute them against your own pool (reference executor + parser in the remora repo).

Loading the adapter in transformers: Qwen3.5 is VLM-classified — use AutoModelForImageTextToText (not AutoModelForCausalLM), or PEFT will silently attach a fresh adapter under a remapped module tree.

from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-4B")
model = PeftModel.from_pretrained(base, "marcodsn/remora-4b-v0")

Cards

Routing quality is card quality. Cards carry measured per-domain accuracies from probe suites (train splits only), a contamination gap (original − perturbed accuracy; routing trusts the perturbed number), token/speed stats for cost estimation, and machine-generated pool-relative notes. Examples shipped in cards/ (local pool + hosted API workers). The conductor reads jittered/renamed card fields semantically, so new fields and new workers don't require retraining — that is the point.

Training recipe (condensed)

SFT bootstrap (RAFT harvest, execution-verified, + ≤30% subagent-authored gold) → GRPO: TRL colocate, LoRA r32 all-linear, lr 5e-6 constant, dapo loss, beta 0, scale_rewards none, G=8, temp 1.0, reward 0/0.5/1 executed live against the pool, workflow dedup + reward caching, ≤4h segments with temp-0 eval gates, stopped at step 75 (<0.5 pp/segment). Anti-memorization during training: random k-of-n pools, per-rollout aliases, ±0.02 order-preserving display noise, card schema jitter, mechanically-enforced personas, hardware-enforced held-out worker. ~12–15 GPU-hours total on an RTX 3090 + RTX PRO 4000.

License

Apache-2.0 (adapter and code). Base model Qwen/Qwen3.5-4B is Apache-2.0.