base-agent-0.6b
The Orama Studio base agent β turns a plain-language request into a verified Pipeline, answers from retrieved facts, asks when a request is ambiguous, declines what is out of scope.
Private. Trained on private product data.
Trained from Qwen/Qwen3-0.6B on
OramaSearch/base-agent-interaction-corpus
with zetta's own native trainer β no foreign runtime (ADR-17). Each version is stored under
v<N>/, matching the corpus version it learned from.
Output contract
One JSON action per turn:
{"kind": "action|answer|clarify|abstain", "message": "...", "tool_calls": [{"tool": "...", "arguments": {}}]}
It emits tool calls, never a graph. The harness builds the Pipeline by running them and mints
the node ids. Asked to write ids directly, a model can only learn what a UUID looks like β an early
checkpoint emitted 11111111-β¦ for every node, so two steps collided and the Pipeline would not
compile.
message is what a user reads. Raw JSON reaching a user is a harness bug, not a model bug β
parse_studio_action is the boundary.
Versions
| corpus | canary grader | note | |
|---|---|---|---|
| v3 | v3 (6,102) | 47% | first with tool calls; one pipeline shape for every ask |
| v7 | v7 (12,744) | 98% | use-case-coherent corpus; Project name predicts its pipeline |
| v9 | v9 (14,889) | 97.5% | ungrounded coverage β the turns a demo opens with |
| v10 | v10 (15,069) | pending | multi-turn follow-ups, teacher-authored objection answers |
The v3 β v7 jump was not a model change. The corpus had been teaching noise: a Project called "Lifetime value forecast" required a gbdt classifier, so the only learnable lesson was that the Project name predicts nothing.
Read this before diagnosing it
A canary score does not predict demo behaviour. v7 scored 98% held-out and still failed three demo
beats, because the canary carries retrieved context and a conversation's opening turns do not. Run
both EvaluateBaseAgentOnCanary and scripts/demo-smoke.sh.
Known limitations
- Ungrounded product answers blend. Trained against a capability listing; served without one it interpolates between memorized answers. Mitigated in v10, not verified.
- Follow-ups need v10 or later. Earlier versions were trained with no conversation history at all, so every turn was a first turn.
- The corpus is ~354 distinct asks presented many ways. A model that memorizes those will score well while having generalized less than the row count suggests.
- RLVR is wired but does not produce these checkpoints. SFT does; goal-anchored GRPO moves a policy and has not yet been run at length.