base-agent-0.6b

The Orama Studio base agent β€” turns a plain-language request into a verified Pipeline, answers from retrieved facts, asks when a request is ambiguous, declines what is out of scope.

Private. Trained on private product data.

Trained from Qwen/Qwen3-0.6B on OramaSearch/base-agent-interaction-corpus with zetta's own native trainer β€” no foreign runtime (ADR-17). Each version is stored under v<N>/, matching the corpus version it learned from.

Output contract

One JSON action per turn:

{"kind": "action|answer|clarify|abstain", "message": "...", "tool_calls": [{"tool": "...", "arguments": {}}]}

It emits tool calls, never a graph. The harness builds the Pipeline by running them and mints the node ids. Asked to write ids directly, a model can only learn what a UUID looks like β€” an early checkpoint emitted 11111111-… for every node, so two steps collided and the Pipeline would not compile.

message is what a user reads. Raw JSON reaching a user is a harness bug, not a model bug β€” parse_studio_action is the boundary.

Versions

corpus canary grader note
v3 v3 (6,102) 47% first with tool calls; one pipeline shape for every ask
v7 v7 (12,744) 98% use-case-coherent corpus; Project name predicts its pipeline
v9 v9 (14,889) 97.5% ungrounded coverage β€” the turns a demo opens with
v10 v10 (15,069) pending multi-turn follow-ups, teacher-authored objection answers

The v3 β†’ v7 jump was not a model change. The corpus had been teaching noise: a Project called "Lifetime value forecast" required a gbdt classifier, so the only learnable lesson was that the Project name predicts nothing.

Read this before diagnosing it

A canary score does not predict demo behaviour. v7 scored 98% held-out and still failed three demo beats, because the canary carries retrieved context and a conversation's opening turns do not. Run both EvaluateBaseAgentOnCanary and scripts/demo-smoke.sh.

Known limitations

  • Ungrounded product answers blend. Trained against a capability listing; served without one it interpolates between memorized answers. Mitigated in v10, not verified.
  • Follow-ups need v10 or later. Earlier versions were trained with no conversation history at all, so every turn was a first turn.
  • The corpus is ~354 distinct asks presented many ways. A model that memorizes those will score well while having generalized less than the row count suggests.
  • RLVR is wired but does not produce these checkpoints. SFT does; goal-anchored GRPO moves a policy and has not yet been run at length.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for OramaSearch/base-agent-0.6b

Finetuned
Qwen/Qwen3-0.6B
Finetuned
(1,497)
this model