onf/docs/concepts.md

# Concepts

The idea, end to end, without the code. Read this before `docs/technical/`.

No measured number appears in this file. Every success rate this repo has produced lives in
[`results.md`](results.md), and nowhere else.

---

## 1. The problem

A vision-language-action policy is frozen. It was trained on demonstrations and it is not going to
be retrained, finetuned, LoRA'd or adapted. Perturb the world and it drifts: the rollout leaves the
demonstrated manifold — a missed grasp, an accumulated tracking error, a distractor that pulled the
arm off course.

That is a monitoring problem. Something has to keep watching and, at every action chunk, say where
on the demonstrated support the arm should be.

## 2. One model

Build a graph out of the training demonstrations. Nodes are short runs of consecutive frames; edges
connect them forward in time, sideways across demonstrations of the same task, and across tasks at
the same point in the episode.

Train one small network that takes a window of recent joint states and returns a distribution over
the graph's nodes: *given what the arm has been doing, which demonstrated moment does this look
like?*

That distribution is the whole method. It is filtered over time: each new observation updates a
belief over nodes, and the belief — not the raw observation — is what says where the arm should be.

Nothing about this is policy-specific. The graph and its head are trained once per suite from
demonstration data alone. The same artifacts drive a frozen StableVLA and a frozen GR00T-N1.7 with
no retraining of either.

## 3. Why a graph rather than a nearest neighbour

The obvious baseline is: store every demo frame, find the closest one to the current joint
configuration, go there.

That fails for a reason worth stating precisely. A manipulation trajectory revisits the same joint
configurations at different times — the arm passes through nearly the same pose reaching for an
object and retracting from it. Position alone does not identify *when* you are. A nearest neighbour
returns a confident answer to the wrong question.

The graph's edges encode what nearest-neighbour lookup throws away: what comes next. The training
task is built around that. The network is never asked "which node is this window closest to" — it is
asked "which node comes a few frames *after* this window", which cannot be answered by a distance
computation.

## 4. Why this needs a filter

The retrieval head is memoryless. Every call answers from scratch. On a suite where each task
involves two objects, reaching for the second is kinematically close to reaching for the first, and
a memoryless matcher aliases them — it will happily report that the arm has jumped backward to an
earlier phase, or teleported forward past work it has not done.

A belief filter fixes this cheaply. Carry a distribution over nodes across checks; push it forward
through a transition model built from the graph's own forward edges; multiply by the new
observation. A backward jump now has to pay transition cost, because the transition model has no
backward edges. Aliasing that the observation cannot resolve gets resolved by dynamics.

## 5. Deciding where the arm should be

The answer is a projection, not a point. The basin around a node is a certified region fitted from
held-out demonstrations. If the arm is already inside it, the correction is exactly a no-op. If not,
the target is the boundary of the region — undo the drift, do not overshoot into a pose the policy
has no reason to expect.

The reading is a short *segment*, not a single configuration. A point target restores position while
leaving velocity wrong, which hands the policy a state that is in-distribution in `q` and
out-of-distribution in `qdot` — the precise failure the correction exists to prevent.

The belief is read one check AHEAD: the target comes off the belief pushed through the transition
kernel, not off the belief itself, so it keeps the direction of travel rather than only the position.

## 6. Applying the reading

A reading is worth nothing until something acts on it, and there are two ways to act. Hand the
reading to a second controller that drives the arm towards it, or fold it into what the policy was
already going to do. This repo did the first and now does the second.

The first way is a servo. It works — it restores position — but it is a second authority over the
same arm, so everything about it is a handover problem: when to take control, when to give it back,
how hard to pull, and what happens to the half-finished motion the policy had in flight. Every one of
those questions needs its own mechanism, and every mechanism is another thing that can fire when it
should not. That whole stack has been deleted.

The second way is a blend. The reference segment is converted into the policy's own action units and
mixed into the action chunk the policy has just produced, at one point, once per chunk. No handover
exists to get wrong, because the policy never stops driving. What replaces the handover question is a
weight: how much of the mixed-in chunk to take.

That weight is the whole difficulty, and it is not a tuning constant. Steering towards a
demonstration helps when the demonstration is still the right one and the policy is merely mis-seeing
the scene; it actively hurts when the demonstration has expired — the object has moved, or the
instruction was reworded and retrieval went to another task entirely. Following a stale
demonstration confidently is worse than doing nothing. So the weight has to be a judgement about the
*pair*: this query, and this thing that was retrieved for it. It is learned, from data that includes
deliberately wrong retrievals, because clean data can only ever teach "the arm has drifted, pull
harder" and never "this demonstration no longer applies".

Two smaller consequences of blending rather than servoing are worth stating, because both were bugs
first. A correction issued in action space has no gain limit unless one is written down, so the
tracking chunk's first row — a raw pose error — is saturated explicitly. And the gripper command is
binary, so it is never mixed at all; the policy's own bit passes straight through.

## 7. What this method does not do

- It does not touch the policy. No gradients, no adapters, no prompt surgery. It changes the actions
  the policy emits, not the policy that emits them.
- It does not look at pixels. Everything runs on joint configurations and end-effector poses.
- It does not know what the objects are, where they are, or what the task means. Task identity
  enters only as a graph label selecting which demonstrations are relevant.
- It does not plan. There is no search, no model-predictive rollout, no cost function over futures.
  The only thing resembling a prediction is one step of the belief filter.

## 8. Where the honest gaps are

Stated here rather than buried, and expanded with numbers in [`results.md`](results.md):

- Two components that run on every single retrieval — the `ENTRY_STATIC` training class and the
  cleanliness weighting — have never been ablated.
- The learned weight has now been evaluated: +7.9 pp over seven axes against the frozen policy. But
  four axes carry all of it and three do not move, and the one axis this project started on —
  Robot_Initial_States — is still 33.6 pp behind the joint-space prefix that was deleted to make room
  for the blend. On that axis the method has recovered about a third of what deleting the servo cost.
- The learned weight's per-check *shape* is worth +4.2 pp over the best constant weight. Its
  *magnitude* had to be corrected by hand, because the training objective (chunk MSE) does not know
  that a missed intervention costs a whole episode. That correction is a calibration with a derived
  value, but it is still a term the loss should have contained and does not.
- No blend arm has been run on a suite other than `long`, or on a policy other than StableVLA.
- The gate that was supposed to decide *whether* to correct at all never fired in the one sweep that
  has been run. Every check corrected.

---

**Next:** [`technical/01-data-and-graph.md`](technical/01-data-and-graph.md) — demonstrations to
graph, with shapes and dtypes. Or [`GLOSSARY.md`](GLOSSARY.md) to look up a term.