UBR-Nav-E4B
We keep publishing the evals that break our own results. Here is the third one — it broke the benchmark itself.
A navigation fine-tune of Gemma 4 E4B (~4B open vision-language model). One camera frame from the robot's point of view, plus the goal as a compass bearing and range — what a real robot actually knows: no map, no global pose, no overhead view. Output: a short local path as ego waypoints (x, y, heading).
We run the base model on a real ground robot today — as its offline voice interface only, no cloud, Jetson Orin Nano class hardware. The navigation fine-tune is not deployed to it. Every result for the model is simulation — no sim2real claim. The newest chapter is different: it is what the robot platform itself has started to measure.
One closed-loop run. Left is the robot's camera — everything the model gets. Right is ground truth it never sees: the optimal route (orange), the path it actually drove (blue), the plan it is holding right now (teal), the goal (red). Watch it recover at corner after corner, tens of metres deep, to the goal's doorstep — and watch how far the blue path has to travel to close on a goal that is only metres away in a straight line. What it can't yet do from the camera alone is decide, at every junction, which corridor leads to the goal. That's the story below.
The story so far (two chapters, both honest)
Chapter 1 — it works, then it doesn't. On a maze where walls are taller than the camera and the goal is occluded in ~80% of frames, base Gemma 4 E4B collides on 99 of 100 held-out scenes — no better than a no-model "point at the goal and drive" baseline. Fine-tuned: 0 collisions. Then a held-out evaluation matrix scored the same adapter at 49% collisions, and a follow-up probe measured exactly what it had learned instead of navigating — a single memorized turn distance, emitted at every corridor width regardless of where the corner actually was. The headline did not survive its own eval.
Chapter 2 — we fixed what the probe found, and the model genuinely learned to read the scene: shown a corner at any distance, it now plans the turn at that distance (measured correlation ≈ 1.0, where 0 = ignores the corner). It also learned to recover from its own drift — off-route collision rate fell sharply, and the failure mode where a slightly mis-aimed robot drove straight into a wall it could see was largely eliminated.
By every single-frame metric, the model was now good.
Then we stopped scoring frames and started driving
Every result above is open-loop: show one frame, score one path. So we closed the loop — execute part of the plan, re-render from wherever that puts the robot, plan again, repeat — because that is what a robot actually does, and it is the first place small errors compound.
The model drives far now. It threads corner after corner, recovers from drift, survives where the earlier versions died at the first wall — runs of tens of metres through a maze. But over 20 held-out mazes it reaches the goal about one time in eight, and the diagnostic says why: on the median run, the closest it ever gets to the goal is ~55% of its starting distance. It is not near-missing at the line. It drives competently and wanders — one run covered 84 metres and never got past 70% of the way to a goal 12 metres away.
Why — and why it is not a bigger-model problem
The model is asked to reach a goal given as a straight-line bearing — which, in a maze, points through walls. At a junction where the goal bears left but the corridor that reaches it goes right first, nothing in a single camera frame distinguishes that from a maze where left is correct. The same observation maps to opposite correct answers depending on global structure the model cannot see. Those junction decisions are not under-trained; from this observation they are not learnable. It is a partial-observability limit, not a capacity limit.
Which is the actual finding, and it is worth more than another point of accuracy:
A single-frame vision-language planner is an excellent local controller — it reads free space, rounds corners, avoids walls, recovers from drift. It is the wrong tool for global routing, which needs the map. Asking one model to do both is asking it to solve a routing problem it cannot observe.
So the fix is architectural, not a larger network: keep the model as the local controller it is demonstrably good at, and hand it reachable subgoals from a separate global layer that owns the map. That is how real robots are built. So we built it.
We built the layer. Here is the number.
We closed the loop again, twice — each move earned by the failure the previous one exposed.
| The model plans from | Reaches the goal |
|---|---|
| a straight-line bearing to the final goal (through walls) | 12% |
| a reachable subgoal from a deterministic layer that owns the map — same model, no retraining | 40% |
| the same subgoal, after the model is trained to act on it | 85% |
First move. A deterministic layer that owns the map hands the model a reachable subgoal instead of a bearing that points through a wall. Nothing about the model changes — no retraining, no new weights — and goal-reach triples. The local controller was right all along; it was being asked a question a single frame cannot answer.
Second move. The model had never been trained to expect a reachable subgoal — it learned on a straight-line goal cue and had half-learned to ignore it. So we trained it to the interface. Goal-reach doubles again, to 85%, and the median run now converges on the goal instead of wandering.
The failures that remain are no longer wandering. They are the occasional fumble from a hard starting pose — a local-control slip, not a routing failure. The pathology the closed loop first exposed is gone.
That is the whole thesis, measured end to end: a ~4B vision-language model is a superb local controller and a hopeless global router. Split the two — deterministic map and routing below, learned perception-grounded control above — and it works.
The map was the last excuse. We took it away.
Every number above hands the global layer a complete map up front. A robot dropped somewhere new does not have one — it has to build the map as it drives, from the same single camera, choosing where to go while the map is still mostly holes. That was the honest gap between this and the field, and last chapter we promised to publish the number whatever it turned out to be.
Here it is — the same learned controller, no prior map — discovering the world as it moves and routing over only what it has seen so far:
| The robot starts with | Reaches the goal |
|---|---|
| a complete map, handed to it | 85% |
| no map at all — it builds one while driving | 92% |
Taking the map away cost nothing. (Read these as the same number, not an improvement — they are different held-out sets, 85% over 20 mazes and 92% over 12, and we will not mine the noise for a win.) The price of exploring instead of being told is paid in distance driven — the robot ranges wider and doubles back before it converges — not in whether it arrives. The layered split holds up even when the map has to be earned, which is the only version of the map a field robot ever gets.
The same maze as the hero at the top — but here the robot starts with no map at all. In the right panel, grey is the unknown it has not seen yet, and it lifts as the robot drives: dark cells are walls it has discovered, bright cells are free space it has confirmed, blue is the path it has actually taken. The amber line is the optimal route, which it never sees. It threads to the goal while drawing the map underneath itself — reaching it just as the goal marker finally comes into view. This is the 92% row, animated.
The camera can't do everything, and we measured exactly what it can't
The maze was corridors. The world is also open ground with a thing standing in it — a pole, a crate, a parked vehicle — that isn't on any map. So we asked the sharp version of the question: when the map-owning layer is blind to an obstacle, can the camera alone save the robot?
We put the model in exactly that bind — scenes where the global layer cannot see a free-standing obstacle, and only the camera frame reveals it — and scored it against a deliberately dumb control: a policy that ignores the image entirely and just follows the plan.
They tie. The vision model drives at the same rate into the same obstacles. It reads the frame, and it still doesn't go around the thing it can see. And when we threw training specifically at that failure — collecting the exact "obstacle looming" moments and teaching the swerve, round after round — the number didn't move. More data was not the lever. A single frame, at range, cannot localize a thin obstacle precisely enough to plan around it. It's a limit of the eye, not the dataset.
Which is the same lesson as before, one layer down:
Don't ask the camera to do a job a sensor does better. The fix for "there's a thing in the way" is not a bigger network or more data — it is letting the layer that owns the map actually see the obstacle.
Give the routing layer a cheap ranging sensor, and the obstacle it could never dodge simply gets routed around — in simulation, that recovers almost all of the ground the blind system lost, back near the numbers it posts with a map. We proved that in sim before spending a euro on hardware, and it's why the next thing bolted to the robot is a ranger, not a bigger model.
That is the whole discipline this card keeps arriving at: know which layer owns which job. Camera for local control. Map, routing, and sensing for everything global. Ask either one to be the other and you get a confident drive into a wall.
The ranger is on the robot now
The last chapter ended with a promise: the next thing bolted to the robot would be a ranging sensor, not a bigger model. It is bolted on — a spinning 2D lidar and an ultrasonic ranger, on the real chassis, wired into the real safety loop. Reality started paying out immediately — in its own currency, not yet the simulation's: pose, safety stops, calibration.
The pose problem fell to the ranger too. The old edge list ended with "a chassis that only ever estimates where it is" — this robot has no wheel encoders at all. The lidar closed that gap: re-align two successive scans and you get the motion that actually happened, measured rather than assumed. The robot now drives autonomous mapping missions in which every single step is verified against the world — three recent missions matched all 120 of their 120 steps, a fourth matched 119 — and two independent rotation estimators agreed on hardware to a tenth of a degree.
They did not always agree. The first time we cross-checked them they disagreed in sign, and the trail ended at the lidar reporting its scan mirrored — while the turn verifier had been "working" all along because of a compensating sign error of its own. Two wrongs, cancelling, invisibly, for weeks. Only instruments checked against each other caught it. We publish that for the same reason we publish broken benchmarks: a sensor you have not cross-checked is a sensor you are trusting on faith.
The "too low to see" failure mode got its first real catch. The ultrasonic halted the robot 23 cm short of an obstacle that the lidar plane, at the same instant, called clear at 1.2 m. An obstacle below one sensor's plane is another layer's job — and the mission map now records that catch as an obstacle the ranging layers found on their own.
And the sensor that navigates turned out to supervise as well. The robot calibrates its own actuator model, on-device, from its own driving — it learned, among other things, that the smallest turn its drivetrain can actually express is about ten degrees, a fact no bench test had surfaced. The estimator found it by refusing to believe the commands.
One number we are proud not to report: the mapping missions' completion criterion must prove the loop closed. In every real room so far it has refused, and shipped its map as partial. An autonomy claim you cannot fail honestly is not a claim.
The navigation fine-tune is still not deployed; every model number above remains simulation. What changed is that the platform the thesis prescribes — deterministic layers that own map, routing, sensing and pose, with a learned controller above — now physically exists and is earning numbers of its own.
The robot hit a stair leg. Every sensor missed it. Here is what that bought.
Between the last chapter and this one, the robot collided with a stair leg — an obstacle a few centimetres wide, low, off the sonar's axis, effectively invisible to the spinning ranger's coarse bearings. Our own earlier finding says the camera would have missed it too. Nobody's sensor suite saw it. So we stopped trying to see it and started noticing it: the scan-matcher cannot be fooled about whether the robot actually moved, and a watchdog built on that fact was written the afternoon of the collision. It fired for real the same week — catching an unseen obstacle mid-mission, marking it on the map, and continuing the mission. Detection by consequence, not by signature.
The moving-obstacle guard got its measurement. A person deliberately walked toward the robot during an autonomous run: the closing-rate sentinel stopped it at 448 mm — an approach rate no static world could produce — well before the firmware's last-line proximity reflex, which then marked the person's position as an obstacle when they stayed put, and the mission turned away and continued. Layer by layer, in the designed order, on the first live trial.
A crossing mover is a different geometry — low closing rate, barely grazing the forward watch — and when we tested it, the guard missed, exactly as the geometry predicts: the person walked across the robot's path in plain view of the ranging sensor and nothing was looking at that part of the sweep. So the recorded miss became a test fixture, a detector was built against it the same afternoon, and the repeat test was caught twice — at 580 mm and 628 mm — with the robot stopping and waiting for the person to pass before resuming its mission. A mover is not an obstacle: it never enters the map, and you do not re-route around someone who will be gone in two seconds. Miss, measure, fix, catch — one day, on the record.
And the platform now goes where it is told. The deterministic half of the layered split — map and routing below, with a scripted follower standing in for the learned controller, which remains undeployed — now runs end to end on hardware: the robot escapes a confined start, builds its map as it drives, and routes over it with the same routing layer that posted the 92% in sim. Three attempts failed, each fixing a real bug en route (one of them replanned mid-run around an obstacle the reflex had just discovered); the fourth arrived, in nine stop-and-go steps, 178 mm from a goal it had selected in its own free space 2.2 m away — and needed no reflex at all. Small distances, honestly counted — and the same refusals as ever when the map says no: a goal region that turns out to be occupied gets a refusal, not a squeeze.
One more number in the honest ledger: the controller that drove every mapping run turned out to be provably unstable — we finally derived why, instead of re-tuning around it for the third time, and the oscillation we kept observing matched the analysis exactly. The fix took a mission's unwanted rotation from thousands of degrees to −87°, and the stability proof is now a unit test: future tuning must re-prove it before it ships.
It reached a goal it had never seen, with no map of the room
The sim's last row — no map at all, build it while driving — now has a hardware counterpart for its deterministic half. Same layers, same scripted stand-in for the still-undeployed learned controller: pointed at a goal roughly five metres of driving away, starting with no map, the robot carved the space as it moved, routed over the half-drawn map, and arrived 284 mm out — 15 stop-and-go steps, no human hand on the run that arrived. This time the goal lay beyond anything the robot had yet seen, not a spot in free space it had already carved. The runs that got there spent recovery layer after recovery layer, in the designed order: mid-mission the robot climbed an obstacle it couldn't see, tipped to 53°, backed itself level, verified it was stable before trusting its own pose again, marked the climb on the map and replanned around it. A route the map, padded for safety, called sealed was in fact passable — and the robot found the gap and took it. A planner that started flip-flopping between two equal corridors was given a one-rule check: a full reversal must survive two consecutive plans before it is believed. None of that is the learned controller; all of it is the platform the thesis prescribes, holding up outside the maze the thesis was scored in.
The tilt event, from the robot's own camera at 2× speed: the horizon keels over as it climbs the obstacle it couldn't see, the firmware halts the tracks at 51°, and the reverse-pulse rescue walks it back down until the world is level again. Nobody touched it. (direct link)
The camera got its first honest job — and its first honest retraction
The earlier chapters demoted the camera twice — wrong tool for global routing, wrong tool for ranging thin obstacles. This week it got the job it is actually built for: find a thing you can describe. A yellow package was dropped somewhere in a cluttered room, out of sight of the robot, and the robot was asked to find it. The lens on this robot is measurably poor — too soft to resolve texture at any focus setting; at range, color is the only signal that survives the blur. First field run: a candidate sighted at about two metres as a dozen-pixel smudge, stalked through eight occlusion losses behind furniture legs to under half a metre, declared FOUND — and the operator looked at the frame and overruled it. Wrong object.
The robot's own eye, 3× speed: the closing minute of that first stalk, threading between chair legs to the thing it believed in. It was wrong, and publishing the drive anyway is the point — the competence and the mistake are the same footage. (direct link)
The failure taught more than the stalk did. At close range we had asked the on-board vision-language model — the base model, the same one that runs the voice interface; this card's navigation fine-tune remains undeployed and took no part — the confirming question "is a yellow package visible?", and it said yes to the wrong object. A model asked a leading question wants to agree. The arbiter is now non-leading: the model is asked to name the object in front of the camera, and dumb code checks the answer against the operator's description. Under the new rule a sighting is a hypothesis — the robot drives to it and investigates up close before any declaration, a false lead is announced and walked away from, and the re-run hunt did exactly that: swept, explored, checked motion cues from the ranging layer, and ended with "not found" rather than a second false positive. The package is still out there; we prefer that sentence to a wrong FOUND.
Epilogue, same day: found. The hunts that followed kept paying tuition — one declared FOUND on a real cardboard box that the operator overruled (right noun, wrong color: brown), so the confirming question learned to REQUIRE the color word, not just the noun. Then, with every lesson of the day active at once, the robot swept, investigated, rejected four more candidates by name — including, at one point, its own chassis — crossed the room, and stopped in front of this:
The model's close-range verdict: "Yellow rectangular bag." Required word present, noun matched, frame checked by a human. The sentence we published above — refusal over a wrong FOUND — held for exactly as long as it needed to: until the machinery earned the right one.
One measured acuity number survived both runs: asked about the package while it was still a 20-pixel smudge, the base model simply said "no" — and asked, on the same frame, about the person sitting across the room, it placed them correctly. A model's eye has an acuity floor the same way a ranging sensor has a plane it cannot see below. The finder is built around both limits rather than in denial of them.
Still honest about the edge
For the model, this is all still simulation. For the platform, the messy rest of reality remains: contact-level sensing for the glancing hits the watchdog can't feel (a bumper, honestly, is next), surfaces the lidar gets nothing back from (we met one — it ended four missions in a row before we gave the robot a turn-away reflex for what it cannot see), pose drift over long wandering detours that only map-level matching will bound, and rooms too open and cluttered to close a clean loop around. Each is its own failure mode and its own number, and we would rather measure them than assume them. We will publish those the same way we published these — including the ones that don't flatter us. That has been the whole point.
Why we report collision rate, not trajectory error
Under a corrupted frame this model scores 0.41 m average displacement error at a 77% "accept" rate — while 60% of those exact paths hit a wall. Displacement error measures how close a path looks to a reference; it does not measure whether the robot survives. Every headline here is a collision rate or a goal-reach rate. And a benchmark without a control tells you a model got worse, never why — ours told us the exact distance it had memorized, and then that it wanders. An eval you cannot fail honestly is not an eval.
What is released
Nothing — no weights, no dataset, no code. This card is the result, including the results that broke our own headlines. The method stays in-house.
UBR Physical AI · follow ubr-physical-ai for the next chapter. · Last updated: 4 August 2026