README.md

---
license: cc-by-4.0
library_name: safetensors
pipeline_tag: robotics
tags: [physical-ai, vision-language-action, vla, manipulation, imitation-learning, on-device, in-browser]
---

# physicalai-bmi/nano-vla-stack

A **multi-step manipulation** vision-language-action policy that runs fully in your
browser. Instruction: *"stack the red on the green."* The arm **grasps** the named
source block, **carries** it, and **places** it on the target block — a real
two-phase task, not a single reach. Run it live at <https://physicalai-bmi.org/research/vla>.

- **Vision:** a 32×32 render of a 2-link arm + a red and a green block. When a block is grasped it is drawn **at the gripper**, so the phase (reaching vs carrying) is visible to the policy.
- **Language:** the source block is parsed from the instruction (first colour mentioned = source; open-vocab colour grounding) → a 2-way source one-hot. This is deliberate: mean-pooled sentence embeddings are **order-blind** ("red on green" vs "green on red" classifies at ~0.67), so we parse the order and let the policy condition on the resolved source/target.
- **Action:** two joint deltas. Grasp/place are automatic on proximity; the policy drives the arm.
- **Params:** 55,666.

## Results (held-out, closed-loop, full episode)

| metric | value |
|---|---|
| **Stack success** (grasp → carry → place the correct block) | **94.0%** |
| Grasp rate | 94.2% |
| Success with the **source instruction flipped** | 22.8% |

The flipped row shows the language matters: tell it to move the *other* block and it
does, so success measured against the original target drops sharply. Trained by
behavior cloning on a Jacobian-IK two-phase expert (100% ceiling) with DART
state-noise injection to survive closed-loop drift across both phases.

Files: `model.safetensors`, `vla.web.json` (float32 for in-browser; forward verified
7.6e-8 vs safetensors), `metrics.json`. CC-BY-4.0, Institute for Physical AI @ BMI.
See also nano-vla-arm-3d (perspective reach) and nano-vla-reach.