AMP Tadpole v0.2.0 (preview)

NOTICE

@@ -7,7 +7,7 @@ config.json and model.yaml are licensed under the Creative Commons
 Attribution-NonCommercial 4.0 International licence (see LICENSE).
 Commercial licences are available from Adaptive Machine Patterns Ltd:
 amp@adaptive-machine-patterns.com,
-https://adaptive-machine-patterns.com/amp-embodied-ai-workbench
+https://adaptive-machine-patterns.com/embodied-ai-workbench
 
 instructions.json lists token ids produced by the tokenizer of
 SmolVLM2-500M-Video-Instruct (HuggingFaceTB), which is licensed under the

README.md

@@ -20,11 +20,11 @@ and runs on a laptop CPU (and, through ONNX, in a web browser).
 ![A scene from the AMP Toy Robotics Benchmark](scene.png)
 
 AMP Tadpole is trained from scratch on the
-[AMP Toy Robotics Benchmark](https://adaptive-machine-patterns.com), and is
+[AMP Toy Robotics Benchmark](https://github.com/Adaptive-Machine-Patterns/amp-toy-robotics), and is
 the first model of [Adaptive Machine Patterns](https://adaptive-machine-patterns.com).
 
-> **v0.1.0 is a preview.** Training is ongoing; later versions will be
-> published under new tags (see [Versions](#versions)).
+> **v0.2.0 is a preview.** Later versions will be published under new
+> tags (see [Versions](#versions)).
 
 ## What it does
 
@@ -45,13 +45,12 @@ Instructions: `pick up the X`, `move to the X`,
 
 ## Results
 
-On the benchmark's standard 300 held-out scenes (never seen in training,
-and not used to choose the checkpoint), with the model driving the arm
-itself:
+On the benchmark's standard 300 held-out scenes (never seen in training),
+with the model driving the arm itself:
 
 | clauses | pick | move | midpoint | open | close | home | all | path vs teacher |
 |---|---|---|---|---|---|---|---|---|
-| 898 | 83% | 90% | 63% | 100% | 100% | 99% | **88.2%** | 1.02 |
+| 898 | 93% | 94% | 75% | 100% | 100% | 100% | **93.7%** | 1.03 |
 
 The reference teacher, a scripted controller that knows where everything
 is, scores 99.4% on the same scenes.
@@ -60,20 +59,28 @@ Grounding tests (100 further held-out scenes; see the benchmark's README):
 
 | named object | other objects | image swap | instruction swap | blank instruction |
 |---|---|---|---|---|
-| 0.96 | 0.31 | 1.00 | 1.00 | 0.92 |
+| 0.98 | 0.31 | 1.00 | 1.00 | 0.95 |
 
 Told "move to the X", its motion points the way the teacher's does
-(0.96, where 1 is the same direction), and not toward the other objects
+(0.98, where 1 is the same direction), and not toward the other objects
 in the scene (0.31). When two objects swap places, or the instruction
 names another object, its motion changes as much as the teacher's does
 (1.00). So it uses both the cameras and the words.
 
-**These numbers are a line in the sand, not a ceiling.** v0.1.0 was
-stopped part-way through training, and further training is expected to
-improve them. They are published as a reference point for small models
-on the benchmark: something to beat.
+**How the checkpoint was chosen.** The training run checked itself every
+2,500 steps on 100 held-out scenes, and those turned out to be the first
+100 of the 300 above (seeds 9,010,000 to 9,010,099); v0.1.0 was picked
+that way. For v0.2.0 the two final candidates (steps 300,000 and 330,000)
+were compared on 300 separate validation scenes (seeds from 9,100,000),
+where they tied at 94.2%; by a rule set beforehand, the tie went to the
+longer-trained one. On the 200 benchmark scenes that no choice ever looked
+at (608 clauses), v0.2.0 scores **93.1%**.
 
-These come from `amp-bench eval --policy amp-tadpole` on a CPU (about 45
+**These numbers are a line in the sand, not a ceiling.** They are
+published as a reference point for small models on the benchmark:
+something to beat.
+
+These come from `amp-bench eval --policy amp-tadpole` on a CPU (about 40
 minutes). Other hardware can differ in the last digits of the arithmetic,
 so a clause or two may come out differently.
 
@@ -82,7 +89,7 @@ so a clause or two may come out differently.
 With the benchmark (`pip install "amp-toy-robotics[packages]"`):
 
 ```bash
-hf download Adaptive-Machine-Patterns/amp-tadpole --revision v0.1.0 --local-dir amp-tadpole
+hf download Adaptive-Machine-Patterns/amp-tadpole --revision v0.2.0 --local-dir amp-tadpole
 amp-bench eval --policy amp-tadpole
 ```
 
@@ -129,7 +136,7 @@ teacher whose action is a fixed function of the state.
 - **Inputs only:** the images, the state and the text, under the
   benchmark's rules. No object positions, no progress reward, no path
   penalty.
-- **Recipe:** 250,000 steps, batch 64, AdamW (peak learning rate 3e-4, 1,000-step warm-up, cosine decay; the run was extended and resumed several times), the
+- **Recipe:** 330,000 steps, batch 64, AdamW (peak learning rate 3e-4, 1,000-step warm-up, cosine decay; the run was extended and resumed several times), the
   state hidden from half the samples (so the model must use the cameras),
   and an auxiliary loss in which each image token redraws its patch of
   the image. The redraw head is used only in training and is not included.
@@ -142,16 +149,16 @@ teacher whose action is a fixed function of the state.
   colours and 5 shapes. It has not seen a real camera.
 - It reads only canonical clauses (`instructions.json`); the benchmark
   rewrites free text into canonical form first.
-- The midpoint instruction is its weakest (see Results).
-- v0.1.0 is a preview, stopped before the end of its planned schedule.
+- The midpoint instruction is its weakest (75%; see Results).
 
 ## Versions
 
 | version | tag | what changed |
 |---|---|---|
-| 0.1.0 | `v0.1.0` | first preview: step 250,000 |
+| 0.2.0 | `v0.2.0` | step 330,000, the end of the training run: 93.7% (from 88.2%), midpoint 75% (from 63%); checkpoint chosen on separate validation scenes |
+| 0.1.0 | `v0.1.0` | first preview: step 250,000. Its card said the 300 scenes were not used to choose the checkpoint; the training run's own checks used 100 of them (see Results) |
 
-Pin a version with `revision="v0.1.0"` (or `--revision v0.1.0`).
+Pin a version with `revision="v0.2.0"` (or `--revision v0.2.0`).
 
 ## Licence
 
@@ -159,7 +166,7 @@ Pin a version with `revision="v0.1.0"` (or `--revision v0.1.0`).
 [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) (see
 LICENSE and NOTICE). **For commercial use**, licences are available from
 Adaptive Machine Patterns Ltd:
-[adaptive-machine-patterns.com/amp-embodied-ai-workbench](https://adaptive-machine-patterns.com/amp-embodied-ai-workbench)
+[adaptive-machine-patterns.com/embodied-ai-workbench](https://adaptive-machine-patterns.com/embodied-ai-workbench)
 or amp@adaptive-machine-patterns.com.
 
 **No warranty.** AMP Tadpole is a research preview, provided "as is",
@@ -184,7 +191,7 @@ a wrong movement could cause harm or damage.
   author    = {Whyte, Andy and {Adaptive Machine Patterns Ltd}},
   title     = {{AMP Tadpole}: a small vision-language-action model},
   year      = {2026},
-  version   = {0.1.0},
+  version   = {0.2.0},
   publisher = {Hugging Face},
   url       = {https://huggingface.co/Adaptive-Machine-Patterns/amp-tadpole}
 }

amp-tadpole.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:04af48ae3dfe4081933f690d2e4a89038bb33c4f62da5a76a57a1dc433117aff
+oid sha256:51c4521cd359b27b1759269610b53f722d43284b6a824dac4a315beeed219ab4
 size 28215128

config.json

@@ -1,6 +1,6 @@
 {
   "model": "amp-tadpole",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "parameters": 6507335,
   "inputs": {
     "images": "uint8 (3, 3, 512, 512): top, side, iso",
@@ -9,13 +9,13 @@
   },
   "output": "(8, 7): the next 8 joint states + gripper",
   "training": {
-    "steps": 250000,
+    "steps": 330000,
     "batch_size": 64,
     "learning_rate": 0.0003,
     "state_dropout": 0.5,
     "seeds": "from 0 (scenes from 9,000,000 up never used)",
     "progress_reward": 0.0,
     "path_weight": 0.0,
-    "success_on_100_held_out_scenes_during_training": 0.8629
+    "success_on_100_held_out_scenes_during_training": 0.893
   }
 }

model.yaml

@@ -1,7 +1,7 @@
 # AMP Tadpole: a model package for the AMP Toy Robotics Benchmark
 # (and the AMP Embodied AI Workbench).  Format: the benchmark's README.
 name: AMP Tadpole
-version: 0.1.0
+version: 0.2.0
 author: Andy Whyte, Adaptive Machine Patterns Ltd
 license: CC-BY-NC-4.0 (commercial licences from Adaptive Machine Patterns Ltd)
 homepage: https://huggingface.co/Adaptive-Machine-Patterns/amp-tadpole

weights.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:febca96f678fcb83292d07e243f6a8d392ceed74c854a9ea4477e8093bab5e0e
+oid sha256:738f7ae7c0152d4dae8be48f131b2cb7d2594700ba60965d8877ca67ceadd730
 size 26039652