amp-robotics commited on
Commit
addcae1
·
verified ·
1 Parent(s): 41e34ee

AMP Tadpole v0.2.0: card (limitations, link to the Demo)

Browse files
Files changed (1) hide show
  1. README.md +17 -1
README.md CHANGED
@@ -23,6 +23,10 @@ AMP Tadpole is trained from scratch on the
23
  [AMP Toy Robotics Benchmark](https://github.com/Adaptive-Machine-Patterns/amp-toy-robotics), and is
24
  the first model of [Adaptive Machine Patterns](https://adaptive-machine-patterns.com).
25
 
 
 
 
 
26
  > **v0.2.0 is a preview.** Later versions will be published under new
27
  > tags (see [Versions](#versions)).
28
 
@@ -149,7 +153,19 @@ teacher whose action is a fixed function of the state.
149
  colours and 5 shapes. It has not seen a real camera.
150
  - It reads only canonical clauses (`instructions.json`); the benchmark
151
  rewrites free text into canonical form first.
152
- - The midpoint instruction is its weakest (75%; see Results).
 
 
 
 
 
 
 
 
 
 
 
 
153
 
154
  ## Versions
155
 
 
23
  [AMP Toy Robotics Benchmark](https://github.com/Adaptive-Machine-Patterns/amp-toy-robotics), and is
24
  the first model of [Adaptive Machine Patterns](https://adaptive-machine-patterns.com).
25
 
26
+ **Try it in your browser:** the
27
+ [AMP Embodied AI Workbench Demo](https://huggingface.co/spaces/Adaptive-Machine-Patterns/amp-embodied-ai-workbench-demo) runs AMP
28
+ Tadpole live (through ONNX), and shows it inside, layer by layer.
29
+
30
  > **v0.2.0 is a preview.** Later versions will be published under new
31
  > tags (see [Versions](#versions)).
32
 
 
153
  colours and 5 shapes. It has not seen a real camera.
154
  - It reads only canonical clauses (`instructions.json`); the benchmark
155
  rewrites free text into canonical form first.
156
+ - The midpoint instruction is its weakest (75%; see Results). It is not
157
+ overfitting: on teacher-labelled samples its errors are the same on
158
+ scenes it trained on and scenes it never saw. Midpoint is simply less
159
+ well learned. Its target is an empty spot between two objects, so there
160
+ is nothing to aim at in the pictures, and it is 10% of the training
161
+ clauses; its error was still shrinking when training ended.
162
+ - **What these results do not show.** The teacher is deterministic and
163
+ unlimited and the cameras are fixed: that is why a 6.5M-parameter model
164
+ does well here. It is not evidence against larger VLAs on benchmarks of
165
+ human demonstrations, which are varied, finite and often use a wrist
166
+ camera.
167
+ - All results come from one training run; it has not been repeated with
168
+ other random seeds.
169
 
170
  ## Versions
171