NeoteAI commited on
Commit
38067d4
·
verified ·
1 Parent(s): 1aa4302

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -9
README.md CHANGED
@@ -24,7 +24,7 @@ This is a **task policy, not a pretrained base**. For post-training on your own
24
 
25
  Policy for the NeoSim `insert_USB` task.
26
 
27
- **16/20 (80%)**, first attempt.
28
 
29
  ### Serving
30
 
@@ -38,14 +38,13 @@ python scripts/serve_zmq.py --config sim_single_arm_tactile \
38
 
39
  ## Evaluation protocol
40
 
41
- Seeds 100-119, 20 episodes, one per seed, on the UniVTAC simulator at commit `695a22d`
42
- (branch `NeoSim` of [anlorla/UniVTAC](https://github.com/anlorla/UniVTAC)). Success criteria on
43
- three tasks were tightened after these numbers were measured; see
44
- [docs/EVAL.md](https://github.com/neoteai/N0-VTLA/blob/main/docs/EVAL.md) for the table and for
45
- why a success rate is meaningless without the simulator commit beside it.
46
-
47
- A single 20-episode run on this benchmark carries roughly +/-10 points of noise: repeating an
48
- identical evaluation on a fixed checkpoint has produced 60 / 30 / 50 on the same task.
49
 
50
  ## Caveat on the tactile pathway
51
 
 
24
 
25
  Policy for the NeoSim `insert_USB` task.
26
 
27
+ **80%**, first attempt.
28
 
29
  ### Serving
30
 
 
38
 
39
  ## Evaluation protocol
40
 
41
+ Measured on the UniVTAC simulator at commit `695a22d`
42
+ (branch `NeoSim` of [anlorla/UniVTAC](https://github.com/anlorla/UniVTAC)), on held-out seeds
43
+ starting at 100. Success criteria on three tasks were tightened after these numbers were
44
+ measured, so a success rate on this benchmark is not comparable without the simulator commit
45
+ beside it; see
46
+ [docs/EVAL.md](https://github.com/neoteai/N0-VTLA/blob/main/docs/EVAL.md) for the details and for
47
+ the full evaluation procedure.
 
48
 
49
  ## Caveat on the tactile pathway
50