Upload folder using huggingface_hub
Browse files- README.md +59 -0
- assets/univtac_single4_joint_norm/norm_stats.json +88 -0
- model.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-sa-4.0
|
| 3 |
+
library_name: n0vtla
|
| 4 |
+
tags:
|
| 5 |
+
- robotics
|
| 6 |
+
- vision-language-action
|
| 7 |
+
- tactile
|
| 8 |
+
- manipulation
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# N0-VTLA - NeoSim Plug USB
|
| 12 |
+
|
| 13 |
+
Task policy for [N0-VTLA](https://github.com/neoteai/N0-VTLA), a vision-tactile-language-action
|
| 14 |
+
model that conditions a flow-matching action expert on predicted latent tactile tokens.
|
| 15 |
+
|
| 16 |
+
This is a **task policy, not a pretrained base**. For post-training on your own robot start from
|
| 17 |
+
[n0-vtla-base](https://huggingface.co/NeoteAI/n0-vtla-base).
|
| 18 |
+
|
| 19 |
+
| | |
|
| 20 |
+
|---|---|
|
| 21 |
+
| Config | `sim_single_arm_tactile` |
|
| 22 |
+
| Tactile pathway | enabled, `n_latent=5`, views `(tactile_a, tactile_b)` |
|
| 23 |
+
| Action space | 8-dim joint |
|
| 24 |
+
|
| 25 |
+
Policy for the NeoSim `insert_USB` task.
|
| 26 |
+
|
| 27 |
+
**16/20 (80%)**, first attempt.
|
| 28 |
+
|
| 29 |
+
### Serving
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
VTLA_ASSET_ID=univtac_single4_joint_norm \
|
| 33 |
+
python scripts/serve_zmq.py --config sim_single_arm_tactile \
|
| 34 |
+
--ckpt <this-dir> --addr "tcp://127.0.0.1:5557"
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
`action_horizon` is 50; set `exec_horizon: 50`. Prompt: `insert USB`
|
| 38 |
+
|
| 39 |
+
## Evaluation protocol
|
| 40 |
+
|
| 41 |
+
Seeds 100-119, 20 episodes, one per seed, on the UniVTAC simulator at commit `695a22d`
|
| 42 |
+
(branch `NeoSim` of [anlorla/UniVTAC](https://github.com/anlorla/UniVTAC)). Success criteria on
|
| 43 |
+
three tasks were tightened after these numbers were measured; see
|
| 44 |
+
[docs/EVAL.md](https://github.com/neoteai/N0-VTLA/blob/main/docs/EVAL.md) for the table and for
|
| 45 |
+
why a success rate is meaningless without the simulator commit beside it.
|
| 46 |
+
|
| 47 |
+
A single 20-episode run on this benchmark carries roughly +/-10 points of noise: repeating an
|
| 48 |
+
identical evaluation on a fixed checkpoint has produced 60 / 30 / 50 on the same task.
|
| 49 |
+
|
| 50 |
+
## Caveat on the tactile pathway
|
| 51 |
+
|
| 52 |
+
This checkpoint carries the tactile pathway, but this benchmark cannot demonstrate that touch
|
| 53 |
+
contributes to the score. Object randomisation is +/-2-5 mm with no domain randomisation, so a
|
| 54 |
+
policy that ignores its cameras and its tactile sensors entirely can still score well. Use
|
| 55 |
+
`scripts/probe_z_tactile_dependence.py` to measure the causal contribution yourself.
|
| 56 |
+
|
| 57 |
+
## License
|
| 58 |
+
|
| 59 |
+
CC BY-SA 4.0, as the parent repository.
|
assets/univtac_single4_joint_norm/norm_stats.json
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"norm_stats": {
|
| 3 |
+
"state": {
|
| 4 |
+
"mean": [
|
| 5 |
+
0.22270573675632477,
|
| 6 |
+
0.42828187346458435,
|
| 7 |
+
0.0011076130904257298,
|
| 8 |
+
-2.094771146774292,
|
| 9 |
+
-0.33571797609329224,
|
| 10 |
+
2.1391403675079346,
|
| 11 |
+
0.5538936853408813,
|
| 12 |
+
0.02270720712840557
|
| 13 |
+
],
|
| 14 |
+
"std": [
|
| 15 |
+
0.3224555552005768,
|
| 16 |
+
0.3646058440208435,
|
| 17 |
+
0.16353751718997955,
|
| 18 |
+
0.39161738753318787,
|
| 19 |
+
0.49961134791374207,
|
| 20 |
+
0.6222327947616577,
|
| 21 |
+
0.6913986206054688,
|
| 22 |
+
0.017062881961464882
|
| 23 |
+
],
|
| 24 |
+
"q01": [
|
| 25 |
+
-0.20300757315754892,
|
| 26 |
+
-0.19426319919526577,
|
| 27 |
+
-0.5974669305801392,
|
| 28 |
+
-2.5377611310005186,
|
| 29 |
+
-1.3382649856567383,
|
| 30 |
+
0.8994710179686546,
|
| 31 |
+
-3.0767715366363526,
|
| 32 |
+
0.0
|
| 33 |
+
],
|
| 34 |
+
"q99": [
|
| 35 |
+
0.9631711997985839,
|
| 36 |
+
1.300018255352974,
|
| 37 |
+
0.3096655229568481,
|
| 38 |
+
-1.0530869839668273,
|
| 39 |
+
0.0488847576141358,
|
| 40 |
+
2.912400931251049,
|
| 41 |
+
1.4333493103027344,
|
| 42 |
+
0.03941735771298409
|
| 43 |
+
]
|
| 44 |
+
},
|
| 45 |
+
"actions": {
|
| 46 |
+
"mean": [
|
| 47 |
+
0.014849782921373844,
|
| 48 |
+
0.025956103578209877,
|
| 49 |
+
-0.002484560012817383,
|
| 50 |
+
0.015539271757006645,
|
| 51 |
+
-0.029026245698332787,
|
| 52 |
+
-0.018883224576711655,
|
| 53 |
+
-0.07346583157777786,
|
| 54 |
+
0.022505538538098335
|
| 55 |
+
],
|
| 56 |
+
"std": [
|
| 57 |
+
0.08398710936307907,
|
| 58 |
+
0.15642578899860382,
|
| 59 |
+
0.05134541541337967,
|
| 60 |
+
0.13844804465770721,
|
| 61 |
+
0.09812822937965393,
|
| 62 |
+
0.14633677899837494,
|
| 63 |
+
0.4166102111339569,
|
| 64 |
+
0.016955165192484856
|
| 65 |
+
],
|
| 66 |
+
"q01": [
|
| 67 |
+
-0.18911727628707886,
|
| 68 |
+
-0.37882136385440823,
|
| 69 |
+
-0.19682232965826985,
|
| 70 |
+
-0.45124487299919125,
|
| 71 |
+
-0.5478497080028057,
|
| 72 |
+
-0.6691574392557145,
|
| 73 |
+
-2.3575491218566893,
|
| 74 |
+
0.0
|
| 75 |
+
],
|
| 76 |
+
"q99": [
|
| 77 |
+
0.32643533232212074,
|
| 78 |
+
0.506714321088791,
|
| 79 |
+
0.17401121686100962,
|
| 80 |
+
0.44344423766136165,
|
| 81 |
+
0.055100237756967596,
|
| 82 |
+
0.31127041721343995,
|
| 83 |
+
0.4629107832908632,
|
| 84 |
+
0.03940946635007859
|
| 85 |
+
]
|
| 86 |
+
}
|
| 87 |
+
}
|
| 88 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b13488f221e25abda0893ff16934cbebed2c47e84706c97b04ce54144fcc779f
|
| 3 |
+
size 8253873072
|