NeoteAI commited on
Commit
1aa4302
·
verified ·
1 Parent(s): 1fc8005

Upload folder using huggingface_hub

Browse files
README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-sa-4.0
3
+ library_name: n0vtla
4
+ tags:
5
+ - robotics
6
+ - vision-language-action
7
+ - tactile
8
+ - manipulation
9
+ ---
10
+
11
+ # N0-VTLA - NeoSim Plug USB
12
+
13
+ Task policy for [N0-VTLA](https://github.com/neoteai/N0-VTLA), a vision-tactile-language-action
14
+ model that conditions a flow-matching action expert on predicted latent tactile tokens.
15
+
16
+ This is a **task policy, not a pretrained base**. For post-training on your own robot start from
17
+ [n0-vtla-base](https://huggingface.co/NeoteAI/n0-vtla-base).
18
+
19
+ | | |
20
+ |---|---|
21
+ | Config | `sim_single_arm_tactile` |
22
+ | Tactile pathway | enabled, `n_latent=5`, views `(tactile_a, tactile_b)` |
23
+ | Action space | 8-dim joint |
24
+
25
+ Policy for the NeoSim `insert_USB` task.
26
+
27
+ **16/20 (80%)**, first attempt.
28
+
29
+ ### Serving
30
+
31
+ ```bash
32
+ VTLA_ASSET_ID=univtac_single4_joint_norm \
33
+ python scripts/serve_zmq.py --config sim_single_arm_tactile \
34
+ --ckpt <this-dir> --addr "tcp://127.0.0.1:5557"
35
+ ```
36
+
37
+ `action_horizon` is 50; set `exec_horizon: 50`. Prompt: `insert USB`
38
+
39
+ ## Evaluation protocol
40
+
41
+ Seeds 100-119, 20 episodes, one per seed, on the UniVTAC simulator at commit `695a22d`
42
+ (branch `NeoSim` of [anlorla/UniVTAC](https://github.com/anlorla/UniVTAC)). Success criteria on
43
+ three tasks were tightened after these numbers were measured; see
44
+ [docs/EVAL.md](https://github.com/neoteai/N0-VTLA/blob/main/docs/EVAL.md) for the table and for
45
+ why a success rate is meaningless without the simulator commit beside it.
46
+
47
+ A single 20-episode run on this benchmark carries roughly +/-10 points of noise: repeating an
48
+ identical evaluation on a fixed checkpoint has produced 60 / 30 / 50 on the same task.
49
+
50
+ ## Caveat on the tactile pathway
51
+
52
+ This checkpoint carries the tactile pathway, but this benchmark cannot demonstrate that touch
53
+ contributes to the score. Object randomisation is +/-2-5 mm with no domain randomisation, so a
54
+ policy that ignores its cameras and its tactile sensors entirely can still score well. Use
55
+ `scripts/probe_z_tactile_dependence.py` to measure the causal contribution yourself.
56
+
57
+ ## License
58
+
59
+ CC BY-SA 4.0, as the parent repository.
assets/univtac_single4_joint_norm/norm_stats.json ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "norm_stats": {
3
+ "state": {
4
+ "mean": [
5
+ 0.22270573675632477,
6
+ 0.42828187346458435,
7
+ 0.0011076130904257298,
8
+ -2.094771146774292,
9
+ -0.33571797609329224,
10
+ 2.1391403675079346,
11
+ 0.5538936853408813,
12
+ 0.02270720712840557
13
+ ],
14
+ "std": [
15
+ 0.3224555552005768,
16
+ 0.3646058440208435,
17
+ 0.16353751718997955,
18
+ 0.39161738753318787,
19
+ 0.49961134791374207,
20
+ 0.6222327947616577,
21
+ 0.6913986206054688,
22
+ 0.017062881961464882
23
+ ],
24
+ "q01": [
25
+ -0.20300757315754892,
26
+ -0.19426319919526577,
27
+ -0.5974669305801392,
28
+ -2.5377611310005186,
29
+ -1.3382649856567383,
30
+ 0.8994710179686546,
31
+ -3.0767715366363526,
32
+ 0.0
33
+ ],
34
+ "q99": [
35
+ 0.9631711997985839,
36
+ 1.300018255352974,
37
+ 0.3096655229568481,
38
+ -1.0530869839668273,
39
+ 0.0488847576141358,
40
+ 2.912400931251049,
41
+ 1.4333493103027344,
42
+ 0.03941735771298409
43
+ ]
44
+ },
45
+ "actions": {
46
+ "mean": [
47
+ 0.014849782921373844,
48
+ 0.025956103578209877,
49
+ -0.002484560012817383,
50
+ 0.015539271757006645,
51
+ -0.029026245698332787,
52
+ -0.018883224576711655,
53
+ -0.07346583157777786,
54
+ 0.022505538538098335
55
+ ],
56
+ "std": [
57
+ 0.08398710936307907,
58
+ 0.15642578899860382,
59
+ 0.05134541541337967,
60
+ 0.13844804465770721,
61
+ 0.09812822937965393,
62
+ 0.14633677899837494,
63
+ 0.4166102111339569,
64
+ 0.016955165192484856
65
+ ],
66
+ "q01": [
67
+ -0.18911727628707886,
68
+ -0.37882136385440823,
69
+ -0.19682232965826985,
70
+ -0.45124487299919125,
71
+ -0.5478497080028057,
72
+ -0.6691574392557145,
73
+ -2.3575491218566893,
74
+ 0.0
75
+ ],
76
+ "q99": [
77
+ 0.32643533232212074,
78
+ 0.506714321088791,
79
+ 0.17401121686100962,
80
+ 0.44344423766136165,
81
+ 0.055100237756967596,
82
+ 0.31127041721343995,
83
+ 0.4629107832908632,
84
+ 0.03940946635007859
85
+ ]
86
+ }
87
+ }
88
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b13488f221e25abda0893ff16934cbebed2c47e84706c97b04ce54144fcc779f
3
+ size 8253873072