README.md

metadata

license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
library_name: n0vtla
tags:
  - robotics
  - vision-language-action
  - tactile
  - simulation

𝒩₀-VTLA — NeoSim dual-arm

A 𝒩₀-VTLA dual-arm policy for the NeoSim simulator, post-trained from the 𝒩₀-VTLA pretrained base. This card documents the dual_bowl_place_stack setup.

Embodiment	dual-arm
Action space	joint, 16-dim (2 x (7 joints + gripper))
Action horizon	16
Tactile views	4
Latent tactile tokens	5

Files

model.safetensors                                     the full checkpoint (~8.25 GB)
assets/n0_dual_bowl_place_stack_norm/norm_stats.json
config.json                                           architecture summary

Serve

The evaluation adapter speaks ZMQ + msgpack, so use scripts/serve_zmq.py from the code repository:

hf download NeoteAI/n0_VTLA_dual_bowl_place_stack --local-dir checkpoints/n0_VTLA_dual_bowl_place_stack

VTLA_ASSET_ID=n0_dual_bowl_place_stack_norm python scripts/serve_zmq.py \
  --config sim_dual_arm_tactile \
  --ckpt checkpoints/n0_VTLA_dual_bowl_place_stack \
  --addr "tcp://*:5557" \
  --default-prompt "Use both arms to place and stack the bowls"

Serving reads only model.safetensors and assets/<asset-id>/norm_stats.json; no dataset is needed. A correct load prints tactile=True with 4 views and no missing or unexpected state-dict keys. If it reports either, the config does not match the checkpoint.

Then run the evaluation from a NeoSim checkout with a deploy YAML pointing at port 5557. Set exec_horizon to 16, the model's action horizon: executing fewer steps clips the tail of each chunk, which is where the grasp-closing motion lives.

Action space

Unlike the pretrained base, which predicts end-effector deltas in a canonical 32-dim rot6d container, this policy predicts joint actions: 7 joints plus gripper per arm. The joint dims are element-wise deltas against the current state; the gripper columns are absolute. Do not feed it end-effector data or reuse an end-effector normalization asset.

License

These weights are derived from Google's PaliGemma/Gemma parameters and are made available under the Gemma Terms of Use and the Gemma Prohibited Use Policy, not under the CC BY-SA 4.0 licence that covers the source code. This is inherited from the base model.