README.md

---
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
library_name: n0vtla
tags:
  - robotics
  - vision-language-action
  - tactile
  - pi0.5
---

# $\mathcal{N}_0$-VTLA — pretrained base

Pretrained base weights for [N0-VTLA](https://github.com/neoteai/N0-VTLA), a
vision–tactile–language–action policy: a π0.5-style VLA with a latent tactile pathway.

## Files

```
model.safetensors   the full checkpoint (~8.25 GB): π0.5 VLM, flow-matching action expert,
                    latent tactile predictor, and frozen DINOv2, in one state dict
config.json         architecture summary
```

## Load

```python
from huggingface_hub import snapshot_download
import safetensors.torch as st
from n0vtla.training.config import get_config
from n0vtla.models_pytorch.n0vtla_policy import N0VTLAPolicy

ckpt = snapshot_download("NeoteAI/n0-vtla-base")
cfg = get_config("vtla_tactile_posttrain")
model = N0VTLAPolicy(cfg.model).eval().cuda()
missing, unexpected = st.load_model(model, f"{ckpt}/model.safetensors", strict=False, device="cuda")
assert not unexpected, unexpected
```

`missing` is expected to contain exactly
`paligemma_with_expert.paligemma.model.language_model.embed_tokens.weight`: it is tied to
`lm_head.weight` and safetensors stores tied weights once. `unexpected` must be empty.

In practice you do not call this yourself — point `VTLA_PRETRAINED_CHECKPOINT` at the download
directory and use the post-training recipe:

```bash
hf download NeoteAI/n0-vtla-base --local-dir checkpoints/n0-vtla-base
export VTLA_PRETRAINED_CHECKPOINT="$PWD/checkpoints/n0-vtla-base"
```

See **Post-training on your own robot data** in the
[repository README](https://github.com/neoteai/N0-VTLA).

## Normalization statistics

**Not included, by design.** Normalization depends on your dataset, robot action layout, and
delta-action convention, so it has to be computed on your own data:

```bash
python scripts/compute_canonical_norm.py \
  --repo-id /path/to/your/canonical_dataset \
  --robot flexiv \
  --train-config-name vtla_tactile_posttrain \
  --asset-id your_asset_id
```

The result lands in `assets/vtla_tactile_posttrain/<asset-id>/norm_stats.json`, and
`VTLA_ASSET_ID` must match the `--asset-id` you used.

## Action format

The policy outputs normalized element-wise delta actions. To run them on a robot:

1. **De-normalize** in the same element-wise (subtraction) space the model was trained in.
   The quantile inverse is `x = (x_norm + 1) / 2 * (q99 - q01) + q01`, applied to the active
   dims only — 10 for single-arm, 20 for dual-arm.
2. **Add back to the current state**, `abs = state + delta`, on the delta dims. Grippers and
   padding are already absolute. This is the standard `AbsoluteActions` step; there is no
   relative-rotation or rotation-matrix math, because the rotation delta is element-wise.
3. **Execute the full chunk.** The action horizon is 50 — execute all 50 steps before
   requesting the next prediction. Executing fewer clips the tail of the chunk, which is where
   the grasp-closing motion lives, and the gripper will look stuck open.

### 32-dim action/state layout

```
left arm  [0:10]:  xyz [0:3],   rot6d [3:9],   gripper [9]
right arm [10:20]: xyz [10:13], rot6d [13:19], gripper [19]
[20:32]:           reserved / unused
```

Rotation is rot6d, not quaternion or Euler. On disk the `action` field is the absolute
end-effector pose; the training transforms form the delta target as `action[t+k] - state[t]`.

## License

These weights are derived from Google's PaliGemma/Gemma parameters and are therefore made
available under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and the
[Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy), **not** under
the CC BY-SA 4.0 licence that covers the source code. This is inherited from the base model, not
a restriction added on top.