README.md

---
license: apache-2.0
base_model:
  - lerobot/pi0_base
  - lerobot/pi05_base
tags:
  - robotics
  - vision-language-action
  - vla
  - pi0
  - pi05
  - vlash
  - rlx
  - gguf
pipeline_tag: robotics
---

# rlx-vlash weights (π₀ / π₀.₅)

RLX-native repackaging of the [VLASH](https://github.com/mit-han-lab/vlash)
**π₀** and **π₀.₅** Vision-Language-Action robot policies, ready to load with the
[`rlx-vlash`](https://crates.io/crates/rlx-vlash) crate. The tensors are the
published `lerobot/pi0_base` / `lerobot/pi05_base` checkpoints, remapped to
rlx-vlash's canonical key names and re-encoded into GGUF / RLX-package formats
(no fine-tuning — same weights, different container + naming).

Both policies pair a **PaliGemma** backbone (SigLIP-So400m/14 @224 vision tower +
Gemma-2B text) with a **Gemma-300M action expert** running through 18 *joint*
transformer layers that share one attention, and generate action chunks by
**flow matching** (a 10-step Euler integration of a learned velocity field).

- **π₀** — state is a suffix token, time is fused into the action embeddings,
  standard Gemma RMSNorm.
- **π₀.₅** — action-only suffix; time drives **adaptive RMSNorm** in the action
  expert (`state_cond=false` for the base checkpoint).

## Contents

Each variant ships in four precisions, in both GGUF and RLX-package containers:

```
pi0/    model.gguf  model-q8_0.gguf  model-q4_k.gguf  model-f32.gguf
        model.rlxp  model-q8_0.rlxp  model-q4_k.rlxp  model-f32.rlxp   config.json
pi05/   (same)
```

| file | format | precision | approx size |
|---|---|---|---|
| `model.gguf` / `model.rlxp` | GGUF / RLX pkg | **f16** (primary) | ≈ 6.4 GB |
| `model-q8_0.gguf` / `model-q8_0.rlxp` | GGUF / RLX pkg | q8_0 (8-bit) | ≈ 3.4 GB |
| `model-q4_k.gguf` / `model-q4_k.rlxp` | GGUF / RLX pkg | q4_K (4-bit) | ≈ 1.9 GB |
| `model-f32.gguf` / `model-f32.rlxp` | GGUF / RLX pkg | f32 (lossless) | ≈ 13 GB |

Pick whichever fits: **f16** is the recommended default (reproduces the action
trajectory exactly), **q8_0** / **q4_K** trade precision for size (norms/biases
stay f16 in the quantized variants), **f32** is bit-exact to source.

**Quantization quality (important).** Measured vs the original VLASH
implementation on `pi05_base`: per-step model outputs stay near-perfect at every
precision (image features & single-step velocity cosine ≥ 0.999 even at q4_K —
the quant/dequant is exact), but the **10-step flow-matching rollout amplifies**
small weight errors into the final action chunk:

| precision | image features | 1-step velocity | **50-action chunk** |
|---|---|---|---|
| f16  | 1.0000 | 1.0000 | **1.0000** |
| q8_0 | 0.99998 | 0.99999 | **0.973** |
| q4_K | 0.9976 | 0.99946 | **0.807** |

(cosine vs reference.) This is the known sensitivity of iterative flow/diffusion
samplers, not a quantization bug. **For faithful robot control use `f16`**; treat
`q8_0` / `q4_K` as size-first options where some action drift is acceptable.

GGUF stores dims in the standard
GGML (llama.cpp) order and loads through rlx-vlash or any GGUF-aware runtime; the
`.rlxp` is the RLX flat-mmap package. All keep rlx-vlash's canonical key names, so
no remap happens at load.

## Usage (rlx-vlash)

Put one variant's files in a directory (the runner prefers `model.gguf`), then:

```rust
use rlx_vlash::{VlashRunner, VlashVariant};
use rlx_runtime::Device;

let mut runner = VlashRunner::builder(VlashVariant::Pi05)
    .device(Device::Cpu)
    .num_images(1)
    .prompt_tokens(200)
    .model_dir("path/to/pi05")   // dir containing model.gguf (+ tokenizer.json)
    .build()?;

let actions = runner.predict_action_chunk(&[image_nchw], &state, "pick up the cube", None)?;
```

The language **tokenizer is not included** (PaliGemma's is gated): fetch
`google/paligemma-3b-pt-224`'s `tokenizer.json` and place it alongside the
weights, or supply token ids directly.

## Parity

rlx-vlash reproduces the original VLASH implementation on these checkpoints at
**cosine = 1.000000** across every stage — SigLIP vision features, the
image+text prefix, a single flow-matching denoise step (joint attention + adaRMS
/ state-token path), and the full 10-step action chunk — for **both** π₀ and
π₀.₅ (max |Δ| ≤ 1.1e-3, attributable to MPS-f32 reference vs CPU-f32 rlx).

## License & attribution

Apache-2.0, inherited from the upstream [VLASH](https://github.com/mit-han-lab/vlash)
project and the `lerobot/pi0_base` / `lerobot/pi05_base` base checkpoints. These
files are a format/naming repackaging of those weights; please cite VLASH and
LeRobot. The `rlx-vlash` inference crate itself is GPL-3.0-only.