---
license: apache-2.0
base_model: lerobot/smolvla_base
tags:
- robotics
- lerobot
- smolvla
- so100
- so101
- vla
- imitation-learning
pipeline_tag: robotics
---

# SmolVLA fine-tuned on SO-100 tic-tac-toe (pick-and-place)

Two full fine-tunes of [`lerobot/smolvla_base`](https://huggingface.co/lerobot/smolvla_base) (450M)
on a LeRobot **SO-100 "tic-tac-toe"** dataset — 180 episodes, 9 pick-and-place tasks
("place the blue cross cube in the `<cell>` box"), top + wrist cameras (640×480 @30 fps), 6-DOF
joint actions. Seed-fixed split: 144 train / 18 val / 18 test.

Camera mapping used in training: `top → observation.images.camera1`, `wrist → observation.images.camera2`.

### A note on the third camera

The underlying dataset has **only two cameras** (`top`, `wrist`). The model config, however, lists
**three** image inputs (`camera1`, `camera2`, `camera3`) — this 3-view layout is inherited from the
`lerobot/smolvla_base` pretrained model, **not** from the data:

| config input | source | content |
|---|---|---|
| `observation.images.camera1` | dataset `top` view | real |
| `observation.images.camera2` | dataset `wrist` view | real |
| `observation.images.camera3` | — | **padded empty/black image** (no information) |

`camera3` is a placeholder kept only to match the base model's expected input shape; SmolVLA tolerates
the missing view by feeding it a blank image. (Note: the saved `config.json` shows `empty_cameras: 0`
because the slot comes straight from the base model's `input_features` rather than being added as an
explicit empty camera — behaviour is unchanged either way.)

**For inference you only need to provide the two real views** (`camera1`=top, `camera2`=wrist),
`observation.state`, and the task string — the third slot is padded automatically.

## Checkpoints

| folder | training data | eval | major-4 joint corr |
|---|---|---|---|
| `in_distribution/`  | all 9 cells, 20k steps (loss 0.009) | all 9 cells (in-distribution fit) | **0.983** (per-joint all >0.96, MAE ~1.8°) |
| `heldout_cells/`    | 6 cells, 15k steps (best ckpt) | 3 **held-out** cells (top-left / center / middle-right) | **0.928** (MAE 3.2°) |

The held-out-cell run demonstrates that a pretrained VLA **generalizes the pick-place skill across
board positions** (0.928 on unseen cells vs 0.983 in-distribution) — far above a bespoke Cosmos3
action policy's 0.715 on *seen* cells. Generalization climbs with training (no overfit collapse).

## Usage

Each folder is a standard LeRobot SmolVLA `pretrained_model/` (model + pre/post-processors). Load with
LeRobot 0.4.4+:

```python
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy
policy = SmolVLAPolicy.from_pretrained("azhicles/smolvla-so100-tictactoe", subfolder="in_distribution")
```

Provide observations as `observation.images.camera1` (top), `observation.images.camera2` (wrist),
`observation.state` (6-DOF), and the task language string; the policy returns a chunked 6-DOF action.

## Inference speed

~4.70 s / episode (16 chunked forwards @ 0.284 s) on 1× NVIDIA GB300 — faster *and* more accurate
than the bespoke Cosmos3 action policy baseline.