README.md

---
license: apache-2.0
base_model: lerobot/smolvla_base
datasets:
  - makepluscode/pick_red
library_name: lerobot
pipeline_tag: robotics
tags:
  - robotics
  - lerobot
  - smolvla
  - so101
  - pick-and-place
---

# pick_red_30k

SmolVLA fine-tuned from `lerobot/smolvla_base` on
[`makepluscode/pick_red`](https://huggingface.co/datasets/makepluscode/pick_red)
for the SO-ARM 101 red-block pick-and-place task. Trained through LeLab.

Companion checkpoint for Part 4 of the book *VLA 로보틱스 입문서*.

## Training

| | |
|---|---|
| Base checkpoint | `lerobot/smolvla_base` |
| Dataset | `makepluscode/pick_red` — 50 episodes, 27,346 frames |
| Steps | 30,000 |
| Batch size | 8 |
| Optimizer | AdamW, lr 1e-4, cosine decay with warmup |
| Start loss | 0.450 (step 250) |
| Final loss | 0.065 |
| Loss plateau | from ~25,000 steps |
| Hardware | RTX 3060 12GB, CUDA 12.6 |
| Wall time | ~3 h 35 min |
| LeRobot | 0.5.2 (`82dffde`) |

`train_config.json` records `policy.pretrained_path` pointing at the
`lerobot/smolvla_base` snapshot, and `load_vlm_weights: true` — the vision and
language weights come from the pretrained base rather than being randomly
initialised.

## Trained with LeLab — and what had to be patched

Training ran through [LeLab](https://github.com/huggingface/leLab)'s web UI, not
the LeRobot CLI. Out of the box that does **not** produce a fine-tune of
`smolvla_base`, so the vendored LeLab used here carries three changes.

**1. Pass a base checkpoint.** LeLab builds its training command with
`--policy.type` only. That is correct for ACT, which has no pretrained base, but
for SmolVLA it means LeRobot constructs the architecture with randomly
initialised vision and language weights and trains only the action expert.
Nothing looks wrong while it happens — the loss falls, checkpoints are written,
inference runs — and LeRobot only warns about it on the PEFT path. The mismatch
shows up solely as `policy.pretrained_path: null` in `train_config.json`.
The patch adds a per-policy default and emits `--policy.path` instead.

**2. Use `=` form for policy arguments.** LeRobot's parser reads
`--policy.path` only as `--policy.path=<value>` (`parse_arg` matches on the
`--<name>=` prefix). Once a policy path is present, `get_cli_overrides("policy")`
also strips every `--policy.*` argument, so a space-separated
`--policy.device cuda` leaves a bare `cuda` behind and argparse rejects it. All
`--policy.*` arguments are emitted as `--policy.x=y`.

**3. Resolve the base to a local path, and align camera keys.** Two more things
break before training starts:

- On Windows, LeRobot turns `policy_path` into a `Path`, so the hub id
  `lerobot/smolvla_base` becomes `lerobot\smolvla_base` and the hub lookup
  fails. LeLab now resolves the base with `snapshot_download` and passes the
  local snapshot directory.
- `smolvla_base` declares `camera1`/`camera2`/`camera3` as its image inputs,
  while this dataset records `top`/`wrist`, which fails feature validation.
  LeLab now reads the expected keys from the base `config.json`, reads the
  dataset keys from `meta/info.json`, and generates a `--rename_map`.

For this run that produced:

```
observation.images.top   ->  observation.images.camera1
observation.images.wrist ->  observation.images.camera2
```

Selecting **SmolVLA** in the LeLab training screen is enough; the base
checkpoint and the rename map are filled in by the backend. To train without a
pretrained base, pass an empty `policy_path`.

## Task string

The same sentence is stored in the dataset and must be supplied at inference:

```
빨간 블록을 집어서 노란색 매트에 놓는다.
```

## Evaluation

Evaluated on the real SO-ARM 101 with the block placed at a single fixed start
position — the same condition the demonstrations were collected under. Results
are in-distribution and should not be read as evidence of visual or positional
generalisation.