xlerobot-coffee-model-real-a-vision-pos

Part of the Robotic-Barista-XLerobot collection — 3 dataset repos + 4 model repos for autonomous coffee service on the XLeRobot platform.

Method A — vision + position. Plain ACT experts for the Coffee Automata chain, trained on real-world demonstrations. No force channel: this is the control arm of the force-family experiment.

One of four model repos on the domain × method grid:

	A — vision + position	B — force-added
real	this repo	…-model-real-b-force
sim	…-model-sim-a-vision-pos	…-model-sim-b-force

Contents

One folder per task; each is a complete pretrained_model directory.

t1_place_cup/                     final (100k-step) checkpoint — config, safetensors, processors
t1_place_cup/checkpoints/020000/  … through 080000/, for the overfit-vs-step sweep

task	status	training data	steps
t1_place_cup	✅ trained	xlerobot-coffee-so101-legacy/b1-common/t1_place_cup — 49 ep / 16,992 fr	100k (+ 20k–80k sweep)
t2_push_button	⬜ not trained
t3_cup_to_tray	⬜ not trained
t4_navigate	⬜ not trained
t5_tray_to_table	⬜ not trained

⚠ Platform

t1_place_cup was trained on the legacy 6-DoF SO-101 data, not on the 17-DoF XLeRobot platform: observation.state is [117] and action is [6]. It will not run on the XLeRobot platform without retraining. Retraining on xlerobot-coffee-real is blocked only by data volume, not by architecture — Method A needs no force channel.

Architecture

Vanilla ACT. observation.state = 6 joint positions; cameras arm + head (RGB). No HPI token, no force input. Loads on stock LeRobot — no branch code required.

Recipe: ACT · chunk_size 100 · n_action_steps 100 · batch 8 · 100k steps · seed 1000 · RTX 3080 Ti.

⚠ 100k steps over 49 episodes ≈ 47 epochs — likely overfit. The intermediate checkpoints/{020000…080000}/ exist so evaluation can sweep the step axis rather than assume the final checkpoint is best.

Loading

PreTrainedPolicy.from_pretrained reads model.safetensors from a repo root and has no subfolder support, so load through the project helper:

from scripts.coffee.load_coffee_policy import load_coffee_policy

policy = load_coffee_policy(domain="real", method="a", task="t1")             # final checkpoint
policy = load_coffee_policy(domain="real", method="a", task="t1", step=60000) # sweep point

It downloads only that task's subtree and calls ACTPolicy.from_pretrained(<local dir>), which does work on a directory. For lerobot-eval / lerobot-rollout, pass --policy.path=<that local dir>.

Provenance

Reorganized 2026-08-03. t1_place_cup/ is the former standalone repo IntelligentDecisionLab/xlerobot-coffee-act-t1-vanilla-real (run xl_t1_A, Coffee_Automata branch), moved unchanged.

Part of the X-Lerobot Coffee Automata project — force-family ACT experts for a coffee-service task. Pair with the -b-force repo for the Method A vs Method B comparison. AS-CITI Intelligent Decision Lab.