xlerobot-coffee-model-sim-a-vision-pos

Part of the Robotic-Barista-XLerobot collection — 3 dataset repos + 4 model repos for autonomous coffee service on the XLeRobot platform.

Method A — vision + position, trained in simulation. Plain ACT experts for the Coffee Automata chain. No force channel: the control arm of the force-family experiment, on the sim side.

No models trained yet. This repo holds the structure so that sim results land in a predictable place rather than in a new repo each time.

One of four model repos on the domain × method grid:

	A — vision + position	B — force-added
real	…-model-real-a-vision-pos	…-model-real-b-force
sim	this repo	…-model-sim-b-force

Planned contents

One folder per task, each a complete pretrained_model directory, with a <task>/checkpoints/<NNNNNN>/ step sweep alongside.

task	sim training data available	status
t1_place_cup	✅ xlerobot-coffee-sim/sim-robosuite-so101/t1_place_cup — 50 ep / 10,231 fr	⬜ not trained
t2_push_button	❌ none recorded	⬜ blocked
t3_cup_to_tray	✅ …/t3_cup_to_tray — 50 ep / 19,485 fr	⬜ not trained
t4_navigate	❌ none recorded	⬜ blocked
t5_tray_to_table	✅ …/t5_tray_to_table — 50 ep / 17,567 fr	⬜ not trained

⚠ Sim is not the platform

The available sim data is a single 6-DoF SO-101 arm in robosuite (robot_type: robosuite_sim, action[6], observation.state[117]), not the 17-DoF XLeRobot. Models trained here are comparable to the legacy real models, not to anything trained on xlerobot-coffee-real. A genuine sim→real transfer result needs the 17-DoF sim recollection reserved at xlerobot-coffee-sim/sim-xlerobot/.

Camera keys also differ across domains — sim uses observation.images.top, real uses …head. Any cross-domain policy needs an explicit key remap.

Architecture

Vanilla ACT. observation.state = 6 joint positions + RGB cameras. No HPI token, no force input. Loads on stock LeRobot. Recipe will mirror the real-domain runs (ACT · chunk_size 100 · n_action_steps 100 · batch 8 · seed 1000) so the domain is the only variable.

Loading

Once populated, load through the project helper — PreTrainedPolicy.from_pretrained has no subfolder support:

from scripts.coffee.load_coffee_policy import load_coffee_policy
policy = load_coffee_policy(domain="sim", method="a", task="t1")

Part of the X-Lerobot Coffee Automata project. AS-CITI Intelligent Decision Lab.