license: apache-2.0
library_name: lerobot
pipeline_tag: robotics
tags:
- lerobot
- act
- robotics
- so-arm101
- coffee-automata
- simulation
- force-aware
xlerobot-coffee-model-sim-b-force
Part of the Robotic-Barista-XLerobot collection — 3 dataset repos + 4 model repos for autonomous coffee service on the XLeRobot platform.
Method B — force-added, trained in simulation. Force-aware ACT experts (architecture A1: HPI token, multiscale dense window, 1D-CNN encoder) for the Coffee Automata chain.
No models trained yet. This repo holds the structure so that sim results land in a predictable place rather than in a new repo each time.
One of four model repos on the domain × method grid:
| A — vision + position | B — force-added | |
|---|---|---|
| real | …-model-real-a-vision-pos |
…-model-real-b-force |
| sim | …-model-sim-a-vision-pos |
this repo |
Planned contents
One folder per task, each a complete pretrained_model directory, with a
<task>/checkpoints/<NNNNNN>/ step sweep alongside.
| task | sim training data available | status |
|---|---|---|
t1_place_cup |
✅ xlerobot-coffee-sim/sim-robosuite-so101/t1_place_cup — 50 ep / 10,231 fr |
⬜ not trained |
t2_push_button |
❌ none recorded | ⬜ blocked |
t3_cup_to_tray |
✅ …/t3_cup_to_tray — 50 ep / 19,485 fr |
⬜ not trained |
t4_navigate |
❌ none recorded | ⬜ blocked |
t5_tray_to_table |
✅ …/t5_tray_to_table — 50 ep / 17,567 fr |
⬜ not trained |
The sim data does carry observation.hpi[9], so unlike the real platform data it is
immediately trainable for Method B.
The reason to train here at all
Simulation is the only place the force signal has ground truth. Alongside the estimated
observation.hpi, the sim datasets carry observation.sim_contact_force[24] and
observation.sim_tcp_wrench[12] — the true external torques and TCP wrench. That makes it possible
to separate two questions the real data confounds:
- does the force channel help the policy, or
- does the estimator's error limit how much it can help?
Train Method B on estimated HPI and again on ground-truth wrench; the gap is the estimator's cost.
⚠ Sim is not the platform
The available sim data is a single 6-DoF SO-101 arm in robosuite (robot_type: robosuite_sim,
action[6], observation.state[117]), not the 17-DoF XLeRobot. Models trained here are comparable
to the legacy real models, not to anything trained on xlerobot-coffee-real. A genuine
sim→real transfer result needs the 17-DoF sim recollection reserved at
xlerobot-coffee-sim/sim-xlerobot/.
Camera keys also differ across domains — sim uses observation.images.top, real uses …head. Any
cross-domain policy needs an explicit key remap.
Architecture
ACT + Module A, variant A1:
hpi_enabled hpi_window_mode=multiscale hpi_encoder=cnn1d hpi_gate=none
observation.state = 6 joint positions; observation.hpi = 9-D signal
(hpi/gripper/{tau_ext, q, dq} + 6-D TCP wrench) in a dedicated token, read through a multiscale
dense window and a 1D-CNN encoder. No contact gate.
Requires the Coffee_Automata / hpi_act branch code to instantiate. Recipe will mirror the
real-domain runs (ACT · chunk_size 100 · n_action_steps 100 · batch 8 · seed 1000) so the
domain is the only variable.
Loading
Once populated, load through the project helper — PreTrainedPolicy.from_pretrained has no
subfolder support:
from scripts.coffee.load_coffee_policy import load_coffee_policy
policy = load_coffee_policy(domain="sim", method="b", task="t1")
Part of the X-Lerobot Coffee Automata project. AS-CITI Intelligent Decision Lab.