xlerobot-coffee-model-real-b-force

Part of the Robotic-Barista-XLerobot collection — 3 dataset repos + 4 model repos for autonomous coffee service on the XLeRobot platform.

Method B — force-added. Force-aware ACT experts (architecture A1: HPI token, multiscale dense window, 1D-CNN encoder) for the Coffee Automata chain, trained on real-world demonstrations.

One of four model repos on the domain × method grid:

	A — vision + position	B — force-added
real	…-model-real-a-vision-pos	this repo
sim	…-model-sim-a-vision-pos	…-model-sim-b-force

Contents

One folder per task; each is a complete pretrained_model directory.

t1_place_cup/                     final (100k-step) checkpoint — config, safetensors, processors
t1_place_cup/checkpoints/020000/  … through 080000/, for the overfit-vs-step sweep

task	status	training data	steps
t1_place_cup	✅ trained	xlerobot-coffee-so101-legacy/b1-common/t1_place_cup — 49 ep / 16,992 fr	100k (+ 20k–80k sweep)
t2_push_button	⬜ not trained
t3_cup_to_tray	⬜ not trained
t4_navigate	⬜ not trained
t5_tray_to_table	⬜ not trained

Platform

t1_place_cup was trained on the legacy 6-DoF SO-101 data (observation.state[117], action[6]), not on the 17-DoF XLeRobot platform. It will not run on XLeRobot without retraining.

Training on 17-DoF platform data takes one extra step

xlerobot-coffee-real stores raw signals only — per-motor current_raw / load_raw / vel_hw, base odometry and IMU — and deliberately no observation.hpi. Method B therefore needs an offline estimation pass over the recordings before training.

That is the point, not an obstacle: the estimator is an experimental axis in its own right (precision across methods, and with vs. without IMU acceleration compensation, which decides how much of a mobile-base torque residual is real contact). Keeping the dataset raw means every estimator variant trains from identical inputs and no variant needs its own dataset upload.

Consequence for this repo: a checkpoint here is only interpretable next to the estimator that produced its inputs. Each task folder's card section records the estimator configuration used, and runs that differ only in estimator variant are named <task>__<estimator> so the axis stays legible on the repo page.

Architecture

ACT + Module A, variant A1 — the winning configuration from the force-aware study:

hpi_enabled  hpi_window_mode=multiscale  hpi_encoder=cnn1d  hpi_gate=none

observation.state = 6 joint positions; observation.hpi = 9-D EKF signal (hpi/gripper/{tau_ext, q, dq} + 6-D TCP wrench) in a dedicated token, read through a multiscale dense window and a 1D-CNN encoder. No contact gate. Cameras arm + head (RGB).

Requires the Coffee_Automata / hpi_act branch code to instantiate — the HPI token is not in stock LeRobot.

Recipe: ACT · chunk_size 100 · n_action_steps 100 · batch 8 · 100k steps · seed 1000 · RTX 3080 Ti — identical to Method A, so the A/B difference is the force channel and nothing else.

⚠ 100k steps over 49 episodes ≈ 47 epochs — likely overfit. Sweep checkpoints/{020000…080000}/ at evaluation rather than assuming the final checkpoint is best.

Loading

PreTrainedPolicy.from_pretrained reads model.safetensors from a repo root and has no subfolder support, so load through the project helper:

from scripts.coffee.load_coffee_policy import load_coffee_policy

policy = load_coffee_policy(domain="real", method="b", task="t1")             # final checkpoint
policy = load_coffee_policy(domain="real", method="b", task="t1", step=60000) # sweep point

It downloads only that task's subtree and calls ACTPolicy.from_pretrained(<local dir>), which does work on a directory. For lerobot-eval / lerobot-rollout, pass --policy.path=<that local dir>.

Provenance

Reorganized 2026-08-03. t1_place_cup/ is the former standalone repo IntelligentDecisionLab/xlerobot-coffee-act-t1-a1-hpi-real (run xl_t1_B, Coffee_Automata branch), moved unchanged.

Part of the X-Lerobot Coffee Automata project — force-family ACT experts for a coffee-service task. Pair with the -a-vision-pos repo for the Method A vs Method B comparison. AS-CITI Intelligent Decision Lab.