README.md

---
license: apache-2.0
library_name: lerobot
pipeline_tag: robotics
tags:
- lerobot
- act
- robotics
- so-arm101
- coffee-automata
- simulation
- force-aware
---

# xlerobot-coffee-model-sim-b-force

> Part of the **[Robotic-Barista-XLerobot](https://huggingface.co/collections/IntelligentDecisionLab/robotic-barista-xlerobot-6a7035e6a9c710bbbdc7f4d1)** collection — 3 dataset repos + 4 model repos for autonomous coffee service on the XLeRobot platform.

**Method B — force-added, trained in simulation.** Force-aware ACT experts (architecture **A1**:
HPI token, multiscale dense window, 1D-CNN encoder) for the Coffee Automata chain.

**No models trained yet.** This repo holds the structure so that sim results land in a predictable
place rather than in a new repo each time.

One of four model repos on the domain × method grid:

| | A — vision + position | B — force-added |
|---|---|---|
| **real** | [`…-model-real-a-vision-pos`](https://huggingface.co/IntelligentDecisionLab/xlerobot-coffee-model-real-a-vision-pos) | [`…-model-real-b-force`](https://huggingface.co/IntelligentDecisionLab/xlerobot-coffee-model-real-b-force) |
| **sim** | [`…-model-sim-a-vision-pos`](https://huggingface.co/IntelligentDecisionLab/xlerobot-coffee-model-sim-a-vision-pos) | **this repo** |

## Planned contents

One folder per task, each a complete `pretrained_model` directory, with a
`<task>/checkpoints/<NNNNNN>/` step sweep alongside.

| task | sim training data available | status |
|---|---|---|
| `t1_place_cup` | ✅ `xlerobot-coffee-sim/sim-robosuite-so101/t1_place_cup` — 50 ep / 10,231 fr | ⬜ not trained |
| `t2_push_button` | ❌ none recorded | ⬜ blocked |
| `t3_cup_to_tray` | ✅ `…/t3_cup_to_tray` — 50 ep / 19,485 fr | ⬜ not trained |
| `t4_navigate` | ❌ none recorded | ⬜ blocked |
| `t5_tray_to_table` | ✅ `…/t5_tray_to_table` — 50 ep / 17,567 fr | ⬜ not trained |

The sim data **does** carry `observation.hpi[9]`, so unlike the real platform data it is
immediately trainable for Method B.

## The reason to train here at all

Simulation is the only place the force signal has ground truth. Alongside the estimated
`observation.hpi`, the sim datasets carry `observation.sim_contact_force[24]` and
`observation.sim_tcp_wrench[12]` — the true external torques and TCP wrench. That makes it possible
to separate two questions the real data confounds:

- does the *force channel* help the policy, or
- does the *estimator's error* limit how much it can help?

Train Method B on estimated HPI and again on ground-truth wrench; the gap is the estimator's cost.

## ⚠ Sim is not the platform

The available sim data is a single **6-DoF SO-101** arm in robosuite (`robot_type: robosuite_sim`,
`action[6]`, `observation.state[117]`), not the 17-DoF XLeRobot. Models trained here are comparable
to the **legacy** real models, not to anything trained on `xlerobot-coffee-real`. A genuine
sim→real transfer result needs the 17-DoF sim recollection reserved at
`xlerobot-coffee-sim/sim-xlerobot/`.

Camera keys also differ across domains — sim uses `observation.images.top`, real uses `…head`. Any
cross-domain policy needs an explicit key remap.

## Architecture

ACT + **Module A, variant A1**:

```
hpi_enabled  hpi_window_mode=multiscale  hpi_encoder=cnn1d  hpi_gate=none
```

`observation.state` = 6 joint positions; `observation.hpi` = 9-D signal
(`hpi/gripper/{tau_ext, q, dq}` + 6-D TCP wrench) in a dedicated token, read through a multiscale
dense window and a 1D-CNN encoder. No contact gate.

**Requires the `Coffee_Automata` / `hpi_act` branch code to instantiate.** Recipe will mirror the
real-domain runs (ACT · `chunk_size` 100 · `n_action_steps` 100 · batch 8 · seed 1000) so the
domain is the only variable.

## Loading

Once populated, load through the project helper — `PreTrainedPolicy.from_pretrained` has no
`subfolder` support:

```python
from scripts.coffee.load_coffee_policy import load_coffee_policy
policy = load_coffee_policy(domain="sim", method="b", task="t1")
```

Part of the **X-Lerobot Coffee Automata** project. AS-CITI Intelligent Decision Lab.