Robotics
LeRobot
robomimic
diffusion-policy
imitation-learning
variable-admittance
force-control
doosan
Instructions to use aleleanza/diffusion-policy-vac-fridge with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use aleleanza/diffusion-policy-vac-fridge with LeRobot:
- Notebooks
- Google Colab
- Kaggle
Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: robomimic
|
| 4 |
+
pipeline_tag: robotics
|
| 5 |
+
tags:
|
| 6 |
+
- robotics
|
| 7 |
+
- diffusion-policy
|
| 8 |
+
- imitation-learning
|
| 9 |
+
- lerobot
|
| 10 |
+
- variable-admittance
|
| 11 |
+
- force-control
|
| 12 |
+
- doosan
|
| 13 |
+
datasets:
|
| 14 |
+
- aleleanza/vac-fridge-single-cam
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# Diffusion Policy — VAC Fridge (input/output ablation)
|
| 18 |
+
|
| 19 |
+
Eight **robomimic Diffusion Policy** checkpoints for the **variable-impedance pick**
|
| 20 |
+
("fridge") task on a Doosan M0609 + Inspire RH56 hand under a **Variable Admittance
|
| 21 |
+
Controller (VAC)**. Trained on
|
| 22 |
+
[`aleleanza/vac-fridge-single-cam`](https://huggingface.co/datasets/aleleanza/vac-fridge-single-cam)
|
| 23 |
+
(200 episodes, 54,833 frames @ 20 Hz, single RGB camera).
|
| 24 |
+
|
| 25 |
+
This is the fridge-task counterpart of
|
| 26 |
+
[`aleleanza/diffusion-policy-vac-pipe`](https://huggingface.co/aleleanza/diffusion-policy-vac-pipe):
|
| 27 |
+
an ablation over two *output* representations (relative to the admittance filter) ×
|
| 28 |
+
input modality, plus a larger-UNet variant per side.
|
| 29 |
+
|
| 30 |
+
## Variants (subfolders)
|
| 31 |
+
|
| 32 |
+
| Subfolder | Output (action) | Inputs (obs) | Action dim | Best val loss | @epoch |
|
| 33 |
+
|---|---|---|---:|---:|---:|
|
| 34 |
+
| `vac_preimg` | **pre**: `user_cmd[6]` + `K` + `ζ` + `hand_binary` | vision | 9 | 0.11807 | 240 |
|
| 35 |
+
| `vac_preimg_state` | pre | vision + state | 9 | 0.09940 | 240 |
|
| 36 |
+
| `vac_preimg_state_wrench` | pre | vision + state + wrench | 9 | 0.10013 | 597 |
|
| 37 |
+
| `vac_preimg_state_wrench_big` | pre (larger UNet) | vision + state + wrench | 9 | 0.10529 | 275 |
|
| 38 |
+
| `vac_postimg` | **post**: `vel_cmd[6]` + `hand_binary` | vision | 7 | 0.10254 | 240 |
|
| 39 |
+
| `vac_postimg_state` | post | vision + state | 7 | 0.09851 | 193 |
|
| 40 |
+
| `vac_postimg_state_wrench` | post | vision + state + wrench | 7 | **0.09467** | 193 |
|
| 41 |
+
| `vac_postimg_state_wrench_big` | post (larger UNet) | vision + state + wrench | 7 | 0.10136 | 263 |
|
| 42 |
+
|
| 43 |
+
- **pre** = predict the operator command *before* admittance, including the compliance
|
| 44 |
+
command itself (`stiffness_cmd` K + damping ζ) → the policy *learns to set compliance*.
|
| 45 |
+
- **post** = predict the Cartesian velocity executed *after* admittance → bypasses the
|
| 46 |
+
admittance filter and drives velocity directly.
|
| 47 |
+
|
| 48 |
+
Each subfolder contains: `best.pth`, `last.pth`, `config.json`, `action_stats.json`
|
| 49 |
+
(action min-max normalization + components + hand binarization), `dataset_summary.json`
|
| 50 |
+
(train/valid split).
|
| 51 |
+
|
| 52 |
+
## Architecture & training
|
| 53 |
+
|
| 54 |
+
- **Algorithm:** robomimic Diffusion Policy (DDPM noise-prediction loss).
|
| 55 |
+
- **Backbone:** conditional UNet `[128, 256, 512]`; the `_big` variants use
|
| 56 |
+
`[256, 512, 1024]` (~91.8M params vs ~89.4M).
|
| 57 |
+
- **Horizons:** observation 2, action 4, prediction 8 (frame stack 2, seq length 8).
|
| 58 |
+
- **Image:** single camera → `front_rgb`, **84×84**.
|
| 59 |
+
- **Diffusion:** DDPM 50 train/infer steps (DDIM 10-step configurable for faster inference).
|
| 60 |
+
- **Schedule:** up to 600 epochs, batch 16, lr 1e-4.
|
| 61 |
+
- **`state`** is built at runtime from current TCP + Inspire hand joints; **`wrench`** from
|
| 62 |
+
`/bota_ft_sensor/wrench`. Hand head is **binarized** (`hand_open_binary`).
|
| 63 |
+
|
| 64 |
+
### Reading the results
|
| 65 |
+
|
| 66 |
+
Validation losses are tightly clustered (~0.095–0.118). Adding **state** helps both pre
|
| 67 |
+
and post; the best overall is **`vac_postimg_state_wrench`** (0.0947). The larger-UNet
|
| 68 |
+
(`_big`) variants did **not** improve validation at this dataset size. Post (vel_cmd)
|
| 69 |
+
targets are marginally easier than pre (user_cmd + K + ζ), consistent with the pipe
|
| 70 |
+
ablation though the gap here is much smaller.
|
| 71 |
+
|
| 72 |
+
## Inference (ROS 2)
|
| 73 |
+
|
| 74 |
+
Run with the project's `robot_learning` real-time inference nodes (Doosan M0609 + Inspire
|
| 75 |
+
hand). The action contract determines the runner:
|
| 76 |
+
|
| 77 |
+
- **`pre*` variants (9D)** → `diffusion_policy_vac_preimg_runner`: publishes
|
| 78 |
+
`action[:6]→/delta_pose_cmd`, `[6]→/predicted_K`, `[7]→/predicted_zeta`,
|
| 79 |
+
`[8]→/inspire_hand/left/cmd`; consumed by `variable_admittance_node` (`variable_K:=true`).
|
| 80 |
+
- **`post*` variants (7D)** → `diffusion_policy_fixed_k_runner` in `vel_cmd` mode: streams
|
| 81 |
+
velocity directly via the DSR `speedl` interface (no admittance node).
|
| 82 |
+
|
| 83 |
+
Pre-wired launchers exist under `robot_learning/scripts/` (e.g.
|
| 84 |
+
`launch_vac_inference.sh preimg_state_wrench`); `*_wrench*` variants additionally need the
|
| 85 |
+
Bota driver. `action_stats.json` provides the exact normalization to undo at inference.
|
| 86 |
+
|
| 87 |
+
## Intended use & limitations
|
| 88 |
+
|
| 89 |
+
- **Use:** research on force-aware / compliance-predicting imitation learning; VAC baselines.
|
| 90 |
+
- **Limitations:** single task, single embodiment (M0609 + RH56), single 84×84 camera,
|
| 91 |
+
binary hand. Validation differences between variants are small. Not validated for
|
| 92 |
+
safety-critical or autonomous deployment.
|
| 93 |
+
|
| 94 |
+
## Related
|
| 95 |
+
|
| 96 |
+
- **Dataset:** [`aleleanza/vac-fridge-single-cam`](https://huggingface.co/datasets/aleleanza/vac-fridge-single-cam)
|
| 97 |
+
- **Pipe counterpart:** [`aleleanza/diffusion-policy-vac-pipe`](https://huggingface.co/aleleanza/diffusion-policy-vac-pipe)
|
| 98 |
+
- **Collection:** *VAC — Fridge*
|