Model card: eval 8/10, training details, run instructions

README.md

@@ -1,5 +1,7 @@
 ---
-datasets: lichtphyz/so101_block_to_bin_v3
+datasets:
+- lichtphyz/so101_block_to_bin_v3
+- lichtphyz/rollout_diffusion_v3_desktop
 library_name: lerobot
 license: apache-2.0
 model_name: diffusion
@@ -8,38 +10,25 @@ tags:
 - robotics
 - lerobot
 - diffusion
+- so101
 ---
 
-# Model Card for diffusion
+# Diffusion Policy: SO-101 block to bin
 
-<!-- Provide a quick summary of what the model is/does. -->
+A [Diffusion Policy](https://huggingface.co/papers/2303.04137) for a single **SO-101** arm that picks up a wooden block
+and drops it into a bin, using a front (workspace) camera and a wrist camera. Trained with
+[LeRobot](https://github.com/huggingface/lerobot) on 125 teleoperated episodes.
 
-
-[Diffusion Policy](https://huggingface.co/papers/2303.04137) treats visuomotor control as a generative diffusion process, producing smooth, multi-step action trajectories that excel at contact-rich manipulation.
-
-
-
-
-<!-- A short demo is worth more than any description! Record a GIF/video of the policy
-running on your robot, upload it to this repo, and embed it here:
-<p align="center">
-  <img src="https://huggingface.co/<hf_user>/<policy_repo_id>/resolve/main/demo.gif" width="60%"/>
-</p>
--->
-
-This policy has been trained and pushed to the Hub using [LeRobot](https://github.com/huggingface/lerobot).
-
-See the [full LeRobot documentation](https://huggingface.co/docs/lerobot/index).
-
-
----
+**Real-robot result: 8/10 successes** (DDIM, 5 denoising steps); see [Evaluation](#evaluation).
+All 10 evaluation episodes, including camera video, are public:
+[lichtphyz/rollout_diffusion_v3_desktop](https://huggingface.co/spaces/lerobot/visualize_dataset?path=lichtphyz/rollout_diffusion_v3_desktop).
 
 ## Model Details
 
 - **License:** apache-2.0
-- **Robot type:** `so_follower`
-- **Cameras:** `front`, `wrist`
-
+- **Robot:** SO-101 follower (`so_follower`), 6 joints (5 + gripper), STS3215 12V servos
+- **Cameras:** `front` (fixed workspace view) and `wrist`, both 640×480 @ 30 fps
+- **Task string:** "Put the block in the bin."
 
 ## Inputs & Outputs
 
@@ -74,14 +63,22 @@ The policy consumes these observation features and produces these action feature
 </a>
 
 
+The 125 training episodes are 100 block-to-bin demonstrations (the block grid-aligned, two block lengths,
+hover-then-descend approach), 15 "idle" episodes (empty table or block already in the bin: the arm stays at rest), and
+10 recovery episodes (missed grasp → regrasp → bin).
+
 ## Training Configuration
 
 | Setting | Value |
 | --- | --- |
-| Training steps | 60000 |
+| Training steps | 60,000 (≈11 h 20 min on one RTX 4080) |
 | Batch size | 32 |
-| Optimizer | adam |
-| Learning rate | 0.0001 |
+| Optimizer / LR | Adam, 1e-4, cosine schedule |
+| Vision backbone | ResNet-18 (ImageNet weights, BatchNorm) |
+| Image size | resized to 320×240, random crop 304×228 (crop ratio 0.95) |
+| Observation steps / horizon / action steps | 2 / 64 / 32 |
+| Training noise schedule | DDPM, 100 timesteps |
+| Image augmentation | brightness, contrast, saturation, hue, sharpness, small affine |
 | Seed | 1000 |
 | LeRobot version | 0.6.2 |
 
@@ -89,63 +86,45 @@ The policy consumes these observation features and produces these action feature
 
 ## How to Get Started with the Model
 
-New to LeRobot? These guides cover the full workflow:
-
-- **[Install LeRobot](https://huggingface.co/docs/lerobot/main/en/installation)** — set up the `lerobot` package.
-- **[Hardware setup](https://huggingface.co/docs/lerobot/main/en/hardware_guide)** — assemble, wire, and calibrate your robot and cameras.
-- **[Record data & train a policy](https://huggingface.co/docs/lerobot/en/il_robots)** — the end-to-end imitation-learning walkthrough.
-- **[CLI cheat-sheet](https://huggingface.co/docs/lerobot/main/en/cheat-sheet)** — quick reference for the `lerobot-*` commands.
-
-The short version to run and train this policy:
-
-### Run the policy on your robot
+Run it on an SO-101 with the settings used in the evaluation (DDIM with 5 denoising steps makes action planning fast
+enough to avoid visible pauses):
 
 ```bash
 lerobot-rollout \
   --strategy.type=base \
-  --robot.type=so_follower \
-  --robot.port=<your_robot_port> \
-  --robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
   --policy.path=lichtphyz/diffusion_v3_desktop \
+  --policy.noise_scheduler_type=DDIM \
+  --policy.num_inference_steps=5 \
+  --robot.type=so101_follower \
+  --robot.port=<your_robot_port> \
+  --robot.cameras="{ front: {type: opencv, index_or_path: <path>, width: 640, height: 480, fps: 30, fourcc: MJPG}, wrist: {type: opencv, index_or_path: <path>, width: 640, height: 480, fps: 30, fourcc: MJPG}}" \
   --task="Put the block in the bin." \
   --duration=60
 ```
 
-Replace the remaining `<...>` placeholders with your own values: `--robot.port` and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.
-
-When `--strategy.type=base` is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at [rollout documentation](https://huggingface.co/docs/lerobot/main/en/inference).
-
-### Train your own policy
-
-```bash
-lerobot-train \
-  --dataset.repo_id=${HF_USER}/<dataset> \
-  --policy.type=diffusion \
-  --output_dir=outputs/train/<policy_repo_id> \
-  --job_name=lerobot_training \
-  --policy.device=cuda \
-  --policy.repo_id=${HF_USER}/<policy_repo_id> \
-  --wandb.enable=true
-```
-
-_Writes checkpoints to `outputs/train/<policy_repo_id>/checkpoints/`._
+The camera names must be `front` and `wrist`. The policy will only work with a camera placement and workspace close to
+the training data; see the dataset visualizer links above.
 
 ---
 
 ## Evaluation
 
-<!-- Report real-robot results here: run the policy several times per task and count the
-successes. Delete the "No evaluation results" line and fill in this table instead:
+Real-robot evaluation with `lerobot-rollout --strategy.type=episodic`: 10 episodes of up to 30 s each, the block reset
+by hand between episodes, inference with DDIM and 5 denoising steps.
 
 | Task | Trials | Successes | Success rate |
 | ---- | ------ | --------- | ------------ |
-| pick the lego brick | 10 | 8 | 80% |
+| Put the block in the bin | 10 | 8 | 80% |
 
-Also worth noting: anything that affects difficulty (new object positions, lighting,
-distractors, a different robot of the same type, ...).
--->
+- One of the two failures was a partial recovery: after a missed grasp the policy regrasped the block, but hit the
+  edge of the bin and dropped it.
+- All 10 episodes (both camera streams, joint states and actions) are in
+  [lichtphyz/rollout_diffusion_v3_desktop](https://huggingface.co/datasets/lichtphyz/rollout_diffusion_v3_desktop).
 
-_No evaluation results have been provided for this policy yet._
+<a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=lichtphyz/rollout_diffusion_v3_desktop">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/>
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/>
+</a>
 
 ---