Initial release: DP finetuned on PushT-with-obstacles, 95% success

README.md

@@ -0,0 +1,108 @@
+---
+license: apache-2.0
+library_name: lerobot
+pipeline_tag: robotics
+tags:
+  - robotics
+  - diffusion-policy
+  - imitation-learning
+  - pusht
+base_model: lerobot/diffusion_pusht
+---
+
+# Diffusion Policy — PushT with Obstacles
+
+Diffusion Policy finetuned from [`lerobot/diffusion_pusht`](https://huggingface.co/lerobot/diffusion_pusht)
+for the PushT manipulation task **with random circular obstacles**: the agent
+pushes a T-shaped block to a goal pose while avoiding 1–3 obstacles per episode.
+
+The base model handles standard PushT but has zero obstacle awareness
+(0% success, 55% obstacle-hit rate as zero-shot baseline). Finetuning on
+101 obstacle-aware demonstrations recovers a working policy.
+
+## Results
+
+| Checkpoint | Success Rate | Obstacle-Hit Rate |
+|---|---|---|
+| Base (`lerobot/diffusion_pusht`, zero-shot on obstacle env) | 0% | 55% |
+| **This model** (best of 30k finetune steps) | **95%** | **0%** |
+
+Evaluated on `PushTObstacleEnv` with 20 episodes per checkpoint, 300 max steps,
+success threshold 0.95 coverage.
+
+> Note: 20 episodes is a noisy estimator (Wilson 95% CI ≈ ±20%). Treat the
+> 95% headline as approximate; a 100-episode re-evaluation is recommended.
+
+## Architecture
+
+Inherited from the base model (no architecture changes, only weight finetuning):
+
+| Field | Value |
+|---|---|
+| Vision backbone | ResNet-18 |
+| Image input | 3×96×96 (random-cropped to 84×84) |
+| State input | 2 (agent_pos) |
+| Action output | 2 |
+| `n_obs_steps` | 2 |
+| `horizon` | 16 |
+| `n_action_steps` | 8 |
+| Diffusion timesteps | 100 |
+| Parameters | 262,709,026 |
+
+## Training
+
+- **Hardware**: 1× NVIDIA H100 (NCSA Delta AI), AMP enabled
+- **Wall time**: ~3 hours for 30k steps
+- **Optimizer**: AdamW, β=(0.95, 0.999), wd=1e-6
+- **LR**: 3e-5 (constant after 100-step warmup)
+- **Batch size**: 64
+- **Dataset**: 101 episodes / 15,758 frames @ 10 fps, recorded with mouse teleop
+  in `pusht_obstacle_env.py`
+- **Normalization**: dataset stats recomputed locally (image mean/std differ
+  from the base PushT distribution due to obstacle pixels)
+
+## Usage
+
+```python
+from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy
+
+policy = DiffusionPolicy.from_pretrained("zengxy0624/diffusion-pusht-obstacles")
+policy.eval()
+```
+
+Drop-in compatible with the base model — same input/output schema, just
+swap the repo id.
+
+## Limitations
+
+- Only trained on circular obstacles with radius 15 px and 1–3 per episode.
+  Out-of-distribution obstacle counts/shapes are not handled.
+- Late-training evaluation showed high variance (occasional collapses to
+  10% success). The released checkpoint is a single best-of-N draw and may
+  not exactly reproduce 95% on a fresh 100-episode evaluation.
+- No EMA was used during training; the base `lerobot/diffusion_pusht` model
+  was trained with EMA. Adding EMA is a known follow-up.
+
+## Citation
+
+If this checkpoint is useful, please cite the original Diffusion Policy work:
+
+```bibtex
+@article{chi2023diffusion,
+  title={Diffusion Policy: Visuomotor Policy Learning via Action Diffusion},
+  author={Chi, Cheng and Feng, Siyuan and Du, Yilun and Xu, Zhenjia and Cousineau, Eric and Burchfiel, Benjamin and Song, Shuran},
+  journal={The International Journal of Robotics Research},
+  year={2023},
+}
+```
+
+And LeRobot:
+
+```bibtex
+@misc{cadene2024lerobot,
+  title={LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
+  author={Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Wolf, Thomas},
+  year={2024},
+  url={https://github.com/huggingface/lerobot},
+}
+```

config.json

@@ -0,0 +1,82 @@
+{
+    "type": "diffusion",
+    "n_obs_steps": 2,
+    "input_features": {
+        "observation.image": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                96,
+                96
+            ]
+        },
+        "observation.state": {
+            "type": "STATE",
+            "shape": [
+                2
+            ]
+        }
+    },
+    "output_features": {
+        "action": {
+            "type": "ACTION",
+            "shape": [
+                2
+            ]
+        }
+    },
+    "device": "cpu",
+    "use_amp": false,
+    "push_to_hub": true,
+    "repo_id": null,
+    "private": null,
+    "tags": null,
+    "license": null,
+    "pretrained_path": null,
+    "horizon": 16,
+    "n_action_steps": 8,
+    "normalization_mapping": {
+        "ACTION": "MIN_MAX",
+        "STATE": "MIN_MAX",
+        "VISUAL": "MEAN_STD"
+    },
+    "drop_n_last_frames": 7,
+    "vision_backbone": "resnet18",
+    "crop_shape": [
+        84,
+        84
+    ],
+    "crop_is_random": true,
+    "pretrained_backbone_weights": null,
+    "use_group_norm": true,
+    "spatial_softmax_num_keypoints": 32,
+    "use_separate_rgb_encoder_per_camera": false,
+    "down_dims": [
+        512,
+        1024,
+        2048
+    ],
+    "kernel_size": 5,
+    "n_groups": 8,
+    "diffusion_step_embed_dim": 128,
+    "use_film_scale_modulation": true,
+    "noise_scheduler_type": "DDPM",
+    "num_train_timesteps": 100,
+    "beta_schedule": "squaredcos_cap_v2",
+    "beta_start": 0.0001,
+    "beta_end": 0.02,
+    "prediction_type": "epsilon",
+    "clip_sample": true,
+    "clip_sample_range": 1.0,
+    "num_inference_steps": null,
+    "do_mask_loss_for_padding": false,
+    "optimizer_lr": 0.0001,
+    "optimizer_betas": [
+        0.95,
+        0.999
+    ],
+    "optimizer_eps": 1e-08,
+    "optimizer_weight_decay": 1e-06,
+    "scheduler_name": "cosine",
+    "scheduler_warmup_steps": 500
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c80dfe0bd0b823af9db3a67f46faba420d35ea5e02d2069df97922a7850f054
+size 1050861448