Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- robotics
|
| 5 |
+
- diffusion-policy
|
| 6 |
+
- flow-matching
|
| 7 |
+
- lerobot
|
| 8 |
+
- rram
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# FMLP-Policy: Flow Matching MLP for Robotic Control
|
| 12 |
+
|
| 13 |
+
This project explores RRAM-compatible neural network architectures for robotic manipulation policies, replacing UNet with pure MLP (Linear + ReLU only) for deployment on analog RRAM accelerators.
|
| 14 |
+
|
| 15 |
+
## Overview
|
| 16 |
+
|
| 17 |
+
Diffusion Policy achieves SOTA on robotic manipulation but requires 50-100 denoising steps — impractical for RRAM deployment (each step needs ADC/DAC conversion). We explore:
|
| 18 |
+
|
| 19 |
+
1. **Streaming Flow Policy (SFP)** — reduces to 1-4 integration steps
|
| 20 |
+
2. **MLP velocity networks** — replaces UNet with RRAM-friendly architecture
|
| 21 |
+
3. **Quantization + noise tolerance** — validates INT8 deployment with device variation
|
| 22 |
+
|
| 23 |
+
## Models
|
| 24 |
+
|
| 25 |
+
| Model | Architecture | Description |
|
| 26 |
+
|-------|--------------|-------------|
|
| 27 |
+
| [pusht_diffusion_v3](https://huggingface.co/Liyux/pusht_diffusion_v3) | ResNet18 + UNet | DP baseline, 136 episodes |
|
| 28 |
+
| [pusht_diffusion_v4](https://huggingface.co/Liyux/pusht_diffusion_v4) | ResNet18 + UNet | DP baseline, 226 episodes |
|
| 29 |
+
| [pusht_diffusion_v5](https://huggingface.co/Liyux/pusht_diffusion_v5) | ResNet18 + UNet | DP baseline, 255 episodes (best) |
|
| 30 |
+
| [pusht_sfp_v9](https://huggingface.co/Liyux/pusht_sfp_v9) | ResNet18 + UNet | SFP working baseline |
|
| 31 |
+
| [pusht_sfp_v14](https://huggingface.co/Liyux/pusht_sfp_v14) | ResNet18 + UNet | SFP with h50/k2/σ1 params |
|
| 32 |
+
| [pusht_sfp_v15](https://huggingface.co/Liyux/pusht_sfp_v15) | ResNet18 + MLP | SFP with cond_residual MLP (RRAM-compatible) |
|
| 33 |
+
|
| 34 |
+
## Dataset
|
| 35 |
+
|
| 36 |
+
| Dataset | Episodes | Description |
|
| 37 |
+
|---------|----------|-------------|
|
| 38 |
+
| [pusht_real_merged](https://huggingface.co/datasets/Liyux/pusht_real_merged) | 255 | Real robot Push-T task, SO-101 arm, 320x240 |
|
| 39 |
+
|
| 40 |
+
## Key Results
|
| 41 |
+
|
| 42 |
+
**Sim (2D Push-T):**
|
| 43 |
+
- MLP achieves 0.86-0.88 FP32 vs UNet 0.74
|
| 44 |
+
- INT8 quantization: Bottleneck128+Skip achieves 0.86
|
| 45 |
+
- Noise tolerance: <6% accuracy drop at 10% multiplicative noise
|
| 46 |
+
|
| 47 |
+
**Real Robot:**
|
| 48 |
+
- DP v5: >90% success rate
|
| 49 |
+
- SFP v9: >70% success rate
|
| 50 |
+
- SFP v14/v15: Testing in progress
|
| 51 |
+
|
| 52 |
+
## Code
|
| 53 |
+
|
| 54 |
+
- [Liyux3/lerobot_MLP-SFP](https://github.com/Liyux3/lerobot_MLP-SFP)
|
| 55 |
+
|
| 56 |
+
## Hardware
|
| 57 |
+
|
| 58 |
+
- Robot: SO-101 (LeRobot compatible)
|
| 59 |
+
- Camera: HB Camera, top-down, 320x240 @ 30fps
|
| 60 |
+
- Training: 2x RTX 4090
|
| 61 |
+
|
| 62 |
+
## Citation
|
| 63 |
+
|
| 64 |
+
Coming soon.
|
| 65 |
+
|
| 66 |
+
## Acknowledgments
|
| 67 |
+
|
| 68 |
+
- [LeRobot](https://github.com/huggingface/lerobot) framework
|
| 69 |
+
- [Streaming Flow Policy](https://arxiv.org/abs/2505.21851) paper
|
| 70 |
+
- HKU EEE
|
| 71 |
+
|
| 72 |
+
---
|
| 73 |
+
|
| 74 |
+
*Part of FYP project at The University of Hong Kong, supervised by Prof. Han Wang.*
|