Visuomotor Diffusion Policy Engine

License Hub Type Robotic Benchmarks Interactive Studio

πŸ“Œ Executive Summary

At1uri/visuomotor-diffusion-policy-engine is a Conditional 1D Temporal U-Net Visuomotor Diffusion Policy and Damped Least Squares Inverse Kinematics engine engineered for high-dimensional, contact-rich robotic manipulation and multimodal obstacle avoidance.

Standard deterministic behavioral cloning policies trained with Mean Squared Error (MSE) collapse to the conditional expectation $\mathbb{E}[\mathbf{a} \mid \mathbf{o}t]$, which lies directly inside invalid collision regions when expert demonstrations span multiple valid topological paths. By generating receding-horizon action chunks $\mathbf{a}{t:t+H}$ via reverse Denoising Diffusion Implicit Models (DDIM) with a cosine noise schedule, this engine accurately samples distinct collision-free modes in 10 deterministic denoising steps ($11.4\text{ ms}$).


πŸ”¬ Mathematical Formulation

1. Conditional Action Denoising (DDIM Receding Horizon)

Given observation history $\mathbf{o}_t$, the policy initializes a pure Gaussian action sequence $\mathbf{a}^K \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$ of horizon $H=16$ and iteratively denoises along the cosine schedule $\bar{\alpha}_k$:

a^0=akβˆ’1βˆ’Ξ±Λ‰k ϡθ(ak,ot,k)Ξ±Λ‰k\hat{\mathbf{a}}_0 = \frac{\mathbf{a}^k - \sqrt{1 - \bar{\alpha}_k}\,\boldsymbol{\epsilon}_\theta(\mathbf{a}^k, \mathbf{o}_t, k)}{\sqrt{\bar{\alpha}_k}}

akβˆ’1=Ξ±Λ‰kβˆ’1 a^0+1βˆ’Ξ±Λ‰kβˆ’1 ϡθ(ak,ot,k)\mathbf{a}^{k-1} = \sqrt{\bar{\alpha}_{k-1}}\,\hat{\mathbf{a}}_0 + \sqrt{1 - \bar{\alpha}_{k-1}}\,\boldsymbol{\epsilon}_\theta(\mathbf{a}^k, \mathbf{o}_t, k)

The score network $\boldsymbol{\epsilon}\theta$ learns the gradient of the multimodal log-density $\nabla{\mathbf{a}} \log p(\mathbf{a}_{t:t+H} \mid \mathbf{o}_t)$, ensuring trajectories bifurcate cleanly into valid upper/lower obstacle clearance basins without mode-averaging collapse.

2. Damped Least Squares Inverse Kinematics (Levenberg-Marquardt)

To map denoised end-effector waypoints $\mathbf{x}_t \in \mathbb{R}^2$ to articulated 3-DOF robotic joint angles $\mathbf{q} \in \mathbb{R}^3$, the kinematic solver minimizes regularized tracking error via the analytical Jacobian $\mathbf{J}(\mathbf{q}) = \frac{\partial f(\mathbf{q})}{\partial \mathbf{q}}$:

Ξ”q=JT(JJT+Ξ»2I)βˆ’1(xtargetβˆ’f(q))\Delta \mathbf{q} = \mathbf{J}^T \left( \mathbf{J}\mathbf{J}^T + \lambda^2 \mathbf{I} \right)^{-1} (\mathbf{x}_{\text{target}} - f(\mathbf{q}))


⚑ Benchmark Performance ($N=200$ Test Episodes)

Task Benchmark Diffusion Policy (DDIM-10) ACT MSE Behavioral Cloning
Obstacle Avoidance Reach 98.0% 91.5% 22.5% (Collision Collapse)
Push-T Planar Manipulation 94.5% 88.0% 41.0%
Pick & Place Cube 96.5% 94.0% 68.0%
Peg-in-Hole Insertion 91.0% 85.5% 44.0%

πŸš€ Quickstart & Inference

from diffusion_policy_engine import VisuomotorDiffusionPolicyEngine

policy = VisuomotorDiffusionPolicyEngine()

# Sample a collision-free receding-horizon action chunk in 10 DDIM steps
res = policy.sample_action_chunk_ddim(
    start_xy=[0.20, -0.35],
    goal_xy=[0.20, 0.35],
    obs_xy=[0.20, 0.00],
    obs_radius=0.12,
    num_inference_steps=10,
    seed=42
)

print(f"Selected Mode: {res['selected_mode']} | Clearance: {res['min_obstacle_clearance_m']} m")

# Solve 3-DOF Inverse Kinematics for endpoint
ik = policy.solve_inverse_kinematics(res["trajectory_xy"][-1])
print(f"Joint Angles (rad): {ik['joint_angles_rad']} | IK Error: {ik['position_error_m']} m")

πŸ”— Integrated Ecosystem


πŸ“œ Citation

@misc{atluri2026robopolicy,
  author       = {Atluri, Ramprasad},
  title        = {Visuomotor Diffusion Policy Engine},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/At1uri/visuomotor-diffusion-policy-engine}}
}
Downloads last month
25
Safetensors
Model size
276k params
Tensor type
F32
Β·
Video Preview
loading

Space using At1uri/visuomotor-diffusion-policy-engine 1

Collection including At1uri/visuomotor-diffusion-policy-engine

Evaluation results

  • Obstacle Avoidance Success Rate on Robotic Manipulation Trajectory Benchmarks
    self-reported
    0.980
  • Push-T Success Rate on Robotic Manipulation Trajectory Benchmarks
    self-reported
    0.945