README.md

---
license: mit
tags:
- robotics
- imitation-learning
- diffusion-policy
- manipulation
- hiro-robot
- lerobot
- goal-conditioned
- sensor-diffusion
datasets:
- roboset_20260112_225816
- roboset_20260113_001336
---

# Proximity Sensor Goal-Conditioned Diffusion Policy

## Model Description

A goal-conditioned Diffusion Policy trained on sensor datasets. The model predicts joint positions (next positions along trajectory) conditioned on the current observation (joint positions, table camera image) and a goal cartesian position.

## Model Architecture

- **Policy Type**: Diffusion Policy
- **Framework**: LeRobot
- **Horizon**: 16 steps
- **Observation Steps**: 1 step (single timestep)

## Inputs

- **`observation.state`**: Shape `(batch, 1, 7)` - Joint positions (7 DOF arm)
- **`observation.goal`**: Shape `(batch, 1, 3)` - Goal cartesian position (X, Y, Z)
- **`observation.images.table_camera`**: Shape `(batch, 1, 3, 480, 640)` - Table camera RGB images

## Outputs

- **`action`**: Shape `(batch, 16, 7)` - Joint positions (7 DOF) for 16-step horizon (next positions along trajectory)

**Note**: The model outputs a full 16-step horizon. Use `select_action()` to get the first step `(batch, 7)`, or `predict_action_chunk()` to get the full horizon `(batch, 16, 7)`.

## Normalization

### Input Normalization

**Images** (`observation.images.table_camera`):
- Normalize from `[0, 255]` to `[0, 1]` by dividing by `255.0`
- Then apply mean-std normalization using dataset statistics (handled by preprocessor)

**State** (`observation.state`):
- Apply min-max normalization: `(state - min) / (max - min)` using dataset statistics (handled by preprocessor)

**Goal** (`observation.goal`):
- Apply min-max normalization: `(goal - min) / (max - min)` using dataset statistics (handled by preprocessor)

### Output Unnormalization

**Actions** (`action`):
- Apply inverse min-max normalization: `action * (max - min) + min` using dataset statistics (handled by postprocessor)
- **Note**: Actions are joint positions (not velocities) - these are the next positions the robot should move to along the trajectory

## Usage

```python
from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy
from lerobot.policies.factory import make_pre_post_processors

# Load model
policy = DiffusionPolicy.from_pretrained("calebescobedo/sensor-diffusion-policy-v1")

# Load preprocessor and postprocessor from the same repo
preprocessor, postprocessor = make_pre_post_processors(
    policy_cfg=policy.config,
    pretrained_path="calebescobedo/sensor-diffusion-policy-v1"
)

# Prepare inputs
batch = {
    'observation.state': state_tensor,  # (batch, 1, 7) - raw joint positions
    'observation.goal': goal_tensor,  # (batch, 1, 3) - raw goal xyz
    'observation.images.table_camera': table_img,  # (batch, 1, 3, 480, 640) - uint8 [0,255] or float [0,1]
}

# Inference
policy.eval()
with torch.no_grad():
    batch = preprocessor(batch)  # Normalizes inputs
    actions = policy.select_action(batch)  # Returns normalized actions
    actions = postprocessor(actions)  # Unnormalizes to raw joint positions
```

## Training Details

- **Training**: Epoch-based (ensures all trajectories seen)
- **Batch Size**: 64
- **Optimizer**: Adam
- **Mixed Precision**: Enabled (AMP)
- **Data Loading**: Optimized with persistent file handles
- **Datasets**: 
  - roboset_20260112_225816.h5 (20 trajectories)
  - roboset_20260113_001336.h5 (50 trajectories)

## Dataset Notes

- All trajectories have the same start and end positions
- Single demonstration repeated 70 times
- Goal: Final cartesian position `[0.454, -0.133, 0.522]` (constant)

## License

MIT License