DP-T perceived-pose checkpoints (bimanual_assembly)
These are pointflow diffusion policies (variant N: surface-normal point tokens, 75 epochs, training seed 42). They are trained on object poses from our own perception instead of the simulator's, for the DexJoCo bimanual_assembly task.
Ground truth enters only as the action labels and the pointflow target. At evaluation the same perception runs live once per replan.
| Folder | Training input (obj_pose) |
Eval input | Success (120 paired episodes, seeds 0–3 × 30) |
|---|---|---|---|
dpt_assembly_rand_obj_decoupled_pointflow_main_N_perc_dpe_s42 |
dpe: RGB tracker + depth tracker on perfect (simulator) depth, every step | same, live | 47.5 % (Oracle with true poses: 51.7 %, p = 0.64) |
dpt_assembly_rand_obj_decoupled_pointflow_main_N_perc_rpe_s42 |
rpe: the same run's RGB tracker only, every step | same, live (no depth) | 28.3 % |
Each folder mirrors outputs/dp/<experiment>/ of the code:
checkpoints/latest.ckpt: epoch 75, about 143 MB;.hydra/config.yamland.hydra/overrides.yaml: the exact training config;logs.json.txt: the training curve.
Code, the full pipeline and a porting guide live in the DP-T repository, branch cadfree-pointflow:
The training data is in the dataset repo b12902136/DP-T-perceived-pose-data.
Use
# 1. the checkpoints, into the code's outputs/dp/
hf download b12902136/DP-T-perceived-pose --local-dir outputs/dp
# or just one: --include 'dpt_assembly_rand_obj_decoupled_pointflow_main_N_perc_dpe_s42/*'
# 2. the perception's object models and the policy's object cloud (in the code repo, 4 MB)
bash assets/perceived_pose/install.sh # -> $VLA_ROOT/DP-T/outputs (else ./outputs)
# 3. evaluate with live perception (1 GPU per seed; ~0.7 GPU-h per 30 episodes)
PV=dpe VARIANT=N POSE=perceived SEED=42 EPISODES=30 RUNS=6 sbatch --array=0-3 slurm/main/eval.sbatch
PV=rpe VARIANT=N POSE=perceived SEED=42 EPISODES=30 RUNS=6 sbatch --array=0-3 slurm/main/eval.sbatch
eval.sbatch finds the checkpoint at outputs/dp/<experiment>/checkpoints/latest.ckpt. PV sets the live perception, which must match the training data:
- dpe:
--perceived-depth perfect --perceived-depth-fix --perceived-depth-model <shared_model_perfect_fused.npz> --perceived-canon --perceived-peg-spin hand --perceived-sam3-tf32 - rpe: the same without the three depth flags.
With plain eval_dp.py, pass these flags yourself (see slurm/main/common.sh).
The checkpoint config records the training machine's absolute path of the object cloud (policy.cloud_override). eval_dp.py (commit c3927e6 or later) maps it onto your outputs/ when that path does not exist; you can also pass --policy-cloud-file <cloud.npz>.
Caveats
- Perfect depth. dpe's depth is the simulator's own, noise-free, so it is an upper bound for an RGB-D sensor, not a real-camera result.
- Task-specific. The perception is assembly-specific: two objects (peg and tray), with their SAM 3 prompts, meshes and canonical rotations. Another task needs its own models (see the porting checklist in the guide).
- Different training data. These are not the GT-pose checkpoints of the main DP-T release.