baseline_3 v4 — DexYCB 162 + OakInk 207 → DP3 (full 3000-epoch run)

3D Diffusion Policy (DP3) trained from scratch on the combined DexYCB 162-episode + OakInk 207-episode baseline_3 v4 grasp dataset (369 episodes, 11 439 timesteps). Single RTX A6000 (49 GB), batch_size=128, AdamW lr=1e-4 cosine, 11h 2min wall time.

Checkpoint inventory

num_epochs=3000 is 0-indexed (epochs 0–2999), and checkpoint_every=200 means the last automatic save fires at epoch 2800. To give human-selectable mid-training snapshots independent of the topk eviction policy, a side watcher also copied latest.ckpt to preserved_epoch_NNNN.ckpt at 6 hand-picked epochs.

Epoch	File	Source
400	`preserved_epoch_0400.ckpt`	preserved
1000	`latest_epoch1000.ckpt`	preserved (uploaded earlier as intermediate-eval snapshot — same as preserved_epoch_1000 on disk)
1400	`preserved_epoch_1400.ckpt`	preserved
1600	`preserved_epoch_1600.ckpt`	preserved
1800	`preserved_epoch_1800.ckpt`	preserved
2000	`preserved_epoch_2000.ckpt`	preserved
2400	`epoch=2400-test_mean_score=-0.001.ckpt`	topk
2600	`epoch=2600-test_mean_score=-0.001.ckpt`	topk
2800	`epoch=2800-test_mean_score=-0.001.ckpt`	topk + latest (final state)

Each ckpt is a single-file DP3 snapshot (model state + EMA + normalizer; DP3 convention, no separate normalizer.pt). All 9 files are ~3.9 GB each (= 4 082 766 418 bytes).

⚠️ The `test_mean_score` in topk filenames is NOT a validation metric

It is -train_loss (train.py:303-304: if env_runner is None: step_log['test_mean_score'] = -train_loss). Topk-best ckpts are biased toward late-training epochs and should NOT be used as proxies for sim performance. Use the preserved snapshots for ckpt-to-ckpt comparison; final selection must be done via sim rollout in IsaacSim.

⚠️ Why the final ckpt is epoch 2800, not 2999

num_epochs=3000 is 0-indexed so training ran epochs 0–2999, but DP3's train.py gates BOTH latest.ckpt and topk saves behind the same (epoch % checkpoint_every == 0) condition (train.py:306). With checkpoint_every=200, the last save fires at epoch 2800; the additional 199 epochs (2801–2999) ran but were never persisted to disk. The vendored DP3 train.py was patched in gate3-curobo-ik after this run completed to force-save at the end of training; future runs will produce an epoch=NNNN-final.ckpt for the true end-of-training state. For this archived run, epoch 2800 is the latest available state.

Loss trajectory

Epoch	test_mean_score (= −train_loss)
0	−0.233
200	−0.004
600	−0.002
800	−0.002
1000	−0.001
2400	−0.001
2600	−0.001
2800	−0.001

Converged to ~1e-3 train loss by epoch 800; later epochs gave diminishing returns on train loss (but may still differ in sim-eval generalization).

Files in this repo

File	Size	Description
9× `*.ckpt`	~3.9 GB each	DP3 weight snapshots (table above)
`config.yaml`	3.3 KB	Resolved Hydra config used for this run
`task_baseline1_b3_v4_dexycb162_oakink207.yaml`	839 B	Task config (zarr path, shape_meta, val_ratio)
`dp3_dexycb162_oakink207_training_artifacts.tar.gz`	24 MB	TRAIN_LAUNCH.md + .hydra/ + train logs + offline wandb run dir
`README.md`	this file

The .tar.gz bundle is meant for full-run reproducibility / archive: it contains the Hydra resolved configs, hand-written TRAIN_LAUNCH.md with launch command + git SHA + hyperparameters, full stdout log, preserve-watcher log, and the offline wandb run that can be wandb sync'd.

Training setup (quick reference; full details in `TRAIN_LAUNCH.md` inside the tar.gz)

Model: DP3 (255.1M params), DDIM scheduler 100 train steps / 10 inference steps
Hardware: 1× RTX A6000 (49 GB)
Batch size: 128, AdamW lr=1e-4, cosine schedule, 500 warmup steps
Horizon: 16, n_obs_steps=2, n_action_steps=8
Action / state dim: 8 (xyz + qwxyz + gripper)
Point cloud: 4 096 pts × 3 dims, layernorm, no color
EMA: enabled (update_after_step=0, power=0.75)
Train/val split: val_ratio: 0.0 (all 369 episodes used as training; eval is sim-based)
Seed: 42, checkpoint_every=200, topk.k=3 by -train_loss

Reproducing

Clone git@github.com:stzabl-png/UCB_Project.git, branch gate3-curobo-ik, SHA d624364
Set up the dp3 conda env per Baseline1/RETRAIN_V4_FULL12.md

Download combined dataset:

huggingface-cli download UCBProject/DP3_DexYCB_training_data \
    --repo-type dataset --local-dir /tmp/dex_dl --include "data/*.hdf5"
huggingface-cli download UCBProject/DP3_OakInk_training_data \
    --repo-type dataset --local-dir /tmp/oak_dl --include "data/*.hdf5"
mkdir -p Baseline1/data/episodes_b3_v4_dexycb162_oakink207_2026-05-26/
cp /tmp/dex_dl/data/*.hdf5 /tmp/oak_dl/data/*.hdf5 \
    Baseline1/data/episodes_b3_v4_dexycb162_oakink207_2026-05-26/
# Expect 369 hdf5

Build zarr:

python Baseline1/convert_to_zarr.py \
    --input_dir Baseline1/data/episodes_b3_v4_dexycb162_oakink207_2026-05-26 \
    --output_zarr Baseline1/data/dp3_train_v4_dexycb162_oakink207.zarr

Place task_baseline1_b3_v4_dexycb162_oakink207.yaml under third_party/3D-Diffusion-Policy/3D-Diffusion-Policy/diffusion_policy_3d/config/task/

Launch (full command in TRAIN_LAUNCH.md):

python train.py --config-name=dp3.yaml task=baseline1_b3_v4_dexycb162_oakink207 ...

Companion artefacts

Source datasets:
- UCBProject/DP3_DexYCB_training_data
- UCBProject/DP3_OakInk_training_data
Previous DexYCB-only DP3 model (162 ep, separate run) is preserved locally on A6000 under dp3_runs/b3_v4_full12_3000/ and not yet uploaded.

Note on the IsaacSim env (sim eval)

The A6000 host this was trained on cannot run IsaacSim 5.0-rc or 5.1 (Ubuntu 20.04 / glibc 2.31 vs required ≥2.32 / ≥2.35). All sim rollout / eval for these checkpoints should be done on a separate machine with glibc ≥ 2.35 (e.g. the project's RTX 5090 dev box).

Verifying upload integrity

Each .ckpt is exactly 4 082 766 418 bytes. The intermediate-eval latest_epoch1000.ckpt (uploaded 2026-05-26 17:02 PDT) and preserved_epoch_1000.ckpt on the source disk are byte-identical (same provenance — latest_epoch1000.ckpt was cp-ed from preserved_epoch_1000.ckpt). For convenience the repo keeps the latest_epoch1000.ckpt filename rather than uploading a renamed duplicate.

Downloads last month: -

Video Preview

Robotics