YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
pi0.5 LoRA fine-tune β realpushmultit (T-shaped block, 3-target push)
Real-robot LoRA fine-tune of lerobot/pi05_base on
harrywang01/RealPushMultiT
(240 demos / 341077 timesteps, 2 cameras img_third + img_wrist, 7-DOF EEF action).
Task instruction (constant across all demos):
Push the T-shaped block to visit three different target locations on the tabletop, without visiting the same target more than once
Training setup
- Base model:
lerobot/pi05_base(3.62B params, frozen except LoRA) - LoRA r=24 Ξ±=48 on
gemma_expert.self_attn.{q,k,v,o}_proj+ r=16 Ξ±=32 onpaligemma.language_model.layers.*.self_attn.{q,v}_proj modules_to_save:action_in_proj,action_out_proj,time_mlp_in,time_mlp_out- Trainable params: 5.93M / 3.62B (0.16%)
- Hardware: 2ΓH100 80GB
- Per-rank batch=128, grad_accum=1 β effective batch=256
- AdamW lr=4e-4, cosine schedule, warmup=300
- bf16 + gradient checkpointing
- num_epochs=20 (manual early-stop planned)
- WandB run: https://wandb.ai/williamcao-uc-san-diego/pi05_realpushmultit_lora/runs/u0e6g8ys
Checkpoints (live β uploaded as training proceeds)
| File | val_loss | opt-step | Notes |
|---|---|---|---|
epoch=0000-val=0.0146.ckpt |
0.0146 | 1266 | end of epoch 0 |
epoch=0001-val=0.0116.ckpt |
0.0116 | 2532 | |
epoch=0002-val=0.0122.ckpt |
0.0122 | 3798 | slight uptick after a transient loss spike at step ~560 that grad-clip absorbed |
epoch=0003-val=0.0114.ckpt |
0.0114 | 5064 | best so far |
epoch=0004-val=0.0115.ckpt |
0.0115 | 6330 | near best, slight uptick |
latest.ckpt |
mid-epoch | (varies) | most recent training state |
milestone_epoch0001.ckpt |
0.0116 | 2532 | preserved milestone |
milestone_epoch0003.ckpt |
0.0114 | 5064 | preserved milestone |
Each file is a Diffusion Policy workspace snapshot (LoRA adapters + modules_to_save + optimizer/EMA state). 23.8 MB per file β base pi05 weights are not embedded (only the trainable 5.93M params).
Loading
The ckpts are workspace-format. To load, you need the matching
memory_diffusion_policy repo + lerobot/pi05_base weights.
See the original config:
memory_diffusion_policy/config/train_pi05_realpushmultit_lora_workspace.yaml.