| # Jason's multi_task_dit — SC1 Final Checkpoint |
|
|
| **成功率: 9/9 (100%) on SO-101 real robot (2026-04-17)** |
| Policy: `multi_task_dit` (lerobot 0.5.1 built-in, ~277M params) |
|
|
| ## 環境 |
|
|
| ```bash |
| conda activate a # Jason's env on idlab2 |
| # lerobot 來自 /home/kunhsiang/jdk/lib/lerobot/ |
| # CLIP weights: /home/kunhsiang/jdk/hf_weights/clip-vit-base-patch16/ |
| ``` |
|
|
| ## 訓練指令（完整復現） |
|
|
| ```bash |
| cd /home/kunhsiang/jdk |
| CUDA_VISIBLE_DEVICES=1 lerobot-train \ |
| --dataset.root=datasets/sc1_newcolor \ |
| --dataset.repo_id=YOUR_DATASET \ |
| --output_dir=outputs/train/multi_task_dit_repro \ |
| --policy.type=multi_task_dit \ |
| --policy.device=cuda \ |
| --policy.horizon=32 \ |
| --policy.n_action_steps=24 \ |
| --policy.objective=diffusion \ |
| --policy.noise_scheduler_type=DDIM \ |
| --policy.num_train_timesteps=100 \ |
| --policy.num_inference_steps=20 \ |
| --policy.clip_sample=true \ |
| --policy.clip_sample_range=1.0 \ |
| --policy.hidden_dim=512 \ |
| --policy.num_layers=6 \ |
| --policy.num_heads=8 \ |
| --policy.repo_id=jedeka30/grasp_box \ |
| --wandb.enable=true \ |
| --wandb.project=act \ |
| --steps=50000 \ |
| --save_freq=5000 \ |
| --batch_size=24 \ |
| --num_workers=8 \ |
| --eval_freq=2000 |
| ``` |
|
|
| ## 超參數摘要 |
|
|
| | 參數 | 值 | |
| |------|-----| |
| | Policy | multi_task_dit | |
| | Objective | diffusion (DDIM) | |
| | horizon | 32 | |
| | n_action_steps | 24 | |
| | **num_inference_steps** | **20** (關鍵，比 10 好很多) | |
| | num_train_timesteps | 100 | |
| | hidden_dim | 512 | |
| | num_layers | 6 | |
| | num_heads | 8 | |
| | batch_size | 24 | |
| | steps | 50,000 | |
| | Dataset | sc1_newcolor (3色: red/blue/green, ~80 episodes) | |
| |
| ## Eval 指令 |
| |
| ```bash |
| cd /home/kunhsiang/jdk |
| |
| # SC1: red/green/blue 三色（自動循環） |
| bash 3kh.sh ckpts/final_ckpts/pretrained_model |
| |
| # SC2/3: on/outside the plate |
| bash 2test.sh ckpts/final_ckpts/pretrained_model |
| ``` |
| |
| ## 硬體設定 |
| |
| | 設備 | 值 | |
| |------|-----| |
| | Robot | SO-101 follower | |
| | Port | /dev/ttyACM1 | |
| | GPU | CUDA_VISIBLE_DEVICES=1 | |
| | cam_front | /dev/video2 (640×480, 30fps, MJPG) | |
| | cam_gripper | /dev/video0 (640×480, 30fps, MJPG) | |
| | cam_top | /dev/video4 (640×480, 30fps, MJPG) | |
| | episode_time_s | 90 | |
|
|
| ## 評測結果 |
|
|
| | Task | SR | Date | |
| |------|-----|------| |
| | grasp sc1 (9 trials: red×3 / green×3 / blue×3) | **9/9 (100%)** | 2026-04-17 | |
| | grasp sc4 (put on plate) | 2/6 (33%) | 2026-05-04 | |
|
|
| ## 注意 |
|
|
| - lerobot 的 `multi_task_dit` 有 offline CLIP hack: 讀 `/home/kunhsiang/jdk/hf_weights/clip-vit-base-patch16/` |
| 標準 lerobot 0.5.1 會從 HF 下載 `openai/clip-vit-base-patch16`（需網路） |
| - `TRANSFORMERS_OFFLINE=1` + `HF_DATASETS_OFFLINE=1` 在訓練時設定（離線模式） |
| - 訓練完後 `make reset` 會把 robot 歸位（Makefile 在 jdk 根目錄） |
|
|
| ## 相關 Repo |
|
|
| - Dataset: `kunhsiang/jason-sc1-newcolor-dataset` |
| - Eval scripts: 此 repo 內 `scripts/` 目錄 |
|
|
