qshou-coder commited on
Commit
4eb45ed
·
verified ·
1 Parent(s): a961787

Add final FastWAM zipper_pulling checkpoint (16380 steps)

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. baseline/fastwam_zipper/README.md +44 -0
  2. baseline/fastwam_zipper/THIRD_PARTY.md +3 -0
  3. baseline/fastwam_zipper/context.pt +3 -0
  4. baseline/fastwam_zipper/inference.py +107 -0
  5. baseline/fastwam_zipper/inference_config.json +78 -0
  6. baseline/fastwam_zipper/inference_manifest.json +539 -0
  7. baseline/fastwam_zipper/models/LICENSE-WAN-APACHE-2.0.txt +202 -0
  8. baseline/fastwam_zipper/models/Wan2.2-TI2V-5B/Wan2.2_VAE.pth +3 -0
  9. baseline/fastwam_zipper/norm_stats.json +136 -0
  10. baseline/fastwam_zipper/provenance.json +20 -0
  11. baseline/fastwam_zipper/requirements-inference.txt +12 -0
  12. baseline/fastwam_zipper/source/LICENSE +24 -0
  13. baseline/fastwam_zipper/source/README.md +437 -0
  14. baseline/fastwam_zipper/source/configs/data/galaxea_r1_lite_3cam.yaml +87 -0
  15. baseline/fastwam_zipper/source/configs/data/libero_2cam.yaml +68 -0
  16. baseline/fastwam_zipper/source/configs/data/libero_2cam_lerobot_v30.yaml +59 -0
  17. baseline/fastwam_zipper/source/configs/data/robotwin.yaml +125 -0
  18. baseline/fastwam_zipper/source/configs/model/fastwam.yaml +59 -0
  19. baseline/fastwam_zipper/source/configs/model/fastwam_idm.yaml +60 -0
  20. baseline/fastwam_zipper/source/configs/model/fastwam_joint.yaml +58 -0
  21. baseline/fastwam_zipper/source/configs/model/fastwam_optional_idm.yaml +62 -0
  22. baseline/fastwam_zipper/source/configs/sim_libero.yaml +59 -0
  23. baseline/fastwam_zipper/source/configs/sim_robotwin.yaml +51 -0
  24. baseline/fastwam_zipper/source/configs/task/galaxea_r1_lite_3cam_1e-4.yaml +46 -0
  25. baseline/fastwam_zipper/source/configs/task/libero_idm_2cam224_1e-4.yaml +27 -0
  26. baseline/fastwam_zipper/source/configs/task/libero_joint_2cam224_1e-4.yaml +27 -0
  27. baseline/fastwam_zipper/source/configs/task/libero_optional_idm_2cam224_1e-4.yaml +24 -0
  28. baseline/fastwam_zipper/source/configs/task/libero_uncond_2cam224_1e-4.yaml +27 -0
  29. baseline/fastwam_zipper/source/configs/task/robotwin_idm_3cam_384_1e-4.yaml +27 -0
  30. baseline/fastwam_zipper/source/configs/task/robotwin_joint_3cam_384_1e-4.yaml +27 -0
  31. baseline/fastwam_zipper/source/configs/task/robotwin_uncond_3cam_384_1e-4.yaml +27 -0
  32. baseline/fastwam_zipper/source/configs/train.yaml +48 -0
  33. baseline/fastwam_zipper/source/pyproject.toml +48 -0
  34. baseline/fastwam_zipper/source/scripts/__init__.py +1 -0
  35. baseline/fastwam_zipper/source/scripts/accelerate_configs/accelerate_zero0.yaml +22 -0
  36. baseline/fastwam_zipper/source/scripts/accelerate_configs/accelerate_zero1_ds.yaml +18 -0
  37. baseline/fastwam_zipper/source/scripts/accelerate_configs/accelerate_zero2_ds.yaml +18 -0
  38. baseline/fastwam_zipper/source/scripts/check_galaxea_r1_lite_contract.py +91 -0
  39. baseline/fastwam_zipper/source/scripts/check_galaxea_text_cache.py +47 -0
  40. baseline/fastwam_zipper/source/scripts/dryrun_fastwam.py +227 -0
  41. baseline/fastwam_zipper/source/scripts/ds_configs/ds_zero1_config.json +18 -0
  42. baseline/fastwam_zipper/source/scripts/ds_configs/ds_zero2_config.json +19 -0
  43. baseline/fastwam_zipper/source/scripts/precompute_text_embeds.py +389 -0
  44. baseline/fastwam_zipper/source/scripts/prepare_galaxea_r1_lite_overlay.py +141 -0
  45. baseline/fastwam_zipper/source/scripts/preprocess_action_dit_backbone.py +240 -0
  46. baseline/fastwam_zipper/source/scripts/train.py +16 -0
  47. baseline/fastwam_zipper/source/scripts/train_galaxea_r1_lite.sh +125 -0
  48. baseline/fastwam_zipper/source/scripts/train_zero1.sh +116 -0
  49. baseline/fastwam_zipper/source/scripts/train_zero2.sh +116 -0
  50. baseline/fastwam_zipper/source/src/fastwam.egg-info/PKG-INFO +37 -0
baseline/fastwam_zipper/README.md ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # FastWAM — zipper_pulling
2
+
3
+ Final checkpoint from `bifrost-2026102616142400-shouqx`: **30 epochs**, **16,380 optimizer steps**. The checkpoint is uploaded unchanged; its SHA-256 is recorded in `provenance.json`.
4
+
5
+ ## Included inference assets
6
+
7
+ - `step_016380.pt`: final MoT (video/action) and proprioception encoder weights.
8
+ - `models/Wan2.2-TI2V-5B/Wan2.2_VAE.pth`: the frozen VAE actually used in training.
9
+ - `context.pt`: exact cached native UMT5 embedding for **Pull the zipper.**.
10
+ - `norm_stats.json`: this task's original state/action q01/q99 statistics, byte-identical to training.
11
+ - `inference_config.json`, `inference.py`, `requirements-inference.txt`, and exact training source in `source/`.
12
+
13
+ The fixed-task path works offline after downloading this entire directory and installing Python dependencies. No external model checkpoint, tokenizer, or text encoder is required: it uses the included text cache. Changing the instruction requires a newly computed matching native UMT5 cache; editing the prompt string alone does not change conditioning.
14
+
15
+ ## Download
16
+
17
+ Download the complete task directory (including the VAE and source):
18
+
19
+ ```python
20
+ from huggingface_hub import snapshot_download
21
+ snapshot_download("qshou-coder/SpaceDreamer_Galexea", allow_patterns=["baseline/fastwam_zipper/**"], local_dir="fastwam_export")
22
+ ```
23
+
24
+ Then enter `fastwam_export/baseline/fastwam_zipper/`.
25
+
26
+ ## Setup and inference
27
+
28
+ Use Python 3.10+ and a CUDA GPU with BF16 support and sufficient free VRAM (training used PyTorch 2.7.1 / CUDA 12.8). From this directory:
29
+
30
+ ```bash
31
+ python -m pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128
32
+ python -m pip install -r requirements-inference.txt
33
+ python inference.py --input observation.npz --output prediction.npz --device cuda:0 --steps 20 --seed 0
34
+ ```
35
+
36
+ `observation.npz` must contain `head_left`, `wrist_left`, `wrist_right` (HxWx3 **RGB uint8**, no BGR conversion), and `state` (float32, shape `[14]`, original physical state units). The order is **left arm 6, left gripper, right arm 6, right gripper**.
37
+
38
+ The head image is resized to 256x320 and the two wrists to 128x160, then packed as head on top, left/right wrists below, giving 384x320. Inference applies the training q01/q99 transform and no random crop/color augmentation. `prediction.npz` contains `actions` `[40,14]` in original physical target units and `normalized_actions`. Actions are **absolute original `motion_target.position` labels**, not deltas, Cartesian commands, or velocity. Keep the native gripper encoding and coordinate conventions. The data rate was 20 Hz; execution/replanning is controlled by the downstream robot integration, not this file-based runner.
39
+
40
+ Python API: `from inference import Policy; policy = Policy(); result = policy.predict(images, state)`. `predict_video=True` additionally uses the native joint sampler to return nine video frames; default inference uses the native cached action-only path. Default 20 denoising steps is the native inference default, not a claimed evaluation optimum.
41
+
42
+ ## Validation scope
43
+
44
+ See `validation.json` for actual checks. Packaging checks cover source identity, checkpoint step and architecture, cached text, training preprocessing and normalization parity, inverse normalization, and relative asset resolution. They do not constitute a new real-robot rollout or GPU performance measurement. Training completion receipts confirm final save and strict reload; the original training smoke used an earlier smoke checkpoint.
baseline/fastwam_zipper/THIRD_PARTY.md ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ # Third-party components
2
+
3
+ The exact FastWAM training source is included under `source/`, with its original MIT license and source headers. The frozen `Wan2.2_VAE.pth` is the original Wan-AI/Wan2.2-TI2V-5B component, SHA-256 `20eb789667fa5e60e7516bf509512f6cb61f01b0aa0695eadaea930c13892b36`. Wan2.2 is distributed under Apache-2.0; see https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B and https://github.com/Wan-Video/Wan2.2 . The native task text cache was produced with the frozen Wan UMT5 encoder.
baseline/fastwam_zipper/context.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24c9c95ba634779ba7aeb33743ee38b3cf688805471889400a2e841fd4ea4ad1
3
+ size 1050977
baseline/fastwam_zipper/inference.py ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Portable, fixed-task FastWAM inference using the exact training source."""
2
+ import argparse
3
+ import json
4
+ import os
5
+ from pathlib import Path
6
+ import sys
7
+ import numpy as np
8
+ import torch
9
+ from torchvision.transforms import functional as F, InterpolationMode
10
+
11
+ CAMERAS=('head_left','wrist_left','wrist_right')
12
+
13
+ def pack_images(images):
14
+ views=[]
15
+ for name in CAMERAS:
16
+ image=np.asarray(images[name])
17
+ if image.ndim!=3 or image.shape[-1]!=3 or image.dtype!=np.uint8:
18
+ raise ValueError(f'{name} must be HxWx3 uint8 RGB (not BGR).')
19
+ v=torch.from_numpy(image.copy()).permute(2,0,1).float()/255.
20
+ size=[256,320] if name=='head_left' else [128,160]
21
+ views.append(F.resize(v,size,InterpolationMode.BILINEAR,antialias=True))
22
+ return torch.cat([views[0],torch.cat(views[1:],dim=-1)],dim=-2).contiguous()*2-1
23
+
24
+ def normalize(x,stats):
25
+ x=np.asarray(x,dtype=np.float32)
26
+ lo,hi=np.array(stats['q01']),np.array(stats['q99'])
27
+ if x.shape[-1]!=14 or lo.shape!=(14,) or hi.shape!=(14,) or not np.isfinite(x).all():
28
+ raise ValueError('Expected finite 14D state/actions and 14D quantile bounds.')
29
+ result=np.zeros_like(x,dtype=np.float32);mask=~np.isclose(hi,lo)
30
+ result[...,mask]=(x[...,mask]-lo[mask])/(hi[mask]-lo[mask])
31
+ result[...,mask]=2*result[...,mask]-1
32
+ return np.clip(result,-1,1)
33
+
34
+ def denormalize(x,stats):
35
+ x=np.asarray(x,dtype=np.float32)
36
+ lo,hi=np.array(stats['q01']),np.array(stats['q99'])
37
+ if x.shape[-1]!=14 or not np.isfinite(x).all():raise ValueError('Invalid model actions.')
38
+ result=(x+1)/2*(hi-lo)+lo
39
+ result[...,np.isclose(hi,lo)]=lo[np.isclose(hi,lo)]
40
+ return result.astype(np.float32)
41
+
42
+ class Policy:
43
+ def __init__(self,bundle=None,device='cuda:0'):
44
+ self.root=Path(bundle or Path(__file__).parent).resolve()
45
+ cfg=json.loads((self.root/'inference_config.json').read_text())
46
+ self.config=cfg
47
+ self.stats=json.loads((self.root/'norm_stats.json').read_text())['norm_stats']
48
+ context=torch.load(self.root/'context.pt',map_location='cpu',weights_only=True)
49
+ if context['prompt']!=cfg['encoded_prompt']:raise ValueError('Task/context mismatch.')
50
+ if context['context'].shape!=(128,4096) or context['mask'].shape!=(128,):raise ValueError('Invalid text cache.')
51
+ self.context=context
52
+ sys.path.insert(0,str(self.root/'source/src'))
53
+ os.environ['DIFFSYNTH_MODEL_BASE_PATH']=str(self.root/'models')
54
+ os.environ['DIFFSYNTH_SKIP_DOWNLOAD']='true'
55
+ if not (self.root/'models/Wan2.2-TI2V-5B/Wan2.2_VAE.pth').is_file():raise FileNotFoundError('Missing bundled VAE.')
56
+ from fastwam.models.wan22.fastwam import FastWAM
57
+ m=cfg['model']
58
+ self.model=FastWAM.from_wan22_pretrained(
59
+ device=device,torch_dtype=torch.bfloat16,
60
+ model_id=m['model_id'],tokenizer_model_id=m['tokenizer_model_id'],
61
+ tokenizer_max_len=m['tokenizer_max_len'],load_text_encoder=False,
62
+ proprio_dim=m['proprio_dim'],redirect_common_files=False,
63
+ video_dit_config=m['video_dit_config'],action_dit_config=m['action_dit_config'],
64
+ skip_dit_load_from_pretrain=True,action_dit_pretrained_path=None,
65
+ mot_checkpoint_mixed_attn=False,compile_training_denoise=False,
66
+ video_train_shift=m['video_scheduler']['train_shift'],video_infer_shift=m['video_scheduler']['infer_shift'],
67
+ video_num_train_timesteps=m['video_scheduler']['num_train_timesteps'],
68
+ action_train_shift=m['action_scheduler']['train_shift'],action_infer_shift=m['action_scheduler']['infer_shift'],
69
+ action_num_train_timesteps=m['action_scheduler']['num_train_timesteps'])
70
+ checkpoint=torch.load(self.root/cfg['checkpoint'],map_location='cpu',weights_only=True,mmap=True)
71
+ if checkpoint['step']!=cfg['final_step']:raise ValueError('Checkpoint step mismatch.')
72
+ self.model.mot.load_state_dict(checkpoint['mot'],strict=True)
73
+ self.model.proprio_encoder.load_state_dict(checkpoint['proprio_encoder'],strict=True)
74
+ self.model.eval()
75
+
76
+ @torch.inference_mode()
77
+ def predict(self,images,state,steps=20,seed=None,predict_video=False):
78
+ state=np.asarray(state,dtype=np.float32)
79
+ if state.shape!=(14,):raise ValueError('state must be shape (14,), in the original physical units.')
80
+ if steps<1:raise ValueError('steps must be positive')
81
+ args=dict(prompt=None,input_image=pack_images(images),action_horizon=40,
82
+ proprio=torch.from_numpy(normalize(state,self.stats['state'])),
83
+ context=self.context['context'],context_mask=self.context['mask'],
84
+ num_inference_steps=steps,seed=seed,compile_action_infer=False)
85
+ if predict_video:
86
+ result=self.model.infer_joint(**args,num_video_frames=9,test_action_with_infer_action=False)
87
+ else:result=self.model.infer_action(**args)
88
+ normalized=result['action'].detach().float().cpu().numpy()
89
+ if normalized.shape!=(40,14):raise ValueError(f'Unexpected action shape {normalized.shape}')
90
+ output={'actions':denormalize(normalized,self.stats['actions']),'normalized_actions':normalized}
91
+ if predict_video:output['video']=result['video']
92
+ return output
93
+
94
+ def main():
95
+ p=argparse.ArgumentParser(description=__doc__)
96
+ p.add_argument('--bundle',type=Path,default=Path(__file__).parent)
97
+ p.add_argument('--input',type=Path,required=True,help='NPZ: head_left, wrist_left, wrist_right RGB uint8 arrays; state float32[14].')
98
+ p.add_argument('--output',type=Path,required=True)
99
+ p.add_argument('--device',default='cuda:0');p.add_argument('--steps',type=int,default=20);p.add_argument('--seed',type=int,default=None)
100
+ a=p.parse_args()
101
+ with np.load(a.input,allow_pickle=False) as data:
102
+ images={name:data[name] for name in CAMERAS};state=data['state']
103
+ result=Policy(a.bundle,a.device).predict(images,state,a.steps,a.seed)
104
+ np.savez(a.output,**result)
105
+ print(json.dumps({'output':str(a.output),'action_shape':list(result['actions'].shape),'action_semantics':'absolute motion_target.position'}))
106
+
107
+ if __name__=='__main__':main()
baseline/fastwam_zipper/inference_config.json ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "task": "zipper_pulling",
3
+ "training_job": "bifrost-2026102616142400-shouqx",
4
+ "final_step": 16380,
5
+ "checkpoint": "step_016380.pt",
6
+ "prompt": "Pull the zipper.",
7
+ "encoded_prompt": "A video recorded from a robot's point of view executing the following instruction: Pull the zipper.",
8
+ "fps": 20,
9
+ "action_horizon": 40,
10
+ "video_frames": 9,
11
+ "model": {
12
+ "model_id": "Wan2.2-TI2V-5B",
13
+ "tokenizer_model_id": "Wan2.1-T2V-1.3B",
14
+ "tokenizer_max_len": 128,
15
+ "load_text_encoder": false,
16
+ "proprio_dim": 14,
17
+ "redirect_common_files": false,
18
+ "mot_checkpoint_mixed_attn": false,
19
+ "compile_training_denoise": false,
20
+ "action_dit_pretrained_path": null,
21
+ "skip_dit_load_from_pretrain": true,
22
+ "video_dit_config": {
23
+ "has_image_input": false,
24
+ "patch_size": [
25
+ 1,
26
+ 2,
27
+ 2
28
+ ],
29
+ "in_dim": 48,
30
+ "hidden_dim": 3072,
31
+ "ffn_dim": 14336,
32
+ "freq_dim": 256,
33
+ "text_dim": 4096,
34
+ "out_dim": 48,
35
+ "num_heads": 24,
36
+ "attn_head_dim": 128,
37
+ "num_layers": 30,
38
+ "eps": 1e-06,
39
+ "seperated_timestep": true,
40
+ "require_clip_embedding": false,
41
+ "require_vae_embedding": false,
42
+ "fuse_vae_embedding_in_latents": true,
43
+ "use_gradient_checkpointing": false,
44
+ "video_attention_mask_mode": "first_frame_causal",
45
+ "action_conditioned": false,
46
+ "action_dim": 14,
47
+ "action_group_causal_mask_mode": "group_diagonal"
48
+ },
49
+ "action_dit_config": {
50
+ "action_dim": 14,
51
+ "hidden_dim": 1024,
52
+ "ffn_dim": 4096,
53
+ "num_heads": 24,
54
+ "attn_head_dim": 128,
55
+ "num_layers": 30,
56
+ "text_dim": 4096,
57
+ "freq_dim": 256,
58
+ "eps": 1e-06,
59
+ "use_gradient_checkpointing": false
60
+ },
61
+ "video_scheduler": {
62
+ "train_shift": 5.0,
63
+ "infer_shift": 5.0,
64
+ "num_train_timesteps": 1000
65
+ },
66
+ "action_scheduler": {
67
+ "train_shift": 1.0,
68
+ "infer_shift": 1.0,
69
+ "num_train_timesteps": 1000
70
+ },
71
+ "loss": {
72
+ "lambda_action": 1.0
73
+ }
74
+ },
75
+ "action_semantics": "absolute_original_motion_target_position",
76
+ "joint_order": "left_arm6,left_gripper,right_arm6,right_gripper",
77
+ "default_denoising_steps": 20
78
+ }
baseline/fastwam_zipper/inference_manifest.json ADDED
@@ -0,0 +1,539 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "files": [
3
+ {
4
+ "path": "step_016380.pt",
5
+ "bytes": 12041813369,
6
+ "sha256": "5e574c8cc7bb5747852cd815cfed9896d8d0f1db83eee28a367bd21beb25f348"
7
+ },
8
+ {
9
+ "path": "README.md",
10
+ "bytes": 3521,
11
+ "sha256": "0fe1530fc1195d0f4ba393fb747a97485d762aefc4f26831ff7ae709ddceb641"
12
+ },
13
+ {
14
+ "path": "THIRD_PARTY.md",
15
+ "bytes": 497,
16
+ "sha256": "085e845844811f3cbf63a07766841a2af78ea80f337753353af868a1a243fc55"
17
+ },
18
+ {
19
+ "path": "context.pt",
20
+ "bytes": 1050977,
21
+ "sha256": "24c9c95ba634779ba7aeb33743ee38b3cf688805471889400a2e841fd4ea4ad1"
22
+ },
23
+ {
24
+ "path": "inference.py",
25
+ "bytes": 6194,
26
+ "sha256": "12e401a2400cc7fdb54560b08c30a1401ef3680261a7bee6fd5912443dadb961"
27
+ },
28
+ {
29
+ "path": "inference_config.json",
30
+ "bytes": 2206,
31
+ "sha256": "e207f5427b3c9a25fc9f1d61074138c6c04dd717f1d253794d2771345cd69f8f"
32
+ },
33
+ {
34
+ "path": "models/LICENSE-WAN-APACHE-2.0.txt",
35
+ "bytes": 11358,
36
+ "sha256": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30"
37
+ },
38
+ {
39
+ "path": "models/Wan2.2-TI2V-5B/Wan2.2_VAE.pth",
40
+ "bytes": 2818839170,
41
+ "sha256": "20eb789667fa5e60e7516bf509512f6cb61f01b0aa0695eadaea930c13892b36"
42
+ },
43
+ {
44
+ "path": "norm_stats.json",
45
+ "bytes": 3306,
46
+ "sha256": "10a5eed0657164c56f13efb0d99434824b201f07e450263f564c828944aa5231"
47
+ },
48
+ {
49
+ "path": "provenance.json",
50
+ "bytes": 563,
51
+ "sha256": "3858af2efaf9c8646d791137a5738db3cb082c7bcb74d9c6f92c2ba5b9573c9a"
52
+ },
53
+ {
54
+ "path": "requirements-inference.txt",
55
+ "bytes": 196,
56
+ "sha256": "87f014a6694ed6b262d730652c68878ea4297a420c6309711346891920a0d4dc"
57
+ },
58
+ {
59
+ "path": "source/LICENSE",
60
+ "bytes": 1210,
61
+ "sha256": "8379dddfbbb1154ad9cc57f008f1f8d4a8d26c3de99bd6b36aa4d8067888aa92"
62
+ },
63
+ {
64
+ "path": "source/README.md",
65
+ "bytes": 15889,
66
+ "sha256": "1fbad9ad9407d54a224864a10ec9576fa731e4a8f291bc9a7e28f545647fef2a"
67
+ },
68
+ {
69
+ "path": "source/configs/data/galaxea_r1_lite_3cam.yaml",
70
+ "bytes": 3145,
71
+ "sha256": "e83ab2ece05ccea8a352404a7cdd38b361325ecf720f64a79e94c6dccb36cc57"
72
+ },
73
+ {
74
+ "path": "source/configs/data/libero_2cam.yaml",
75
+ "bytes": 2448,
76
+ "sha256": "d2ba368bd906992a2f3ca6229da9a5abd58163e2e64a8c4569d012efef039008"
77
+ },
78
+ {
79
+ "path": "source/configs/data/libero_2cam_lerobot_v30.yaml",
80
+ "bytes": 1935,
81
+ "sha256": "9c243358de9e978f210d827569cf8bf074fec6b7a3d47b079e8827342562edc1"
82
+ },
83
+ {
84
+ "path": "source/configs/data/robotwin.yaml",
85
+ "bytes": 4110,
86
+ "sha256": "f1ba509fac21fc59d5a210d6f68bcf94640729ce00dce695b7ca6df24c7b5aa7"
87
+ },
88
+ {
89
+ "path": "source/configs/model/fastwam.yaml",
90
+ "bytes": 1536,
91
+ "sha256": "0c9378106366d35f99ea3a82ef30afdfbab9a1d330af3e6de6c94c4490d24868"
92
+ },
93
+ {
94
+ "path": "source/configs/model/fastwam_idm.yaml",
95
+ "bytes": 1567,
96
+ "sha256": "7c95ab549c677ade48c7664beb85202aae15ad8c86dba99fbe4b14f2047aeb76"
97
+ },
98
+ {
99
+ "path": "source/configs/model/fastwam_joint.yaml",
100
+ "bytes": 1510,
101
+ "sha256": "ab2a243f7452478b6ba779cc2507e08d503874e7e7ae6ddcc3f081055b0730ad"
102
+ },
103
+ {
104
+ "path": "source/configs/model/fastwam_optional_idm.yaml",
105
+ "bytes": 1607,
106
+ "sha256": "f5c01951f483d453f7a0ef01bc7026eb8c2922b32361230a99e0d09d898904db"
107
+ },
108
+ {
109
+ "path": "source/configs/sim_libero.yaml",
110
+ "bytes": 1082,
111
+ "sha256": "005c4da6d40abc2304ff6f9a06feca1204f330c1ac7c3919f807d1b58dfd016c"
112
+ },
113
+ {
114
+ "path": "source/configs/sim_robotwin.yaml",
115
+ "bytes": 975,
116
+ "sha256": "b96b579a8f243c935f1e17f8f3d6e9b7cb6bf2f2a6823272f626ecc33d68056f"
117
+ },
118
+ {
119
+ "path": "source/configs/task/galaxea_r1_lite_3cam_1e-4.yaml",
120
+ "bytes": 1263,
121
+ "sha256": "6553c8c22fafb7b3a563eb81fb310bf1b36d6b4d5ba7451c89b63d722a2543be"
122
+ },
123
+ {
124
+ "path": "source/configs/task/libero_idm_2cam224_1e-4.yaml",
125
+ "bytes": 409,
126
+ "sha256": "5df6cf62843806b0d9d2fb48b873945865bbfa74d135c58032651d54d6992c4e"
127
+ },
128
+ {
129
+ "path": "source/configs/task/libero_joint_2cam224_1e-4.yaml",
130
+ "bytes": 411,
131
+ "sha256": "1c46f34058c9c5363bab6814a0b86eb13be3a0077e944db7d39cf151da07636e"
132
+ },
133
+ {
134
+ "path": "source/configs/task/libero_optional_idm_2cam224_1e-4.yaml",
135
+ "bytes": 381,
136
+ "sha256": "367794491e728cba5fdb63929e69ab19f3c75b89eb6631754ae6a67d114d542c"
137
+ },
138
+ {
139
+ "path": "source/configs/task/libero_uncond_2cam224_1e-4.yaml",
140
+ "bytes": 405,
141
+ "sha256": "27bdc30545aa06bca81137690a241d469d5a67a433881976bd79da3d4171d1a0"
142
+ },
143
+ {
144
+ "path": "source/configs/task/robotwin_idm_3cam_384_1e-4.yaml",
145
+ "bytes": 405,
146
+ "sha256": "2e67c9f301105752962af825e57e5b46ef5e6ba4290e397e5efc7abb88652619"
147
+ },
148
+ {
149
+ "path": "source/configs/task/robotwin_joint_3cam_384_1e-4.yaml",
150
+ "bytes": 407,
151
+ "sha256": "daf4cb5246728ba60ae6903fbfd9d5e029d7172f30916b8143f81783450fa874"
152
+ },
153
+ {
154
+ "path": "source/configs/task/robotwin_uncond_3cam_384_1e-4.yaml",
155
+ "bytes": 401,
156
+ "sha256": "2f78a207415b452b05e0238f3e00eb9942d7960413076b5b3cdcf1ce09caef9f"
157
+ },
158
+ {
159
+ "path": "source/configs/train.yaml",
160
+ "bytes": 697,
161
+ "sha256": "42a60765bfdd842af01aacdde6066ce63be00698377fe7729110b795be684016"
162
+ },
163
+ {
164
+ "path": "source/pyproject.toml",
165
+ "bytes": 1049,
166
+ "sha256": "161e4be4e5a0c3f9888aa7b8b8416c45441eabf0aed4374ac193a08de02c043a"
167
+ },
168
+ {
169
+ "path": "source/scripts/__init__.py",
170
+ "bytes": 37,
171
+ "sha256": "5a71e51c716ea7579151f36996bebb1ff82c6fedd0c45c68f2918cf5812f1053"
172
+ },
173
+ {
174
+ "path": "source/scripts/accelerate_configs/accelerate_zero0.yaml",
175
+ "bytes": 489,
176
+ "sha256": "dacdf1cee3675246567f1f61e943c6762a4700b8b40c888af8fdf89773adc35b"
177
+ },
178
+ {
179
+ "path": "source/scripts/accelerate_configs/accelerate_zero1_ds.yaml",
180
+ "bytes": 496,
181
+ "sha256": "5b9e24f38c199fdff6bbd17981d86e2624f855f707660094c99f711efc8e0833"
182
+ },
183
+ {
184
+ "path": "source/scripts/accelerate_configs/accelerate_zero2_ds.yaml",
185
+ "bytes": 496,
186
+ "sha256": "3b838fbf44cd279613797e9ac3265f306806391963be593cb47787d38240a2e1"
187
+ },
188
+ {
189
+ "path": "source/scripts/check_galaxea_r1_lite_contract.py",
190
+ "bytes": 3820,
191
+ "sha256": "585200c8bea8e37a832381a00ec86cdb0fc3176840b35cd0df26e6700819c522"
192
+ },
193
+ {
194
+ "path": "source/scripts/check_galaxea_text_cache.py",
195
+ "bytes": 1749,
196
+ "sha256": "2ef7e6dcfc821b8a65ad75ca1af3e70d58100a2e129105d9dd0fcf389e1c7689"
197
+ },
198
+ {
199
+ "path": "source/scripts/dryrun_fastwam.py",
200
+ "bytes": 7168,
201
+ "sha256": "102e89470ae21392ab0e868f4df20eef19ee902cecebf4c0fcf9d937d6c29c50"
202
+ },
203
+ {
204
+ "path": "source/scripts/ds_configs/ds_zero1_config.json",
205
+ "bytes": 400,
206
+ "sha256": "8b65ebcb8de8c3c89113695a61783cdb97c2aa5003477f0ca743aa511bc68312"
207
+ },
208
+ {
209
+ "path": "source/scripts/ds_configs/ds_zero2_config.json",
210
+ "bytes": 428,
211
+ "sha256": "93928339afe3946e603128ed42aae1f44279f694e20be911527e6e02ab4f9c72"
212
+ },
213
+ {
214
+ "path": "source/scripts/precompute_text_embeds.py",
215
+ "bytes": 14454,
216
+ "sha256": "6be7ade7c56287ee640683a2dceedaf0bb5fc1b783802ca6a81fe27a53017edc"
217
+ },
218
+ {
219
+ "path": "source/scripts/prepare_galaxea_r1_lite_overlay.py",
220
+ "bytes": 5255,
221
+ "sha256": "dcab484f9105cde123d2b240d764082a7c165e2fc73af1948039bb837c51726d"
222
+ },
223
+ {
224
+ "path": "source/scripts/preprocess_action_dit_backbone.py",
225
+ "bytes": 9948,
226
+ "sha256": "0fb11f04756d71e6b1b1d26cdf9e5831dfead339da38aa25c02f171d7f484a94"
227
+ },
228
+ {
229
+ "path": "source/scripts/train.py",
230
+ "bytes": 358,
231
+ "sha256": "9fb7bff4bcc0eb47294b6815c547e8cf47c02532da4d58bcb072e1f18ba67ef8"
232
+ },
233
+ {
234
+ "path": "source/scripts/train_galaxea_r1_lite.sh",
235
+ "bytes": 6889,
236
+ "sha256": "d9e6ec7442afb4346d3a53ea97553262001fe84bed21c59a00a1c8a1e1dc912a"
237
+ },
238
+ {
239
+ "path": "source/scripts/train_zero1.sh",
240
+ "bytes": 3241,
241
+ "sha256": "0ee0031b65193953ab2c449201f9ff64574d93d7956a9d5fd6644317bdaf7654"
242
+ },
243
+ {
244
+ "path": "source/scripts/train_zero2.sh",
245
+ "bytes": 3235,
246
+ "sha256": "b75455a0c6519b6b7339ab6a3b4c8e6b6aa882c599a6029aff2d349620bdf43a"
247
+ },
248
+ {
249
+ "path": "source/src/fastwam/__init__.py",
250
+ "bytes": 23,
251
+ "sha256": "c4149c5ff0f9abd1517360daf761e692feb8570eb7a0caa2dfde52d4412c3e23"
252
+ },
253
+ {
254
+ "path": "source/src/fastwam/datasets/__init__.py",
255
+ "bytes": 0,
256
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
257
+ },
258
+ {
259
+ "path": "source/src/fastwam/datasets/dataset_utils.py",
260
+ "bytes": 3735,
261
+ "sha256": "3bc17590049c7cb54061acf8d731b07892e7f530c5d7eeac2613fa77a285a392"
262
+ },
263
+ {
264
+ "path": "source/src/fastwam/datasets/lerobot/base_lerobot_dataset.py",
265
+ "bytes": 19162,
266
+ "sha256": "c076b4028c5414d3c08199f1501698c3a4e15ac4fe09c6c7a67fa7f462a13b4a"
267
+ },
268
+ {
269
+ "path": "source/src/fastwam/datasets/lerobot/constants.py",
270
+ "bytes": 1915,
271
+ "sha256": "11ac9260341e1560c5eb0dd7acb79ed63c74180b8ab4113c27622ff213db417f"
272
+ },
273
+ {
274
+ "path": "source/src/fastwam/datasets/lerobot/lerobot/datasets/compute_stats.py",
275
+ "bytes": 7067,
276
+ "sha256": "7d7daab219332b5abdfd5ec67ccaf3502e8eb583a17a4e482c3f3d4f7a691e89"
277
+ },
278
+ {
279
+ "path": "source/src/fastwam/datasets/lerobot/lerobot/datasets/utils.py",
280
+ "bytes": 33439,
281
+ "sha256": "40f1b94e6ceb50765fb8dfa3d498887c2a0fc2eb49f09770c5f36190e7110c98"
282
+ },
283
+ {
284
+ "path": "source/src/fastwam/datasets/lerobot/lerobot/datasets/video_utils.py",
285
+ "bytes": 19421,
286
+ "sha256": "cb647a1e955c3279e39ce8d7af7a37ea7432bbd71534c183f9ceebc8557dd89f"
287
+ },
288
+ {
289
+ "path": "source/src/fastwam/datasets/lerobot/lerobot/lerobot_dataset.py",
290
+ "bytes": 55233,
291
+ "sha256": "a71b26a73a0e460198b8316db2acb77b43a719d4ab9706150e2ba53773ffda64"
292
+ },
293
+ {
294
+ "path": "source/src/fastwam/datasets/lerobot/processors/base_processor.py",
295
+ "bytes": 10811,
296
+ "sha256": "1f47bc85fee11e3d16714b2f1e3a2dc874f4609a1706a788617156d34b48b58e"
297
+ },
298
+ {
299
+ "path": "source/src/fastwam/datasets/lerobot/processors/fastwam_processor.py",
300
+ "bytes": 13458,
301
+ "sha256": "3aa0e275b3815b67785c6e38ac5fc400472f9d09ac18b2ab269ea77bf574ade4"
302
+ },
303
+ {
304
+ "path": "source/src/fastwam/datasets/lerobot/robot_video_dataset.py",
305
+ "bytes": 13058,
306
+ "sha256": "04f830e4c7dcc93be8e1d3514e502719aed94e62074364edbc1257e533a27de9"
307
+ },
308
+ {
309
+ "path": "source/src/fastwam/datasets/lerobot/transforms/action_state_merger.py",
310
+ "bytes": 2522,
311
+ "sha256": "669d6991415fc5eca44b32b129d15fee1e06e2f17d59d90350c252598033b427"
312
+ },
313
+ {
314
+ "path": "source/src/fastwam/datasets/lerobot/transforms/image.py",
315
+ "bytes": 725,
316
+ "sha256": "4d7ab61d0844918525244db653831f0f632730046b2714a6163ff57c441a16fa"
317
+ },
318
+ {
319
+ "path": "source/src/fastwam/datasets/lerobot/transforms/misc.py",
320
+ "bytes": 422,
321
+ "sha256": "8b505bd9e3a54ab2db7efcd64f0929d2e0303f5db1123de1074fd01fa9887e2b"
322
+ },
323
+ {
324
+ "path": "source/src/fastwam/datasets/lerobot/transforms/relative_action.py",
325
+ "bytes": 5736,
326
+ "sha256": "2b4f05297de6cd364e507b5874a752003a9dfb770a528ea73d2781c8a1a12059"
327
+ },
328
+ {
329
+ "path": "source/src/fastwam/datasets/lerobot/transforms/rotation.py",
330
+ "bytes": 5603,
331
+ "sha256": "0de34b252d60b956cda07c0c19baebbd3c8a28916face700c6664c5815e11e91"
332
+ },
333
+ {
334
+ "path": "source/src/fastwam/datasets/lerobot/utils/normalizer.py",
335
+ "bytes": 11274,
336
+ "sha256": "5ac89a646cd6395e98feaa832f8fdf7c7e00857080af3d9c15710e243c43ad43"
337
+ },
338
+ {
339
+ "path": "source/src/fastwam/datasets/lerobot/utils/rotation.py",
340
+ "bytes": 21110,
341
+ "sha256": "6169162f98faf02a3fe5201e2b8a1069fe299840d036ee7cad8af5fa8f561542"
342
+ },
343
+ {
344
+ "path": "source/src/fastwam/datasets/lerobot3/__init__.py",
345
+ "bytes": 389,
346
+ "sha256": "26f0c3a4afdbd54f723cab81d2288781844e61ee367fd88bc0a821c7929cb79f"
347
+ },
348
+ {
349
+ "path": "source/src/fastwam/datasets/lerobot3/base_lerobot_dataset.py",
350
+ "bytes": 389,
351
+ "sha256": "7dc96f86caf8239795a0dce0161e25969ac002ebf037dfb9695608c40d89d769"
352
+ },
353
+ {
354
+ "path": "source/src/fastwam/datasets/lerobot3/lerobot_dataset.py",
355
+ "bytes": 14998,
356
+ "sha256": "4acae53b736c47c1ed6cb42ad41a68cb0a64cf6578791c5f9e50a8dd9f5541e1"
357
+ },
358
+ {
359
+ "path": "source/src/fastwam/datasets/lerobot3/robot_video_dataset.py",
360
+ "bytes": 310,
361
+ "sha256": "62230146222dfe0719c43f72d9f8e913cd4998d8f68a3f9d09b9b8d3db7441d5"
362
+ },
363
+ {
364
+ "path": "source/src/fastwam/models/__init__.py",
365
+ "bytes": 35,
366
+ "sha256": "ae4c5733f4bbf7caf31df2ae6f789516f6a10cd84a321f7d0375a7034253ac6d"
367
+ },
368
+ {
369
+ "path": "source/src/fastwam/models/wan22/__init__.py",
370
+ "bytes": 0,
371
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
372
+ },
373
+ {
374
+ "path": "source/src/fastwam/models/wan22/action_dit.py",
375
+ "bytes": 14329,
376
+ "sha256": "631fdf598fbe06a95a41c1a27acd9de16f2187f3a2009e10690197527d18283b"
377
+ },
378
+ {
379
+ "path": "source/src/fastwam/models/wan22/fastwam.py",
380
+ "bytes": 54376,
381
+ "sha256": "154c339efd33fae119fe902d6efb91cde7475e3cfd0e7b50e606278c1c3eac09"
382
+ },
383
+ {
384
+ "path": "source/src/fastwam/models/wan22/fastwam_idm.py",
385
+ "bytes": 25404,
386
+ "sha256": "5ebdaabb42d4e6c0c6543760ad2ea9b3c8ef9415ebab44fbf845f73f52280bba"
387
+ },
388
+ {
389
+ "path": "source/src/fastwam/models/wan22/fastwam_joint.py",
390
+ "bytes": 10242,
391
+ "sha256": "a61aab79cfc607af02ab816849329a13c1dcd0bbcbf63c6584a345d8a67bb2b6"
392
+ },
393
+ {
394
+ "path": "source/src/fastwam/models/wan22/fastwam_optional_idm.py",
395
+ "bytes": 4420,
396
+ "sha256": "bd508293545895bae159a42998992cece9c1f1aca373e6f03521ea05f55749b2"
397
+ },
398
+ {
399
+ "path": "source/src/fastwam/models/wan22/helpers/__init__.py",
400
+ "bytes": 415,
401
+ "sha256": "f26116de229557a37a4812dc4ee75971f5f9e8b80240786ea4d0e5b0ff703cbf"
402
+ },
403
+ {
404
+ "path": "source/src/fastwam/models/wan22/helpers/gradient.py",
405
+ "bytes": 565,
406
+ "sha256": "ba5d8f7272eb029dc6cd2849ca99b70f6ad5abb838d21c818beb0590620dc793"
407
+ },
408
+ {
409
+ "path": "source/src/fastwam/models/wan22/helpers/io.py",
410
+ "bytes": 7751,
411
+ "sha256": "1c6c1bc8a975b47092912833194fce25e2e3ac886fbab72d73af3ff5a00b3c62"
412
+ },
413
+ {
414
+ "path": "source/src/fastwam/models/wan22/helpers/loader.py",
415
+ "bytes": 8279,
416
+ "sha256": "792f66bd0609c1365256aa24d43301340f81260c1651d89b04d2ae8224a08726"
417
+ },
418
+ {
419
+ "path": "source/src/fastwam/models/wan22/helpers/state_dict_converters.py",
420
+ "bytes": 5226,
421
+ "sha256": "81d213373e5056a8d8bf31768b9b04164138ea78d0b306e316334841499f4bf9"
422
+ },
423
+ {
424
+ "path": "source/src/fastwam/models/wan22/mot.py",
425
+ "bytes": 31234,
426
+ "sha256": "2383e4cfa00efa9cde889493e0b6c861bc7b17d24d731f5ff38d3ebf8ab7f541"
427
+ },
428
+ {
429
+ "path": "source/src/fastwam/models/wan22/schedulers/scheduler_continuous.py",
430
+ "bytes": 4068,
431
+ "sha256": "2ececce284f7b9fe876965fbf5e11ca7b69caaf693979266b0f63062a589f181"
432
+ },
433
+ {
434
+ "path": "source/src/fastwam/models/wan22/wan22.py",
435
+ "bytes": 16925,
436
+ "sha256": "b85349beeeb018d5be3f5b932bee3d640274d09c5e1e0fa0fa01e8ad0e02356e"
437
+ },
438
+ {
439
+ "path": "source/src/fastwam/models/wan22/wan_video_dit.py",
440
+ "bytes": 34764,
441
+ "sha256": "578712e83fac2815d3e3a3fd9134178028082deba361433122fe3c5764982dd6"
442
+ },
443
+ {
444
+ "path": "source/src/fastwam/models/wan22/wan_video_text_encoder.py",
445
+ "bytes": 11459,
446
+ "sha256": "aedf7f131556a39b99a273deec3a85d67f41b5367cef41f310f33a807ae60a87"
447
+ },
448
+ {
449
+ "path": "source/src/fastwam/models/wan22/wan_video_vae.py",
450
+ "bytes": 49604,
451
+ "sha256": "cb2b1b53f35a1a201c640ec52d12a7c4cb4ba1df322155addd3dc409e04dc07c"
452
+ },
453
+ {
454
+ "path": "source/src/fastwam/runtime.py",
455
+ "bytes": 22779,
456
+ "sha256": "230a814b7a42a3f74145513f5775a3c6cd56ffa244635c536b3d1f384c5f82d0"
457
+ },
458
+ {
459
+ "path": "source/src/fastwam/trainer.py",
460
+ "bytes": 37779,
461
+ "sha256": "00f9a8b7c7c222fe8572ec2a8f26ffe5fc5779a3fe07afd77369deab6253ddd3"
462
+ },
463
+ {
464
+ "path": "source/src/fastwam/utils/__init__.py",
465
+ "bytes": 96,
466
+ "sha256": "2893c05b99b17162941c70a41859c2808f8ae390be367a26cac7d5ab93ee197b"
467
+ },
468
+ {
469
+ "path": "source/src/fastwam/utils/config_resolvers.py",
470
+ "bytes": 2526,
471
+ "sha256": "01c48cbd42b86e636b70ccc3e193a81147162ba890777102b2c61086fa415340"
472
+ },
473
+ {
474
+ "path": "source/src/fastwam/utils/fs.py",
475
+ "bytes": 76,
476
+ "sha256": "69e6df236b43360428a613686c8f37c531c0d16a3a2c5b06a725c46b49d657c5"
477
+ },
478
+ {
479
+ "path": "source/src/fastwam/utils/logging_config.py",
480
+ "bytes": 4679,
481
+ "sha256": "cadc4f6ad487c3f37171a6defad36d23d9ab8bf262f73a461cd9ee9b03b757e7"
482
+ },
483
+ {
484
+ "path": "source/src/fastwam/utils/misc.py",
485
+ "bytes": 1223,
486
+ "sha256": "05aa51247936a8d3faac27f90b78b65ff83a688259bed55d114b6a617ced532d"
487
+ },
488
+ {
489
+ "path": "source/src/fastwam/utils/pytorch_utils.py",
490
+ "bytes": 5051,
491
+ "sha256": "24de921fe7305b51f94222f4163c6f81eb5e077e46272314d83eb44f9f4bde71"
492
+ },
493
+ {
494
+ "path": "source/src/fastwam/utils/samplers.py",
495
+ "bytes": 1339,
496
+ "sha256": "4d3bd9b2d498f4ad8f2cf9765d60c4bf5368fa6fa2a7d42d0db6210f66899652"
497
+ },
498
+ {
499
+ "path": "source/src/fastwam/utils/video_io.py",
500
+ "bytes": 819,
501
+ "sha256": "9e9898b021a44e87aa1b4e6d13e3755a9869a2e27a7eb53c72a6802808d5bd80"
502
+ },
503
+ {
504
+ "path": "source/src/fastwam/utils/video_metrics.py",
505
+ "bytes": 3446,
506
+ "sha256": "aaa37e458cc9e4bb79a17923b5ff5900c97b4309f7abd50db44cf76e9143ffd6"
507
+ },
508
+ {
509
+ "path": "source/src/fastwam.egg-info/PKG-INFO",
510
+ "bytes": 1161,
511
+ "sha256": "e129d204188c26725819c3994455d55762913c8345b8ae1b2fdabc469a1c3420"
512
+ },
513
+ {
514
+ "path": "source/src/fastwam.egg-info/SOURCES.txt",
515
+ "bytes": 2462,
516
+ "sha256": "dbb1f8a6580463d20f8bd783f10ba5b7a29e020ea86832f976691b855f250105"
517
+ },
518
+ {
519
+ "path": "source/src/fastwam.egg-info/dependency_links.txt",
520
+ "bytes": 1,
521
+ "sha256": "01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b"
522
+ },
523
+ {
524
+ "path": "source/src/fastwam.egg-info/requires.txt",
525
+ "bytes": 521,
526
+ "sha256": "c82af4bdebdaa050271d8eecd4aca96dede56d6a42bb2cf6db0372023503a1c8"
527
+ },
528
+ {
529
+ "path": "source/src/fastwam.egg-info/top_level.txt",
530
+ "bytes": 8,
531
+ "sha256": "8a42966a9145ab062768faad7503d31d40a654dcd024e8a8fb1885fe4bddeedb"
532
+ },
533
+ {
534
+ "path": "validation.json",
535
+ "bytes": 700,
536
+ "sha256": "95d3f67742d93369c73ed436ad412dab078ec8094d7dab2330a4a0f0cca1bc9c"
537
+ }
538
+ ]
539
+ }
baseline/fastwam_zipper/models/LICENSE-WAN-APACHE-2.0.txt ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
baseline/fastwam_zipper/models/Wan2.2-TI2V-5B/Wan2.2_VAE.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20eb789667fa5e60e7516bf509512f6cb61f01b0aa0695eadaea930c13892b36
3
+ size 2818839170
baseline/fastwam_zipper/norm_stats.json ADDED
@@ -0,0 +1,136 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "norm_stats": {
3
+ "state": {
4
+ "mean": [
5
+ -0.32184191477392804,
6
+ 1.5207721980826827,
7
+ -0.7150242090298449,
8
+ -0.28362794490829607,
9
+ -0.0326558641526937,
10
+ -0.013098741542747098,
11
+ 64.66151715104193,
12
+ 0.21104983370396177,
13
+ 1.7986193640856323,
14
+ -1.03603124575021,
15
+ -0.2070677500803398,
16
+ 0.49687352638322985,
17
+ 0.6771545717346843,
18
+ 62.77788487973063
19
+ ],
20
+ "std": [
21
+ 0.3021247081483893,
22
+ 1.0365802698172673,
23
+ 0.6522747017660155,
24
+ 0.4776692517096447,
25
+ 0.17428160840573853,
26
+ 0.21532609078838216,
27
+ 43.57897586492507,
28
+ 0.1853564774548537,
29
+ 1.0606944989455414,
30
+ 0.7407364588573183,
31
+ 0.45290236233056375,
32
+ 0.41556147349584566,
33
+ 0.6096365556465444,
34
+ 43.38786400775888
35
+ ],
36
+ "q01": [
37
+ -0.9572340250015259,
38
+ 0.2140425592660904,
39
+ -2.039149045944214,
40
+ -1.4893616437911987,
41
+ -0.5359574556350708,
42
+ -0.7127659320831299,
43
+ 3.0947201251983643,
44
+ -0.0908510610461235,
45
+ 0.21531914174556732,
46
+ -2.0451064109802246,
47
+ -1.52042555809021,
48
+ -0.1923404186964035,
49
+ -0.08297872543334961,
50
+ 3.2300286293029785
51
+ ],
52
+ "q99": [
53
+ 0.10489361733198166,
54
+ 3.0346808433532715,
55
+ -0.0019148936262354255,
56
+ 0.6214893460273743,
57
+ 0.31340426206588745,
58
+ 0.7790850949287501,
59
+ 100.67488861083984,
60
+ 0.6489361524581909,
61
+ 3.085106372833252,
62
+ -0.001276595750823617,
63
+ 0.7149106299877201,
64
+ 1.212978720664978,
65
+ 1.943404197692871,
66
+ 101.49044036865234
67
+ ]
68
+ },
69
+ "actions": {
70
+ "mean": [
71
+ -0.3302394307611646,
72
+ 1.3351088878676973,
73
+ -0.730087379438977,
74
+ -0.3051273035432853,
75
+ -0.05725393898553614,
76
+ -0.03795729405196406,
77
+ 63.83399157797273,
78
+ 0.1917094052418155,
79
+ 1.6079172755673306,
80
+ -1.0522424038321934,
81
+ -0.22567802180755175,
82
+ 0.4700642734741484,
83
+ 0.6497894015838603,
84
+ 59.816608312332825
85
+ ],
86
+ "std": [
87
+ 0.2976434695253568,
88
+ 1.074422013023125,
89
+ 0.6661020136243821,
90
+ 0.4836939282841582,
91
+ 0.17474814803096714,
92
+ 0.2161063544717659,
93
+ 47.573759196581726,
94
+ 0.18222145953683355,
95
+ 1.1005178847180743,
96
+ 0.7551834604102067,
97
+ 0.4582503537690994,
98
+ 0.4172776054469184,
99
+ 0.6124397993031185,
100
+ 48.446447570698396
101
+ ],
102
+ "q01": [
103
+ -0.9526020884513855,
104
+ 0.0,
105
+ -2.0,
106
+ -1.5201749801635742,
107
+ -0.5614369511604309,
108
+ -0.7378447651863098,
109
+ 0.0,
110
+ -0.12118448317050934,
111
+ 0.0,
112
+ -2.0,
113
+ -1.5707999467849731,
114
+ -0.22856314480304718,
115
+ -0.13499030470848083,
116
+ 0.0
117
+ ],
118
+ "q99": [
119
+ 0.10124272853136063,
120
+ 3.0096702575683594,
121
+ 0.0,
122
+ 0.607456386089325,
123
+ 0.29452431201934814,
124
+ 0.754718542098999,
125
+ 100.0,
126
+ 0.6181942820549011,
127
+ 3.0142722129821777,
128
+ 0.0,
129
+ 0.7148350477218628,
130
+ 1.1888351440429688,
131
+ 1.9190099239349365,
132
+ 100.27293395996094
133
+ ]
134
+ }
135
+ }
136
+ }
baseline/fastwam_zipper/provenance.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "training_job": "bifrost-2026102616142400-shouqx",
3
+ "task": "zipper_pulling",
4
+ "final_step": 16380,
5
+ "epochs": 30,
6
+ "filename": "step_016380.pt",
7
+ "bytes": 12041813369,
8
+ "sha256": "5e574c8cc7bb5747852cd815cfed9896d8d0f1db83eee28a367bd21beb25f348",
9
+ "checkpoint_keys": [
10
+ "mot",
11
+ "step",
12
+ "torch_dtype",
13
+ "proprio_encoder"
14
+ ],
15
+ "torch_dtype": "torch.bfloat16",
16
+ "tensor_elements": 6020749518,
17
+ "training_status": "JOB_COMPLETE",
18
+ "uploaded_as": "original unchanged final weights",
19
+ "verified_utc": "2026-10-27T09:34:00.717440+00:00"
20
+ }
baseline/fastwam_zipper/requirements-inference.txt ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ torch==2.7.1
2
+ torchvision==0.22.1
3
+ numpy==2.2.6
4
+ pillow==12.0.0
5
+ einops==0.8.1
6
+ safetensors==0.5.3
7
+ transformers==4.49.0
8
+ regex==2025.11.3
9
+ imageio==2.37.0
10
+ imageio-ffmpeg==0.6.0
11
+ tqdm==4.68.3
12
+ rich==14.2.0
baseline/fastwam_zipper/source/LICENSE ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 The FastWAM Authors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ Except for third-party code and files that retain their original license
16
+ headers, this repository is released under the MIT License.
17
+
18
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
19
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
20
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
21
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24
+ SOFTWARE.
baseline/fastwam_zipper/source/README.md ADDED
@@ -0,0 +1,437 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # FastWAM
2
+
3
+ Official codebase for **Fast-WAM: Do World Action Models Need Test-time Future Imagination?**
4
+
5
+ [![English](https://img.shields.io/badge/README-English-111111.svg)](./README.md)
6
+ [![中文](https://img.shields.io/badge/README-%E4%B8%AD%E6%96%87-d14836.svg)](./README_zh.md)
7
+
8
+ [![arXiv](https://img.shields.io/badge/arXiv-2603.16666-b31b1b.svg)](https://arxiv.org/abs/2603.16666)
9
+ [![Project Page](https://img.shields.io/badge/Project_Page-Fast--WAM-2ea44f.svg)](https://yuantianyuan01.github.io/FastWAM/)
10
+ [![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-f7c843)](https://huggingface.co/yuanty/fastwam)
11
+ [![Hugging Face Dataset - LIBERO](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset%20LIBERO-f7c843)](https://huggingface.co/datasets/yuanty/LIBERO-fastwam)
12
+ [![Hugging Face Dataset - RoboTwin](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset%20RoboTwin-f7c843)](https://huggingface.co/datasets/yuanty/robotwin2.0-fastwam)
13
+
14
+ This repository contains the training and evaluation code for FastWAM on LIBERO / RoboTwin.
15
+
16
+ ## What's New
17
+
18
+ FastWAM is now faster, better suited to large-scale datasets, and more flexible
19
+ for research. This update brings substantially faster training and inference,
20
+ native LeRobot v3.0 support, and a new model that can switch between acting with
21
+ and without future imagination.
22
+
23
+ ### ⚡ Approximately 2x faster inference
24
+
25
+ End-to-end FastWAM inference is now approximately **2x faster**, including text
26
+ encoding and VAE encoding:
27
+
28
+ - **NVIDIA H20:** 470 ms → 210 ms
29
+ - **NVIDIA RTX 4090:** 190 ms → 110 ms
30
+
31
+ The accelerated path is enabled by default for LIBERO with
32
+ `EVALUATION.compile_action_infer=true`. We gratefully acknowledge
33
+ [PR #43](https://github.com/yuantianyuan01/FastWAM/pull/43) for proposing the
34
+ optimization ideas that inspired this work. Existing checkpoints remain fully
35
+ compatible with the accelerated inference path.
36
+
37
+ ### 🚀 Approximately 10% faster training
38
+
39
+ FastWAM training is approximately **10% faster on NVIDIA H20 GPUs**. The new
40
+ training path combines a compiled denoising core with batched VAE encoding and
41
+ a lightweight CUDA Graph backend. Enable denoising compilation with:
42
+
43
+ ```bash
44
+ bash scripts/train_zero1.sh 8 task=libero_uncond_2cam224_1e-4 \
45
+ model.compile_training_denoise=true
46
+ ```
47
+
48
+ Both cached text embeddings and on-the-fly T5 encoding are supported. The
49
+ latter skips text-cache preprocessing and is more convenient, at the cost of
50
+ approximately 10% lower training throughput.
51
+
52
+ ### 📦 Native LeRobot 2.1 and 3.0 support
53
+
54
+ FastWAM now supports both **LeRobot 2.1 and LeRobot 3.0** datasets. LeRobot 3.0's
55
+ chunked parquet and video layout scales better to large datasets, with faster
56
+ data loading and dataset-statistics computation as the dataset grows.
57
+
58
+ Download the released LeRobot 3.0 LIBERO dataset from
59
+ [Hugging Face](https://huggingface.co/datasets/yuanty/LIBERO-fastwam) and select
60
+ the v3.0 data config:
61
+
62
+ ```bash
63
+ huggingface-cli download yuanty/LIBERO-fastwam \
64
+ --repo-type dataset \
65
+ --include "lerobot_v30/**" \
66
+ --local-dir ./data
67
+
68
+ python scripts/train.py task=libero_uncond_2cam224_1e-4 \
69
+ data=libero_2cam_lerobot_v30
70
+ ```
71
+
72
+ For another LeRobot 3.0 dataset, copy
73
+ `configs/data/libero_2cam_lerobot_v30.yaml`, update `train.dataset_dirs`, and
74
+ select the new config with `data=<config_name>`. Existing LeRobot 2.1 configs
75
+ continue to work unchanged.
76
+
77
+ ### 🧠 Optional IDM: one model, two thinking modes
78
+
79
+ Optional IDM is a new FastWAM variant that supports **two inference modes in a
80
+ single model**:
81
+
82
+ - **IDM mode:** imagine the future video first, then predict actions.
83
+ - **First-frame mode (Fast-WAM):** skip test-time future imagination and predict
84
+ actions directly from the current observation.
85
+
86
+ Download the released Optional IDM checkpoint from
87
+ [Hugging Face](https://huggingface.co/yuanty/fastwam):
88
+
89
+ ```bash
90
+ huggingface-cli download yuanty/fastwam \
91
+ libero_optional_idm_2cam224.pt \
92
+ libero_optional_idm_2cam224_dataset_stats.json \
93
+ --local-dir ./checkpoints/fastwam_release
94
+ ```
95
+
96
+ Train the optional-IDM variant once:
97
+
98
+ ```bash
99
+ bash scripts/train_zero1.sh 8 task=libero_optional_idm_2cam224_1e-4
100
+ ```
101
+
102
+ Then choose either inference mode at evaluation time without retraining, making
103
+ it easy to study when future imagination helps:
104
+
105
+ ```bash
106
+ python experiments/libero/run_libero_manager.py \
107
+ task=libero_optional_idm_2cam224_1e-4 \
108
+ ckpt=./checkpoints/fastwam_release/libero_optional_idm_2cam224.pt \
109
+ EVALUATION.dataset_stats_path=./checkpoints/fastwam_release/libero_optional_idm_2cam224_dataset_stats.json \
110
+ EVALUATION.sigma_shift=1.0 \
111
+ +EVALUATION.action_infer_mode=idm \
112
+ MULTIRUN.num_gpus=8
113
+ ```
114
+
115
+ Replace `idm` with `first_frame` to use the Fast-WAM inference mode.
116
+
117
+ The released checkpoint, trained with action scheduler shift `1.0`, achieves
118
+ the following success rates on the full LIBERO benchmark (40 tasks, 50 episodes
119
+ per task):
120
+
121
+ | Inference mode | Spatial | Goal | Object | Long | Average |
122
+ | --- | ---: | ---: | ---: | ---: | ---: |
123
+ | IDM | 99.0% | 98.6% | 99.6% | 97.0% | **98.55%** |
124
+ | First-frame (Fast-WAM) | 98.2% | 97.8% | 99.2% | 95.8% | **97.75%** |
125
+
126
+ ### Other improvements
127
+
128
+ - The action scheduler shift now defaults to `1.0` for both training and
129
+ evaluation; shifts from `1.0` to `3.0` perform similarly in our experiments.
130
+ When evaluating the original released checkpoints, set
131
+ `EVALUATION.sigma_shift=5.0` to reproduce the original setting.
132
+ - Upgraded LIBERO evaluation with persistent model workers, dynamic task
133
+ scheduling, bad-GPU quarantine, failure recovery, and resumable results.
134
+ - Optimized action-only inference for IDM and Optional IDM: `infer_action`
135
+ returns actions and video latents directly, while VAE decoding is performed
136
+ only by `infer_joint` when video output is requested, eliminating redundant
137
+ computation in action-only deployments.
138
+
139
+ ## Index
140
+
141
+ - [File Structure](#file-structure)
142
+ - [Environment Setup](#environment-setup)
143
+ - [Model Preparation](#model-preparation)
144
+ - [Dataset Download](#dataset-download)
145
+ - [Inference with Released Checkpoints](#inference-with-released-checkpoints)
146
+ - [Training](#training)
147
+ - [Inference with Your Trained Checkpoints](#inference-with-your-trained-checkpoints)
148
+ - [Acknowledgements](#acknowledgements)
149
+ - [BibTeX](#bibtex)
150
+
151
+ ## File Structure
152
+
153
+ ```text
154
+ FastWAM/
155
+ ├── configs/
156
+ │ ├── data/ # Dataset configs (LIBERO, RoboTwin, etc.)
157
+ │ ├── model/ # Model architecture and component configs
158
+ │ └── task/ # Task-level configs (training task names)
159
+ ├── scripts/
160
+ │ ├── train.py
161
+ │ ├── train_zero1.sh # Deepspeed zero1 training entrypoint
162
+ │ ├── preprocess_action_dit_backbone.py # Preprocess ActionDiT backbone before training
163
+ │ └── precompute_text_embeds.py # Precompute T5 text embedding cache before training
164
+ ├── experiments/
165
+ │ ├── libero/
166
+ │ │ └── run_libero_manager.py
167
+ │ └── robotwin/
168
+ │ └── run_robotwin_manager.py
169
+ ├── src/fastwam/ # Core code
170
+ ├── runs/ # Training outputs (ckpt, logs)
171
+ ├── checkpoints/ # Pretrained or external checkpoints
172
+ ├── data/ # Data directory
173
+ └── evaluate_results/ # Inference / evaluation results
174
+ ```
175
+
176
+ ## Environment Setup
177
+
178
+ ```bash
179
+ conda create -n fastwam python=3.10 -y
180
+ conda activate fastwam
181
+ pip install -U pip
182
+ pip install torch==2.7.1+cu128 torchvision==0.22.1+cu128 --extra-index-url https://download.pytorch.org/whl/cu128
183
+ pip install -e .
184
+ ```
185
+
186
+ ## Model Preparation
187
+
188
+ This step is required before both training and inference.
189
+
190
+ Step 1: set the Wan model directory first (opional, default `./checkpoints`):
191
+
192
+ ```bash
193
+ mkdir -p checkpoints
194
+ export DIFFSYNTH_MODEL_BASE_PATH="$(pwd)/checkpoints"
195
+ ```
196
+
197
+ Step 2: pre-generate the ActionDiT backbone (interpolated from Wan22 DiT):
198
+
199
+ ```bash
200
+ # uncond (fastwam)
201
+ python scripts/preprocess_action_dit_backbone.py \
202
+ --model-config configs/model/fastwam.yaml \
203
+ --output checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt \
204
+ --device cuda \
205
+ --dtype bfloat16
206
+ ```
207
+
208
+ ## Dataset Download
209
+
210
+ ### LIBERO
211
+
212
+ The preprocessed LIBERO dataset used by Fast-WAM is available at:
213
+
214
+ - https://huggingface.co/datasets/yuanty/LIBERO-fastwam
215
+
216
+ Download all compressed files first, then extract them all:
217
+
218
+ ```bash
219
+ mkdir -p data/libero_mujoco3.3.2
220
+ cd data/libero_mujoco3.3.2
221
+
222
+ # Run after downloading all 4 tar.gz files
223
+ for f in *.tar.gz; do
224
+ tar -xzf "$f"
225
+ done
226
+ ```
227
+
228
+ The extracted directory structure should be:
229
+
230
+ ```text
231
+ data/libero_mujoco3.3.2/
232
+ ├── libero_10_no_noops_lerobot/
233
+ ├── libero_goal_no_noops_lerobot/
234
+ ├── libero_object_no_noops_lerobot/
235
+ └── libero_spatial_no_noops_lerobot/
236
+ ```
237
+
238
+ ### RoboTwin
239
+
240
+ The preprocessed RoboTwin dataset used by Fast-WAM is available at:
241
+
242
+ - https://huggingface.co/datasets/yuanty/robotwin2.0-fastwam
243
+
244
+ Download all split archive files first, then concatenate and extract:
245
+
246
+ ```bash
247
+ mkdir -p data/robotwin2.0
248
+ cd data/robotwin2.0
249
+
250
+ # Run after downloading all robotwin2.0.tar.gz.part-* files
251
+ cat robotwin2.0.tar.gz.part-* | tar -xzf -
252
+ ```
253
+
254
+ The extracted directory structure should be:
255
+
256
+ ```text
257
+ data/robotwin2.0/
258
+ └── robotwin2.0/
259
+ ├── data/
260
+ ├── meta/
261
+ └── videos/
262
+ ```
263
+
264
+ If you also keep:
265
+
266
+ ```text
267
+ data/robotwin2.0/dataset_stats.json
268
+ ```
269
+
270
+ in the root directory, it can be used directly as the statistics file for the current configs in this repo. You can also recompute it.
271
+
272
+ ## Inference with Released Checkpoints
273
+
274
+ The released checkpoints and their corresponding dataset stats are available on [Hugging Face](https://huggingface.co/yuanty/fastwam).
275
+
276
+ Optional: download released checkpoints and dataset stats from Hugging Face:
277
+
278
+ ```bash
279
+ pip install -U huggingface_hub
280
+
281
+ huggingface-cli download yuanty/fastwam \
282
+ libero_uncond_2cam224.pt \
283
+ libero_uncond_2cam224_dataset_stats.json \
284
+ libero_optional_idm_2cam224.pt \
285
+ libero_optional_idm_2cam224_dataset_stats.json \
286
+ robotwin_uncond_3cam_384.pt \
287
+ robotwin_uncond_3cam_384_dataset_stats.json \
288
+ --local-dir ./checkpoints/fastwam_release
289
+ ```
290
+
291
+ After downloading, the local directory is expected to contain:
292
+
293
+ ```text
294
+ checkpoints/fastwam_release/
295
+ ├── libero_uncond_2cam224.pt
296
+ ├── libero_uncond_2cam224_dataset_stats.json
297
+ ├── libero_optional_idm_2cam224.pt
298
+ ├── libero_optional_idm_2cam224_dataset_stats.json
299
+ ├── robotwin_uncond_3cam_384.pt
300
+ └── robotwin_uncond_3cam_384_dataset_stats.json
301
+ ```
302
+
303
+ Before running the `LIBERO` benchmark, install the official LIBERO environment first
304
+ from the [LIBERO repository](https://github.com/Lifelong-Robot-Learning/LIBERO).
305
+ Then run this final step:
306
+
307
+ ```bash
308
+ pip install mujoco==3.3.2
309
+ ```
310
+
311
+ The `mujoco` environment should ideally stay consistent with the LIBERO data version.
312
+
313
+ We have already copied the `RoboTwin` evaluation-related code into `third_party/RoboTwin`.
314
+ You still need to follow the official RoboTwin instructions from the
315
+ [RoboTwin repository](https://github.com/RoboTwin-Platform/RoboTwin) to finish environment installation and download the required assets, then create the policy symlink:
316
+
317
+ ```bash
318
+ ln -sfn "$(pwd)/experiments/robotwin/fastwam_policy" "$(pwd)/third_party/RoboTwin/policy/fastwam_policy"
319
+ ```
320
+
321
+ Optional: evaluate released LIBERO checkpoint:
322
+
323
+ The released `LIBERO` / `RoboTwin` evaluation managers default to `8` GPUs
324
+ (`MULTIRUN.num_gpus=8` in `configs/sim_libero.yaml` and `configs/sim_robotwin.yaml`).
325
+ If you want to evaluate with fewer GPUs, pass a smaller value such as
326
+ `MULTIRUN.num_gpus=4`.
327
+
328
+ ```bash
329
+ python experiments/libero/run_libero_manager.py \
330
+ task=libero_uncond_2cam224_1e-4 \
331
+ ckpt=./checkpoints/fastwam_release/libero_uncond_2cam224.pt \
332
+ EVALUATION.dataset_stats_path=./checkpoints/fastwam_release/libero_uncond_2cam224_dataset_stats.json \
333
+ EVALUATION.sigma_shift=5.0 \
334
+ MULTIRUN.num_gpus=8
335
+ ```
336
+
337
+ Optional: evaluate released RoboTwin checkpoint:
338
+
339
+ ```bash
340
+ python experiments/robotwin/run_robotwin_manager.py \
341
+ task=robotwin_uncond_3cam_384_1e-4 \
342
+ ckpt=./checkpoints/fastwam_release/robotwin_uncond_3cam_384.pt \
343
+ EVALUATION.dataset_stats_path=./checkpoints/fastwam_release/robotwin_uncond_3cam_384_dataset_stats.json \
344
+ EVALUATION.sigma_shift=5.0 \
345
+ MULTIRUN.num_gpus=8
346
+ ```
347
+
348
+ For faster RoboTwin evaluation, we have enabled `EVALUATION.skip_get_obs_within_replan=true` in [`configs/sim_robotwin.yaml`](./configs/sim_robotwin.yaml).
349
+ This skips RGB rendering while consecutively executing an action chunk within one replan window, which speeds up evaluation but makes the saved video look very low-FPS.
350
+ Set it to `false` if you want to save a fully rendered video.
351
+
352
+ **Note:** We evaluate with **unseen** instructions, following Motus. [Lingbot-VA](https://github.com/Robbyant/lingbot-va/blob/661d52a59dc634a650efcd10a79d06bbb17ea81f/evaluation/robotwin/eval_polict_client_openpi.py#L308) uses **seen** instructions instead. You can try `EVALUATION.instruction_type=seen` to use **seen** instructions, which should theoretically improve performance by one or two points.
353
+
354
+ ## Training
355
+
356
+ ### 1) Precompute T5 embedding cache before training
357
+
358
+ Use `scripts/precompute_text_embeds.py` to precompute embeddings for each training task:
359
+
360
+ ```bash
361
+ # LIBERO
362
+ python scripts/precompute_text_embeds.py task=libero_uncond_2cam224_1e-4
363
+
364
+ # RoboTwin
365
+ python scripts/precompute_text_embeds.py task=robotwin_uncond_3cam_384_1e-4
366
+ ```
367
+
368
+ For multi-GPU:
369
+
370
+ ```bash
371
+ torchrun --standalone --nproc_per_node=8 scripts/precompute_text_embeds.py task=libero_uncond_2cam224_1e-4
372
+ ```
373
+
374
+ ### 2) Training (using `fastwam` as an example)
375
+
376
+ When running a new task for the first time, set `pretrained_norm_stats` in the corresponding `configs/data/*.yaml` to `null` first.
377
+ After one training run, a `dataset_stats.json` file will be generated in the current run directory (for example, `runs/{task_name}/{run_id}/dataset_stats.json`).
378
+ You can then update `pretrained_norm_stats` to that file path for subsequent runs.
379
+
380
+ ```bash
381
+ # LIBERO
382
+ bash scripts/train_zero1.sh 8 task=libero_uncond_2cam224_1e-4
383
+
384
+ # RoboTwin
385
+ bash scripts/train_zero1.sh 8 task=robotwin_uncond_3cam_384_1e-4
386
+ ```
387
+
388
+ For LIBERO, we train on a single node with 8 GPUs. For RoboTwin, we use 64 GPUs to accelerate training. You can try reducing the GPU count or training epochs.
389
+
390
+ ## Inference with Your Trained Checkpoints
391
+
392
+ The `mujoco` environment should ideally stay consistent with the LIBERO data version. Then run LIBERO evaluation:
393
+
394
+ ```bash
395
+ # LIBERO
396
+ python experiments/libero/run_libero_manager.py task={task_name} ckpt={ckpt_path}
397
+ ```
398
+
399
+ We have already copied the `RoboTwin` evaluation-related code into `third_party/RoboTwin`.
400
+ You still need to follow the official RoboTwin instructions from the
401
+ [RoboTwin repository](https://github.com/RoboTwin-Platform/RoboTwin).
402
+ Finish installation and download the required assets, then create the policy symlink:
403
+
404
+ ```bash
405
+ ln -sfn "$(pwd)/experiments/robotwin/fastwam_policy" "$(pwd)/third_party/RoboTwin/policy/fastwam_policy"
406
+ ```
407
+
408
+ Then run RoboTwin evaluation:
409
+
410
+ ```bash
411
+ python experiments/robotwin/run_robotwin_manager.py task={task_name} ckpt={ckpt_path}
412
+ ```
413
+
414
+ Common `task_name` examples:
415
+
416
+ ```text
417
+ libero_uncond_2cam224_1e-4
418
+ robotwin_uncond_3cam_384_1e-4
419
+ ```
420
+
421
+ ## Acknowledgements
422
+
423
+ The RoboTwin evaluation code in this repository is adapted from the official [RoboTwin repository](https://github.com/RoboTwin-Platform/RoboTwin). We thank the RoboTwin team for releasing their codebase and assets.
424
+
425
+ ## BibTeX
426
+
427
+ If you find our work helpful, please consider citing:
428
+
429
+ ```bibtex
430
+ @article{yuan2026fastwam,
431
+ title={Fast-WAM: Do World Action Models Need Test-time Future Imagination?},
432
+ author={Tianyuan Yuan and Zibin Dong and Yicheng Liu and Hang Zhao},
433
+ journal={arXiv preprint arXiv:2603.16666},
434
+ year={2026},
435
+ url={https://arxiv.org/abs/2603.16666}
436
+ }
437
+ ```
baseline/fastwam_zipper/source/configs/data/galaxea_r1_lite_3cam.yaml ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ train:
2
+ _target_: fastwam.datasets.lerobot.robot_video_dataset.RobotVideoDataset
3
+ dataset_dirs:
4
+ - ${oc.env:GALAXEA_FASTWAM_TRAIN_DATA,/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/fastwam_lerobot_v2/train}
5
+ shape_meta: &shape_meta
6
+ images:
7
+ - key: head_left
8
+ raw_shape: [3, 480, 640]
9
+ shape: [3, 240, 320]
10
+ - key: wrist_left
11
+ raw_shape: [3, 480, 640]
12
+ shape: [3, 240, 320]
13
+ - key: wrist_right
14
+ raw_shape: [3, 480, 640]
15
+ shape: [3, 240, 320]
16
+ action:
17
+ - key: default
18
+ raw_shape: 14
19
+ shape: 14
20
+ state:
21
+ - key: default
22
+ raw_shape: 14
23
+ shape: 14
24
+ # 25 native 20 Hz steps give the shared 24-action horizon. Sampling every
25
+ # third image yields 9 video frames, satisfying Wan's 4n+1 contract.
26
+ num_frames: 25
27
+ global_sample_stride: 1
28
+ action_video_freq_ratio: 3
29
+ video_size: [384, 320]
30
+ camera_key: null
31
+ val_set_proportion: 0.0
32
+ is_training_set: true
33
+ skip_padding_as_possible: false
34
+ concat_multi_camera: robotwin
35
+ tolerance_s: 0.001
36
+ video_backend: pyav
37
+ processor: &processor
38
+ _target_: fastwam.datasets.lerobot.processors.fastwam_processor.FastWAMProcessor
39
+ shape_meta: ${data.train.shape_meta}
40
+ num_obs_steps: ${data.train.num_frames}
41
+ num_output_cameras: 3
42
+ action_output_dim: 14
43
+ proprio_output_dim: 14
44
+ action_state_transforms:
45
+ - _target_: fastwam.datasets.lerobot.transforms.relative_action.MaskedRelativeJointTransform
46
+ masks:
47
+ default: [true, true, true, true, true, true, false, true, true, true, true, true, true, false]
48
+ use_stepwise_action_norm: false
49
+ norm_default_mode: z-score
50
+ norm_exception_mode: null
51
+ action_state_merger:
52
+ _target_: fastwam.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
53
+ train_transforms:
54
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
55
+ - _target_: torchvision.transforms.Resize
56
+ size: [240, 320]
57
+ val_transforms:
58
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
59
+ - _target_: torchvision.transforms.Resize
60
+ size: [240, 320]
61
+ text_embedding_cache_dir: ${oc.env:GALAXEA_FASTWAM_TEXT_CACHE,/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/fastwam_text_cache}
62
+ use_text_embed_cache: true
63
+ context_len: 128
64
+
65
+ val:
66
+ _target_: fastwam.datasets.lerobot.robot_video_dataset.RobotVideoDataset
67
+ dataset_dirs:
68
+ - ${oc.env:GALAXEA_FASTWAM_VAL_DATA,/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/fastwam_lerobot_v2/validation}
69
+ shape_meta: *shape_meta
70
+ num_frames: 25
71
+ global_sample_stride: 1
72
+ action_video_freq_ratio: 3
73
+ video_size: [384, 320]
74
+ camera_key: null
75
+ val_set_proportion: 0.0
76
+ is_training_set: false
77
+ skip_padding_as_possible: false
78
+ concat_multi_camera: robotwin
79
+ tolerance_s: 0.001
80
+ video_backend: pyav
81
+ processor:
82
+ <<: *processor
83
+ shape_meta: ${data.val.shape_meta}
84
+ num_obs_steps: ${data.val.num_frames}
85
+ text_embedding_cache_dir: ${data.train.text_embedding_cache_dir}
86
+ use_text_embed_cache: true
87
+ context_len: 128
baseline/fastwam_zipper/source/configs/data/libero_2cam.yaml ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ train:
2
+ _target_: fastwam.datasets.lerobot.robot_video_dataset.RobotVideoDataset
3
+ dataset_dirs:
4
+ - ./data/libero_mujoco3.3.2/libero_spatial_no_noops_lerobot
5
+ - ./data/libero_mujoco3.3.2/libero_object_no_noops_lerobot
6
+ - ./data/libero_mujoco3.3.2/libero_goal_no_noops_lerobot
7
+ - ./data/libero_mujoco3.3.2/libero_10_no_noops_lerobot
8
+ shape_meta:
9
+ images:
10
+ - key: image
11
+ # - key: observation.images.image
12
+ raw_shape: [3, 512, 512]
13
+ shape: [3, 224, 224]
14
+ - key: wrist_image
15
+ # - key: observation.images.wrist_image
16
+ raw_shape: [3, 512, 512]
17
+ shape: [3, 224, 224]
18
+ action:
19
+ - key: default
20
+ # - key: action
21
+ raw_shape: 7 # before transform
22
+ shape: 7 # after transform
23
+ state:
24
+ - key: default
25
+ # - key: observation.state
26
+ raw_shape: 8 # before transform
27
+ shape: 8 # after transform
28
+ num_frames: 33
29
+ global_sample_stride: 1
30
+ action_video_freq_ratio: 4 # 32 action, 9 video frames
31
+ video_size: [224, 448] # final resize video
32
+ camera_key: null
33
+ val_set_proportion: 0.0
34
+ is_training_set: true
35
+ skip_padding_as_possible: false
36
+ concat_multi_camera: "horizontal"
37
+ processor:
38
+ _target_: fastwam.datasets.lerobot.processors.fastwam_processor.FastWAMProcessor
39
+ shape_meta: ${data.train.shape_meta}
40
+ num_obs_steps: ${data.train.num_frames}
41
+ num_output_cameras: 2
42
+ action_output_dim: 7 # eef_pose (6) + gripper (1)
43
+ proprio_output_dim: 8 # eef_pose (6) + gripper (2)
44
+ delta_action_dim_mask:
45
+ default: [true, true, true, true, true, true, false] # eef poses are delta, gripper is not
46
+
47
+ # action & state normalization
48
+ action_state_transforms: null
49
+ # action_state_transforms:
50
+ # - _target_: fastwam.datasets.lerobot.transforms.libero.LiberoTransform
51
+ use_stepwise_action_norm: False
52
+ norm_default_mode: min/max
53
+ norm_exception_mode: null
54
+
55
+ action_state_merger:
56
+ _target_: fastwam.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
57
+
58
+ train_transforms:
59
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
60
+ - _target_: torchvision.transforms.Resize
61
+ size: [224, 224]
62
+ val_transforms:
63
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
64
+ - _target_: torchvision.transforms.Resize
65
+ size: [224, 224]
66
+ text_embedding_cache_dir: ./data/text_embeds_cache/libero
67
+ use_text_embed_cache: true
68
+ context_len: 128
baseline/fastwam_zipper/source/configs/data/libero_2cam_lerobot_v30.yaml ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ train:
2
+ _target_: fastwam.datasets.lerobot3.robot_video_dataset.RobotVideoDataset
3
+ dataset_dirs:
4
+ - ./data/lerobot_v30/libero_spatial_no_noops_lerobot
5
+ - ./data/lerobot_v30/libero_object_no_noops_lerobot
6
+ - ./data/lerobot_v30/libero_goal_no_noops_lerobot
7
+ - ./data/lerobot_v30/libero_10_no_noops_lerobot
8
+ shape_meta:
9
+ images:
10
+ - key: image
11
+ raw_shape: [3, 512, 512]
12
+ shape: [3, 224, 224]
13
+ - key: wrist_image
14
+ raw_shape: [3, 512, 512]
15
+ shape: [3, 224, 224]
16
+ action:
17
+ - key: default
18
+ raw_shape: 7
19
+ shape: 7
20
+ state:
21
+ - key: default
22
+ raw_shape: 8
23
+ shape: 8
24
+ num_frames: 33
25
+ global_sample_stride: 1
26
+ action_video_freq_ratio: 4
27
+ video_size: [224, 448]
28
+ camera_key: null
29
+ val_set_proportion: 0.0
30
+ is_training_set: true
31
+ skip_padding_as_possible: false
32
+ concat_multi_camera: "horizontal"
33
+ processor:
34
+ _target_: fastwam.datasets.lerobot.processors.fastwam_processor.FastWAMProcessor
35
+ shape_meta: ${data.train.shape_meta}
36
+ num_obs_steps: ${data.train.num_frames}
37
+ num_output_cameras: 2
38
+ action_output_dim: 7
39
+ proprio_output_dim: 8
40
+ delta_action_dim_mask:
41
+ default: [true, true, true, true, true, true, false]
42
+ action_state_transforms: null
43
+ use_stepwise_action_norm: false
44
+ norm_default_mode: min/max
45
+ norm_exception_mode: null
46
+ action_state_merger:
47
+ _target_: fastwam.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
48
+ train_transforms:
49
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
50
+ - _target_: torchvision.transforms.Resize
51
+ size: [224, 224]
52
+ val_transforms:
53
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
54
+ - _target_: torchvision.transforms.Resize
55
+ size: [224, 224]
56
+ text_embedding_cache_dir: ./data/text_embeds_cache/libero
57
+ use_text_embed_cache: true
58
+ context_len: 128
59
+ video_backend: pyav
baseline/fastwam_zipper/source/configs/data/robotwin.yaml ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ train:
2
+ _target_: fastwam.datasets.lerobot.robot_video_dataset.RobotVideoDataset
3
+ dataset_dirs:
4
+ - ./data/robotwin2.0/robotwin2.0
5
+ shape_meta:
6
+ images:
7
+ - key: cam_high
8
+ raw_shape: [3, 480, 640]
9
+ shape: [3, 240, 320]
10
+ - key: cam_left_wrist
11
+ raw_shape: [3, 480, 640]
12
+ shape: [3, 240, 320]
13
+ - key: cam_right_wrist
14
+ raw_shape: [3, 480, 640]
15
+ shape: [3, 240, 320]
16
+ action:
17
+ - key: default
18
+ raw_shape: 14 # before transform
19
+ shape: 14 # after transform
20
+ state:
21
+ - key: default
22
+ raw_shape: 14 # before transform
23
+ shape: 14 # after transform
24
+ num_frames: 33
25
+ global_sample_stride: 1
26
+ action_video_freq_ratio: 4 # 32 action, 9 video frames
27
+ video_size: [384, 320] # final resize video
28
+ camera_key: null
29
+ val_set_proportion: 0.01
30
+ is_training_set: true
31
+ pretrained_norm_stats: ./data/robotwin2.0/dataset_stats.json
32
+ skip_padding_as_possible: false
33
+ concat_multi_camera: "robotwin"
34
+ processor:
35
+ _target_: fastwam.datasets.lerobot.processors.fastwam_processor.FastWAMProcessor
36
+ shape_meta: ${data.train.shape_meta}
37
+ num_obs_steps: ${data.train.num_frames}
38
+ num_output_cameras: 3
39
+ action_output_dim: 14
40
+ proprio_output_dim: 14
41
+
42
+ # action & state normalization
43
+ action_state_transforms: null
44
+ # action_state_transforms:
45
+ # - _target_: fastwam.datasets.lerobot.transforms.libero.LiberoTransform
46
+ use_stepwise_action_norm: False
47
+ norm_default_mode: "z-score"
48
+ norm_exception_mode: null
49
+
50
+ action_state_merger:
51
+ _target_: fastwam.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
52
+
53
+ train_transforms:
54
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
55
+ - _target_: torchvision.transforms.Resize
56
+ size: [240, 320]
57
+ val_transforms:
58
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
59
+ - _target_: torchvision.transforms.Resize
60
+ size: [240, 320]
61
+ text_embedding_cache_dir: ./data/text_embeds_cache/robotwin
62
+ context_len: 128
63
+
64
+ val:
65
+ _target_: fastwam.datasets.lerobot.robot_video_dataset.RobotVideoDataset
66
+ dataset_dirs:
67
+ - ./data/robotwin2.0/robotwin2.0/
68
+ shape_meta:
69
+ images:
70
+ - key: cam_high
71
+ raw_shape: [3, 480, 640]
72
+ shape: [3, 240, 320]
73
+ - key: cam_left_wrist
74
+ raw_shape: [3, 480, 640]
75
+ shape: [3, 240, 320]
76
+ - key: cam_right_wrist
77
+ raw_shape: [3, 480, 640]
78
+ shape: [3, 240, 320]
79
+ action:
80
+ - key: default
81
+ raw_shape: 14 # before transform
82
+ shape: 14 # after transform
83
+ state:
84
+ - key: default
85
+ raw_shape: 14 # before transform
86
+ shape: 14 # after transform
87
+ num_frames: 33
88
+ global_sample_stride: 1
89
+ action_video_freq_ratio: 4 # 32 action, 9 video frames
90
+ video_size: [384, 320] # final resize video
91
+ camera_key: null
92
+ val_set_proportion: 0.01
93
+ is_training_set: false
94
+ pretrained_norm_stats: ./data/robotwin2.0/dataset_stats.json
95
+ skip_padding_as_possible: false
96
+ concat_multi_camera: "robotwin"
97
+ processor:
98
+ _target_: fastwam.datasets.lerobot.processors.fastwam_processor.FastWAMProcessor
99
+ shape_meta: ${data.train.shape_meta}
100
+ num_obs_steps: ${data.train.num_frames}
101
+ num_output_cameras: 3
102
+ action_output_dim: 14
103
+ proprio_output_dim: 14
104
+
105
+ # action & state normalization
106
+ action_state_transforms: null
107
+ # action_state_transforms:
108
+ # - _target_: fastwam.datasets.lerobot.transforms.libero.LiberoTransform
109
+ use_stepwise_action_norm: False
110
+ norm_default_mode: "z-score"
111
+ norm_exception_mode: null
112
+
113
+ action_state_merger:
114
+ _target_: fastwam.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
115
+
116
+ train_transforms:
117
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
118
+ - _target_: torchvision.transforms.Resize
119
+ size: [240, 320]
120
+ val_transforms:
121
+ - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
122
+ - _target_: torchvision.transforms.Resize
123
+ size: [240, 320]
124
+ text_embedding_cache_dir: ./data/text_embeds_cache/robotwin
125
+ context_len: 128
baseline/fastwam_zipper/source/configs/model/fastwam.yaml ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _target_: fastwam.runtime.create_fastwam
2
+ model_id: Wan-AI/Wan2.2-TI2V-5B
3
+ tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
4
+ tokenizer_max_len: 128
5
+ load_text_encoder: false
6
+ proprio_dim: ${data.train.processor.proprio_output_dim}
7
+ redirect_common_files: true
8
+ mot_checkpoint_mixed_attn: false
9
+ compile_training_denoise: false
10
+ action_dit_pretrained_path: checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
11
+ skip_dit_load_from_pretrain: false
12
+
13
+ video_dit_config:
14
+ has_image_input: false
15
+ patch_size: [1, 2, 2]
16
+ in_dim: 48
17
+ hidden_dim: 3072
18
+ ffn_dim: 14336
19
+ freq_dim: 256
20
+ text_dim: 4096
21
+ out_dim: 48
22
+ num_heads: 24
23
+ attn_head_dim: 128
24
+ num_layers: 30
25
+ eps: 1.0e-06
26
+ seperated_timestep: true
27
+ require_clip_embedding: false
28
+ require_vae_embedding: false
29
+ fuse_vae_embedding_in_latents: true
30
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
31
+ video_attention_mask_mode: "first_frame_causal" # default
32
+ action_conditioned: false
33
+ action_dim: ${data.train.processor.action_output_dim}
34
+ action_group_causal_mask_mode: "group_diagonal"
35
+
36
+ action_dit_config:
37
+ action_dim: ${data.train.processor.action_output_dim}
38
+ hidden_dim: 1024
39
+ ffn_dim: 4096
40
+ num_heads: 24
41
+ attn_head_dim: 128
42
+ num_layers: 30
43
+ text_dim: 4096
44
+ freq_dim: 256
45
+ eps: 1.0e-06
46
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
47
+
48
+ video_scheduler:
49
+ train_shift: 5.0
50
+ infer_shift: 5.0
51
+ num_train_timesteps: 1000
52
+
53
+ action_scheduler:
54
+ train_shift: 1.0
55
+ infer_shift: 1.0
56
+ num_train_timesteps: 1000
57
+
58
+ loss:
59
+ lambda_action: 1.0
baseline/fastwam_zipper/source/configs/model/fastwam_idm.yaml ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _target_: fastwam.runtime.create_fastwam_idm
2
+ model_id: Wan-AI/Wan2.2-TI2V-5B
3
+ tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
4
+ tokenizer_max_len: 128
5
+ load_text_encoder: false
6
+ proprio_dim: ${data.train.processor.proprio_output_dim}
7
+ redirect_common_files: true
8
+ mot_checkpoint_mixed_attn: false
9
+ compile_training_denoise: false
10
+ video_cond_noise_prob: 0.5
11
+ action_dit_pretrained_path: checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
12
+ skip_dit_load_from_pretrain: false
13
+
14
+ video_dit_config:
15
+ has_image_input: false
16
+ patch_size: [1, 2, 2]
17
+ in_dim: 48
18
+ hidden_dim: 3072
19
+ ffn_dim: 14336
20
+ freq_dim: 256
21
+ text_dim: 4096
22
+ out_dim: 48
23
+ num_heads: 24
24
+ attn_head_dim: 128
25
+ num_layers: 30
26
+ eps: 1.0e-06
27
+ seperated_timestep: true
28
+ require_clip_embedding: false
29
+ require_vae_embedding: false
30
+ fuse_vae_embedding_in_latents: true
31
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
32
+ video_attention_mask_mode: "first_frame_causal" # default
33
+ action_conditioned: false
34
+ action_dim: ${data.train.processor.action_output_dim}
35
+ action_group_causal_mask_mode: "group_diagonal"
36
+
37
+ action_dit_config:
38
+ action_dim: ${data.train.processor.action_output_dim}
39
+ hidden_dim: 1024
40
+ ffn_dim: 4096
41
+ num_heads: 24
42
+ attn_head_dim: 128
43
+ num_layers: 30
44
+ text_dim: 4096
45
+ freq_dim: 256
46
+ eps: 1.0e-06
47
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
48
+
49
+ video_scheduler:
50
+ train_shift: 5.0
51
+ infer_shift: 5.0
52
+ num_train_timesteps: 1000
53
+
54
+ action_scheduler:
55
+ train_shift: 1.0
56
+ infer_shift: 1.0
57
+ num_train_timesteps: 1000
58
+
59
+ loss:
60
+ lambda_action: 1.0
baseline/fastwam_zipper/source/configs/model/fastwam_joint.yaml ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _target_: fastwam.runtime.create_fastwam_joint
2
+ model_id: Wan-AI/Wan2.2-TI2V-5B
3
+ tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
4
+ tokenizer_max_len: 128
5
+ load_text_encoder: false
6
+ proprio_dim: ${data.train.processor.proprio_output_dim}
7
+ redirect_common_files: true
8
+ mot_checkpoint_mixed_attn: false
9
+ action_dit_pretrained_path: checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
10
+ skip_dit_load_from_pretrain: false
11
+
12
+ video_dit_config:
13
+ has_image_input: false
14
+ patch_size: [1, 2, 2]
15
+ in_dim: 48
16
+ hidden_dim: 3072
17
+ ffn_dim: 14336
18
+ freq_dim: 256
19
+ text_dim: 4096
20
+ out_dim: 48
21
+ num_heads: 24
22
+ attn_head_dim: 128
23
+ num_layers: 30
24
+ eps: 1.0e-06
25
+ seperated_timestep: true
26
+ require_clip_embedding: false
27
+ require_vae_embedding: false
28
+ fuse_vae_embedding_in_latents: true
29
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
30
+ video_attention_mask_mode: "first_frame_causal" # default
31
+ action_conditioned: false
32
+ action_dim: ${data.train.processor.action_output_dim}
33
+ action_group_causal_mask_mode: "group_diagonal"
34
+
35
+ action_dit_config:
36
+ action_dim: ${data.train.processor.action_output_dim}
37
+ hidden_dim: 1024
38
+ ffn_dim: 4096
39
+ num_heads: 24
40
+ attn_head_dim: 128
41
+ num_layers: 30
42
+ text_dim: 4096
43
+ freq_dim: 256
44
+ eps: 1.0e-06
45
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
46
+
47
+ video_scheduler:
48
+ train_shift: 5.0
49
+ infer_shift: 5.0
50
+ num_train_timesteps: 1000
51
+
52
+ action_scheduler:
53
+ train_shift: 1.0
54
+ infer_shift: 1.0
55
+ num_train_timesteps: 1000
56
+
57
+ loss:
58
+ lambda_action: 1.0
baseline/fastwam_zipper/source/configs/model/fastwam_optional_idm.yaml ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _target_: fastwam.runtime.create_fastwam_optional_idm
2
+ model_id: Wan-AI/Wan2.2-TI2V-5B
3
+ tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
4
+ tokenizer_max_len: 128
5
+ load_text_encoder: false
6
+ proprio_dim: ${data.train.processor.proprio_output_dim}
7
+ redirect_common_files: true
8
+ mot_checkpoint_mixed_attn: false
9
+ compile_training_denoise: false
10
+ action_idm_prob: 0.5
11
+ video_cond_noise_prob: 0.0
12
+ action_dit_pretrained_path: checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
13
+ skip_dit_load_from_pretrain: false
14
+
15
+ video_dit_config:
16
+ has_image_input: false
17
+ patch_size: [1, 2, 2]
18
+ in_dim: 48
19
+ hidden_dim: 3072
20
+ ffn_dim: 14336
21
+ freq_dim: 256
22
+ text_dim: 4096
23
+ out_dim: 48
24
+ num_heads: 24
25
+ attn_head_dim: 128
26
+ num_layers: 30
27
+ eps: 1.0e-06
28
+ seperated_timestep: true
29
+ require_clip_embedding: false
30
+ require_vae_embedding: false
31
+ fuse_vae_embedding_in_latents: true
32
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
33
+ video_attention_mask_mode: "first_frame_causal"
34
+ action_conditioned: false
35
+ action_dim: ${data.train.processor.action_output_dim}
36
+ action_group_causal_mask_mode: "group_diagonal"
37
+
38
+ action_dit_config:
39
+ action_dim: ${data.train.processor.action_output_dim}
40
+ hidden_dim: 1024
41
+ ffn_dim: 4096
42
+ num_heads: 24
43
+ attn_head_dim: 128
44
+ num_layers: 30
45
+ text_dim: 4096
46
+ freq_dim: 256
47
+ eps: 1.0e-06
48
+ use_gradient_checkpointing: ${model.mot_checkpoint_mixed_attn}
49
+
50
+ video_scheduler:
51
+ train_shift: 5.0
52
+ infer_shift: 5.0
53
+ num_train_timesteps: 1000
54
+
55
+ action_scheduler:
56
+ train_shift: 1.0
57
+ infer_shift: 1.0
58
+ num_train_timesteps: 1000
59
+
60
+ loss:
61
+ lambda_video: 1.0
62
+ lambda_action: 1.0
baseline/fastwam_zipper/source/configs/sim_libero.yaml ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - train
5
+ - override /task: libero_uncond_2cam224_1e-4
6
+ - _self_
7
+
8
+ ckpt: null
9
+ gpu_id: 0
10
+
11
+ model:
12
+ load_text_encoder: true
13
+ skip_dit_load_from_pretrain: true
14
+ action_dit_pretrained_path: null
15
+
16
+ EVALUATION:
17
+ task_suite_name: libero_spatial
18
+ task_id: 0
19
+ num_trials: 50
20
+ output_dir: ./evaluate_results/libero/${hydra:runtime.choices.task}/${now:%Y%m%d_%H%M%S}
21
+
22
+ # Runtime behavior
23
+ env_num: 1
24
+ num_steps_wait: 30
25
+ replan_steps: 10
26
+ binarize_gripper: true
27
+ use_action_ensembler: false
28
+ visualize_future_video: false
29
+
30
+ # Action inference
31
+ action_horizon: null
32
+ num_inference_steps: ${eval_num_inference_steps}
33
+ sigma_shift: null
34
+ text_cfg_scale: 1.0
35
+ negative_prompt: ""
36
+ rand_device: cpu
37
+ tiled: false
38
+ compile_action_infer: true
39
+
40
+ # Optional external inputs
41
+ dataset_stats_path: null
42
+ device: cuda
43
+
44
+ MULTIRUN:
45
+ create_only: false
46
+ task_file: null
47
+ task_suite_names:
48
+ - libero_10
49
+ - libero_goal
50
+ - libero_spatial
51
+ - libero_object
52
+ num_gpus: 8
53
+
54
+ hydra:
55
+ job:
56
+ chdir: false
57
+ run:
58
+ dir: .
59
+ output_subdir: null
baseline/fastwam_zipper/source/configs/sim_robotwin.yaml ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - train
5
+ - override /task: robotwin_uncond_3cam_384_1e-4
6
+ - _self_
7
+
8
+ ckpt: null
9
+ gpu_id: 0
10
+
11
+ model:
12
+ load_text_encoder: true
13
+ skip_dit_load_from_pretrain: true
14
+ action_dit_pretrained_path: null
15
+
16
+ EVALUATION:
17
+ robotwin_root: third_party/RoboTwin
18
+ policy_name: fastwam_policy
19
+ task_name: null
20
+ task_config: demo_randomized
21
+ instruction_type: unseen
22
+ eval_num_episodes: 100
23
+ output_dir: ./evaluate_results/robotwin/${hydra:runtime.choices.task}/${now:%Y%m%d_%H%M%S}
24
+
25
+ # Action inference
26
+ action_horizon: null
27
+ replan_steps: 24
28
+ num_inference_steps: ${eval_num_inference_steps}
29
+ sigma_shift: null
30
+ text_cfg_scale: 1.0
31
+ negative_prompt: ""
32
+ rand_device: cpu
33
+ tiled: false
34
+ timing_enabled: false
35
+ skip_get_obs_within_replan: true
36
+
37
+ # Runtime inputs
38
+ dataset_stats_path: null
39
+ device: cuda
40
+
41
+ MULTIRUN:
42
+ enabled: false
43
+ num_gpus: 8
44
+ max_tasks_per_gpu: 2
45
+
46
+ hydra:
47
+ job:
48
+ chdir: false
49
+ run:
50
+ dir: .
51
+ output_subdir: null
baseline/fastwam_zipper/source/configs/task/galaxea_r1_lite_3cam_1e-4.yaml ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: galaxea_r1_lite_3cam
5
+ - override /model: fastwam
6
+ - _self_
7
+
8
+ # Formal launcher: 8 samples/GPU x 8 GPUs x 2 accumulation = GBS 128.
9
+ batch_size: 8
10
+ num_workers: 8
11
+ gradient_accumulation_steps: 2
12
+
13
+ model:
14
+ # ModelConfig joins these ids to DIFFSYNTH_MODEL_BASE_PATH. The validated
15
+ # local snapshots are stored without a redundant `Wan-AI/` directory level.
16
+ model_id: ${oc.env:FASTWAM_WAN_MODEL_ID,Wan2.2-TI2V-5B}
17
+ tokenizer_model_id: ${oc.env:FASTWAM_TOKENIZER_MODEL_ID,Wan2.1-T2V-1.3B}
18
+ # The released RobotWin 3-camera checkpoint supplies both experts. Avoid a
19
+ # redundant 5B DiT load before the weight-only resume is applied.
20
+ skip_dit_load_from_pretrain: true
21
+ action_dit_pretrained_path: null
22
+ redirect_common_files: false
23
+ mot_checkpoint_mixed_attn: false
24
+ compile_training_denoise: false
25
+
26
+ learning_rate: 1.0e-4
27
+ lr_scheduler_type: cosine
28
+ num_epochs: 10
29
+ max_steps: null
30
+ log_every: 10
31
+ save_every: 1000
32
+ eval_every: 500
33
+ eval_num_inference_steps: 10
34
+ weight_decay: 1.0e-2
35
+ mixed_precision: bf16
36
+ max_grad_norm: 1.0
37
+ seed: 20261002
38
+ resume: null
39
+
40
+ wandb:
41
+ enabled: true
42
+ workspace: null
43
+ project: galaxea-real-baselines
44
+ name: fastwam-galaxea-r1-lite-3cam
45
+ group: galaxea-real-baselines-v1
46
+ mode: online
baseline/fastwam_zipper/source/configs/task/libero_idm_2cam224_1e-4.yaml ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: libero_2cam
5
+ - override /model: fastwam_idm
6
+ - _self_
7
+
8
+ # dataloading
9
+ batch_size: 16
10
+ num_workers: 8
11
+
12
+ model:
13
+ mot_checkpoint_mixed_attn: false
14
+
15
+ # scheduler
16
+ lr_scheduler_type: "cosine"
17
+ learning_rate: 1e-4
18
+ num_epochs: 10
19
+ max_steps: null
20
+ log_every: 10
21
+ save_every: 2000
22
+ eval_every: 200
23
+
24
+ # training
25
+ gradient_accumulation_steps: 1
26
+ weight_decay: 1e-2
27
+ resume: null
baseline/fastwam_zipper/source/configs/task/libero_joint_2cam224_1e-4.yaml ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: libero_2cam
5
+ - override /model: fastwam_joint
6
+ - _self_
7
+
8
+ # dataloading
9
+ batch_size: 16
10
+ num_workers: 8
11
+
12
+ model:
13
+ mot_checkpoint_mixed_attn: false
14
+
15
+ # scheduler
16
+ lr_scheduler_type: "cosine"
17
+ learning_rate: 1e-4
18
+ num_epochs: 10
19
+ max_steps: null
20
+ log_every: 10
21
+ save_every: 2000
22
+ eval_every: 200
23
+
24
+ # training
25
+ gradient_accumulation_steps: 1
26
+ weight_decay: 1e-2
27
+ resume: null
baseline/fastwam_zipper/source/configs/task/libero_optional_idm_2cam224_1e-4.yaml ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: libero_2cam
5
+ - override /model: fastwam_optional_idm
6
+ - _self_
7
+
8
+ batch_size: 16
9
+ num_workers: 8
10
+
11
+ model:
12
+ mot_checkpoint_mixed_attn: false
13
+
14
+ lr_scheduler_type: "cosine"
15
+ learning_rate: 1e-4
16
+ num_epochs: 10
17
+ max_steps: null
18
+ log_every: 10
19
+ save_every: 2000
20
+ eval_every: 200
21
+
22
+ gradient_accumulation_steps: 1
23
+ weight_decay: 1e-2
24
+ resume: null
baseline/fastwam_zipper/source/configs/task/libero_uncond_2cam224_1e-4.yaml ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: libero_2cam
5
+ - override /model: fastwam
6
+ - _self_
7
+
8
+ # dataloading
9
+ batch_size: 16
10
+ num_workers: 8
11
+
12
+ model:
13
+ mot_checkpoint_mixed_attn: false
14
+
15
+ # scheduler
16
+ lr_scheduler_type: "cosine"
17
+ learning_rate: 1e-4
18
+ num_epochs: 10
19
+ max_steps: null
20
+ log_every: 10
21
+ save_every: 2000
22
+ eval_every: 200
23
+
24
+ # training
25
+ gradient_accumulation_steps: 1
26
+ weight_decay: 1e-2
27
+ resume: null
baseline/fastwam_zipper/source/configs/task/robotwin_idm_3cam_384_1e-4.yaml ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: robotwin
5
+ - override /model: fastwam_idm
6
+ - _self_
7
+
8
+ # dataloading
9
+ batch_size: 16
10
+ num_workers: 8
11
+
12
+ model:
13
+ mot_checkpoint_mixed_attn: false
14
+
15
+ # scheduler
16
+ lr_scheduler_type: "cosine"
17
+ learning_rate: 1e-4
18
+ num_epochs: 5
19
+ max_steps: null
20
+ log_every: 10
21
+ save_every: 2500
22
+ eval_every: 500
23
+
24
+ # training
25
+ gradient_accumulation_steps: 1
26
+ weight_decay: 1e-2
27
+ resume: null
baseline/fastwam_zipper/source/configs/task/robotwin_joint_3cam_384_1e-4.yaml ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: robotwin
5
+ - override /model: fastwam_joint
6
+ - _self_
7
+
8
+ # dataloading
9
+ batch_size: 16
10
+ num_workers: 8
11
+
12
+ model:
13
+ mot_checkpoint_mixed_attn: false
14
+
15
+ # scheduler
16
+ lr_scheduler_type: "cosine"
17
+ learning_rate: 1e-4
18
+ num_epochs: 5
19
+ max_steps: null
20
+ log_every: 10
21
+ save_every: 2500
22
+ eval_every: 500
23
+
24
+ # training
25
+ gradient_accumulation_steps: 1
26
+ weight_decay: 1e-2
27
+ resume: null
baseline/fastwam_zipper/source/configs/task/robotwin_uncond_3cam_384_1e-4.yaml ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - override /data: robotwin
5
+ - override /model: fastwam
6
+ - _self_
7
+
8
+ # dataloading
9
+ batch_size: 16
10
+ num_workers: 8
11
+
12
+ model:
13
+ mot_checkpoint_mixed_attn: false
14
+
15
+ # scheduler
16
+ lr_scheduler_type: "cosine"
17
+ learning_rate: 1e-4
18
+ num_epochs: 5
19
+ max_steps: null
20
+ log_every: 10
21
+ save_every: 2500
22
+ eval_every: 500
23
+
24
+ # training
25
+ gradient_accumulation_steps: 1
26
+ weight_decay: 1e-2
27
+ resume: null
baseline/fastwam_zipper/source/configs/train.yaml ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # @package _global_
2
+
3
+ defaults:
4
+ - _self_
5
+ - data: null
6
+ - model: null
7
+ - task: null
8
+
9
+
10
+ output_dir: ./runs/train/${now:%Y-%m-%d}_${now:%H-%M-%S}
11
+
12
+ # dataloading
13
+ batch_size: 2
14
+ num_workers: 4
15
+
16
+ # scheduler
17
+ lr_scheduler_type: "cosine"
18
+ learning_rate: 1.0e-4
19
+ num_epochs: 1
20
+ max_steps: null
21
+ log_every: 10
22
+ save_every: 100
23
+ eval_every: 50
24
+ eval_num_inference_steps: 10
25
+
26
+ # training
27
+ gradient_accumulation_steps: 1
28
+ mixed_precision: "bf16" # one of: [no, fp16, bf16]
29
+ seed: 42
30
+ max_grad_norm: 1.0
31
+ weight_decay: 0.0
32
+ resume: null
33
+
34
+ # wandb
35
+ wandb:
36
+ enabled: false
37
+ workspace: null
38
+ project: fast-wam
39
+ name: train
40
+ group: null
41
+ mode: online
42
+
43
+ hydra:
44
+ job:
45
+ chdir: false
46
+ run:
47
+ dir: .
48
+ output_subdir: null
baseline/fastwam_zipper/source/pyproject.toml ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [build-system]
2
+ requires = ["setuptools>=68", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "fastwam"
7
+ version = "0.1.0"
8
+ description = "FastWAM: standalone Wan2.2-TI2V-5B training/inference package"
9
+ requires-python = ">=3.10"
10
+ dependencies = [
11
+ "accelerate==1.12.0",
12
+ "av==15.1.0",
13
+ "boto3==1.35.99",
14
+ "datasets==4.8.5",
15
+ "deepspeed==0.18.7",
16
+ "einops==0.8.1",
17
+ "gitpython==3.1.45",
18
+ "huggingface-hub==0.29.2",
19
+ "hydra-core==1.3.2",
20
+ "imageio==2.37.0",
21
+ "imageio-ffmpeg==0.6.0",
22
+ "jsonlines==4.0.0",
23
+ "modelscope==1.34.0",
24
+ "numpy==2.2.6",
25
+ "omegaconf==2.3.0",
26
+ "packaging==25.0",
27
+ "pandas==2.2.3",
28
+ "pillow==12.0.0",
29
+ "pyarrow==24.0.0",
30
+ "regex==2025.11.3",
31
+ "rich==14.2.0",
32
+ "safetensors==0.5.3",
33
+ "termcolor==2.5.0",
34
+ "torch==2.7.1+cu128",
35
+ "torchcodec==0.4.0",
36
+ "torchvision==0.22.1+cu128",
37
+ "tqdm==4.68.3",
38
+ "transformers==4.49.0",
39
+ "typing-extensions==4.15.0",
40
+ "wandb==0.23.1",
41
+ ]
42
+
43
+ [tool.setuptools.package-dir]
44
+ "" = "src"
45
+
46
+ [tool.setuptools.packages.find]
47
+ where = ["src"]
48
+ include = ["fastwam*"]
baseline/fastwam_zipper/source/scripts/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ """CLI entry scripts for fastwam."""
baseline/fastwam_zipper/source/scripts/accelerate_configs/accelerate_zero0.yaml ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ compute_environment: LOCAL_MACHINE
2
+ debug: false
3
+ distributed_type: DEEPSPEED
4
+ deepspeed_config:
5
+ gradient_accumulation_steps: 1
6
+ offload_optimizer_device: none
7
+ offload_param_device: none
8
+ zero3_init_flag: false
9
+ zero_stage: 0
10
+ downcast_bf16: "no"
11
+ enable_cpu_affinity: false
12
+ machine_rank: 0
13
+ main_training_function: main
14
+ mixed_precision: "bf16"
15
+ num_machines: 1
16
+ num_processes: 1
17
+ rdzv_backend: static
18
+ same_network: true
19
+ tpu_env: []
20
+ tpu_use_cluster: false
21
+ tpu_use_sudo: false
22
+ use_cpu: false
baseline/fastwam_zipper/source/scripts/accelerate_configs/accelerate_zero1_ds.yaml ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ compute_environment: LOCAL_MACHINE
2
+ debug: false
3
+ distributed_type: DEEPSPEED
4
+ deepspeed_config:
5
+ deepspeed_config_file: scripts/ds_configs/ds_zero1_config.json
6
+ zero3_init_flag: false
7
+
8
+ # Keep this null when using deepspeed_config_file; precision is controlled by
9
+ # training config (Trainer -> Accelerator(mixed_precision=...)) and DS JSON.
10
+ mixed_precision: null
11
+
12
+ machine_rank: 0
13
+ main_training_function: main
14
+ num_machines: 1
15
+ num_processes: 1
16
+ rdzv_backend: static
17
+ same_network: true
18
+ use_cpu: false
baseline/fastwam_zipper/source/scripts/accelerate_configs/accelerate_zero2_ds.yaml ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ compute_environment: LOCAL_MACHINE
2
+ debug: false
3
+ distributed_type: DEEPSPEED
4
+ deepspeed_config:
5
+ deepspeed_config_file: scripts/ds_configs/ds_zero2_config.json
6
+ zero3_init_flag: false
7
+
8
+ # Keep this null when using deepspeed_config_file; precision is controlled by
9
+ # training config (Trainer -> Accelerator(mixed_precision=...)) and DS JSON.
10
+ mixed_precision: null
11
+
12
+ machine_rank: 0
13
+ main_training_function: main
14
+ num_machines: 1
15
+ num_processes: 1
16
+ rdzv_backend: static
17
+ same_network: true
18
+ use_cpu: false
baseline/fastwam_zipper/source/scripts/check_galaxea_r1_lite_contract.py ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Fail-closed checks for the shared Galaxea LeRobot-v2 FastWAM input."""
3
+
4
+ from __future__ import annotations
5
+
6
+ import argparse
7
+ import hashlib
8
+ import json
9
+ from pathlib import Path
10
+
11
+
12
+ EXPECTED_MANIFEST_SHA256 = "36dcbf10695ec1f6c79a90375f396e59e70ae7643595f51dbb3e1299765be5b2"
13
+ EXPECTED_IMAGE_KEYS = {
14
+ "observation.images.head_left",
15
+ "observation.images.wrist_left",
16
+ "observation.images.wrist_right",
17
+ }
18
+
19
+
20
+ def read_json(path: Path) -> dict:
21
+ value = json.loads(path.read_text(encoding="utf-8"))
22
+ if not isinstance(value, dict):
23
+ raise TypeError(f"expected mapping: {path}")
24
+ return value
25
+
26
+
27
+ def check_pack(root: Path, split: str, expected_episodes: int) -> dict:
28
+ info = read_json(root / "meta/info.json")
29
+ qc = read_json(root / "meta/alignment_qc.json")
30
+ features = info.get("features", {})
31
+ if str(info.get("codebase_version")) != "v2.0":
32
+ raise ValueError(f"{split}: expected LeRobot v2.0")
33
+ if int(info.get("fps", -1)) != 20:
34
+ raise ValueError(f"{split}: expected 20 Hz")
35
+ if int(info.get("total_episodes", -1)) != expected_episodes:
36
+ raise ValueError(f"{split}: episode count mismatch")
37
+ if not EXPECTED_IMAGE_KEYS.issubset(features):
38
+ raise ValueError(f"{split}: missing camera features")
39
+ for key in ("observation.state", "action"):
40
+ if features.get(key, {}).get("shape") != [14]:
41
+ raise ValueError(f"{split}: {key} must be 14D")
42
+ if qc.get("split") != split:
43
+ raise ValueError(f"{split}: alignment report split mismatch")
44
+ if qc.get("pass") is not True or qc.get("strictly_causal") is not True:
45
+ raise ValueError(f"{split}: alignment QC did not pass")
46
+ if qc.get("all_required_timestamps_from_ros_header") is not True:
47
+ raise ValueError(f"{split}: timestamp fallback is forbidden")
48
+ if float(qc.get("max_source_minus_anchor_seconds", 1.0)) > 0.0:
49
+ raise ValueError(f"{split}: future selections are forbidden")
50
+ for episode in qc.get("episodes", []):
51
+ for stream in episode.get("streams", {}).values():
52
+ if int(stream.get("future_selected_frames", -1)) != 0:
53
+ raise ValueError(f"{split}: future-selected camera/state sample")
54
+ for side in episode.get("gripper_latching", {}).values():
55
+ if int(side.get("future_selected_frames", -1)) != 0:
56
+ raise ValueError(f"{split}: future-selected gripper sample")
57
+ return {
58
+ "root": str(root.resolve()),
59
+ "episodes": expected_episodes,
60
+ "frames": int(info["total_frames"]),
61
+ }
62
+
63
+
64
+ def main() -> None:
65
+ parser = argparse.ArgumentParser()
66
+ parser.add_argument("--train", type=Path, required=True)
67
+ parser.add_argument("--validation", type=Path, required=True)
68
+ parser.add_argument("--manifest", type=Path, required=True)
69
+ parser.add_argument("--expected-train-episodes", type=int, default=511)
70
+ parser.add_argument("--expected-validation-episodes", type=int, default=57)
71
+ parser.add_argument("--allow-smoke-manifest", action="store_true")
72
+ args = parser.parse_args()
73
+ digest = hashlib.sha256(args.manifest.read_bytes()).hexdigest()
74
+ if digest != EXPECTED_MANIFEST_SHA256 and not args.allow_smoke_manifest:
75
+ raise ValueError(f"source manifest SHA256 mismatch: {digest}")
76
+ result = {
77
+ "status": "galaxea_fastwam_contract_ok",
78
+ "manifest_sha256": digest,
79
+ "action_horizon": 24,
80
+ "video_frames": 9,
81
+ "views": ["head_left", "wrist_left", "wrist_right"],
82
+ "train": check_pack(args.train, "train", args.expected_train_episodes),
83
+ "validation": check_pack(
84
+ args.validation, "validation", args.expected_validation_episodes
85
+ ),
86
+ }
87
+ print(json.dumps(result, indent=2, sort_keys=True))
88
+
89
+
90
+ if __name__ == "__main__":
91
+ main()
baseline/fastwam_zipper/source/scripts/check_galaxea_text_cache.py ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Validate all fixed Galaxea prompt embeddings without loading UMT5."""
3
+
4
+ from __future__ import annotations
5
+
6
+ import argparse
7
+ import hashlib
8
+ import json
9
+ from pathlib import Path
10
+
11
+ import torch
12
+
13
+
14
+ PROMPT = "A video recorded from a robot's point of view executing the following instruction: {task}"
15
+
16
+
17
+ def main() -> None:
18
+ parser = argparse.ArgumentParser()
19
+ parser.add_argument("--train", type=Path, required=True)
20
+ parser.add_argument("--validation", type=Path, required=True)
21
+ parser.add_argument("--cache", type=Path, required=True)
22
+ args = parser.parse_args()
23
+
24
+ prompts: set[str] = set()
25
+ for dataset in (args.train, args.validation):
26
+ tasks = dataset / "meta/tasks.jsonl"
27
+ for line in tasks.read_text(encoding="utf-8").splitlines():
28
+ if line.strip():
29
+ prompts.add(PROMPT.format(task=json.loads(line)["task"]))
30
+ if len(prompts) != 5:
31
+ raise RuntimeError(f"expected five fixed task prompts, got {len(prompts)}")
32
+
33
+ for prompt in sorted(prompts):
34
+ digest = hashlib.sha256(prompt.encode("utf-8")).hexdigest()
35
+ path = args.cache / f"{digest}.t5_len128.wan22ti2v5b.pt"
36
+ payload = torch.load(path, map_location="cpu", weights_only=True)
37
+ context = payload["context"]
38
+ mask = payload["mask"]
39
+ if context.ndim != 2 or context.shape[0] != 128 or not torch.isfinite(context).all():
40
+ raise RuntimeError(f"invalid context tensor in {path}: {tuple(context.shape)}")
41
+ if mask.shape != (128,):
42
+ raise RuntimeError(f"invalid mask tensor in {path}: {tuple(mask.shape)}")
43
+ print(f"FASTWAM_TEXT_CACHE_OK prompts={len(prompts)} cache={args.cache.resolve()}")
44
+
45
+
46
+ if __name__ == "__main__":
47
+ main()
baseline/fastwam_zipper/source/scripts/dryrun_fastwam.py ADDED
@@ -0,0 +1,227 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Benchmark FastWAM action inference with synthetic task-shaped inputs.
2
+
3
+ Example:
4
+ python scripts/dryrun_fastwam.py \
5
+ task=libero_uncond_2cam224_1e-4 \
6
+ ckpt=./checkpoints/fastwam_release/libero_uncond_2cam224.pt \
7
+ +DRYRUN.compile_action_infer=true \
8
+ +DRYRUN.warmup=2 +DRYRUN.iters=10
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import logging
14
+ import inspect
15
+ import statistics
16
+ import time
17
+
18
+ import hydra
19
+ import torch
20
+ from hydra.utils import instantiate
21
+ from omegaconf import DictConfig, OmegaConf
22
+
23
+
24
+ logger = logging.getLogger(__name__)
25
+
26
+
27
+ def _dryrun_config(cfg: DictConfig) -> DictConfig:
28
+ defaults = OmegaConf.create(
29
+ {
30
+ "warmup": 2,
31
+ "iters": 10,
32
+ "seed": 42,
33
+ "action_horizon": None,
34
+ "num_inference_steps": None,
35
+ "compile_action_infer": False,
36
+ "benchmark_both": False,
37
+ "use_random_context": False,
38
+ "action_infer_mode": "idm",
39
+ }
40
+ )
41
+ return OmegaConf.merge(defaults, cfg.get("DRYRUN", {}))
42
+
43
+
44
+ def _model_dtype(mixed_precision: str) -> torch.dtype:
45
+ return {
46
+ "no": torch.float32,
47
+ "fp16": torch.float16,
48
+ "bf16": torch.bfloat16,
49
+ }[str(mixed_precision)]
50
+
51
+
52
+ def _timed_infer_with_output(
53
+ model,
54
+ kwargs: dict,
55
+ device: torch.device,
56
+ ) -> tuple[float, torch.Tensor]:
57
+ torch.cuda.synchronize(device)
58
+ start = time.perf_counter()
59
+ with torch.no_grad():
60
+ output = model.infer_action(**kwargs)
61
+ torch.cuda.synchronize(device)
62
+ logger.info("action shape=%s", tuple(output["action"].shape))
63
+ return time.perf_counter() - start, output["action"].detach().clone()
64
+
65
+
66
+ def _timed_infer(model, kwargs: dict, device: torch.device) -> float:
67
+ elapsed, _ = _timed_infer_with_output(model, kwargs, device)
68
+ return elapsed
69
+
70
+
71
+ def _benchmark_mode(
72
+ model,
73
+ infer_kwargs: dict,
74
+ device: torch.device,
75
+ *,
76
+ label: str,
77
+ warmup: int,
78
+ iters: int,
79
+ ) -> tuple[list[float], float]:
80
+ for index in range(warmup):
81
+ elapsed = _timed_infer(model, infer_kwargs, device)
82
+ logger.info("%s warmup %d/%d: %.4fs", label, index + 1, warmup, elapsed)
83
+
84
+ torch.cuda.reset_peak_memory_stats(device)
85
+ timings = []
86
+ for index in range(iters):
87
+ elapsed = _timed_infer(model, infer_kwargs, device)
88
+ timings.append(elapsed)
89
+ logger.info("%s iteration %d/%d: %.4fs", label, index + 1, iters, elapsed)
90
+ return timings, torch.cuda.max_memory_allocated(device) / 1024**3
91
+
92
+
93
+ @hydra.main(config_path="../configs", config_name="sim_libero.yaml", version_base="1.3")
94
+ def main(cfg: DictConfig) -> None:
95
+ logging.basicConfig(level=logging.INFO)
96
+ dryrun = _dryrun_config(cfg)
97
+ device = torch.device(str(cfg.EVALUATION.device))
98
+ model = instantiate(
99
+ cfg.model,
100
+ model_dtype=_model_dtype(cfg.mixed_precision),
101
+ device=str(device),
102
+ )
103
+ if cfg.ckpt is not None:
104
+ model.load_checkpoint(str(cfg.ckpt))
105
+ model.eval()
106
+ logger.info(
107
+ "device=%s gpu=%s dtype=%s",
108
+ device,
109
+ torch.cuda.get_device_name(device),
110
+ next(model.parameters()).dtype,
111
+ )
112
+
113
+ height, width = (int(value) for value in cfg.data.train.video_size)
114
+ action_horizon = (
115
+ int(cfg.data.train.num_frames) - 1
116
+ if dryrun.action_horizon is None
117
+ else int(dryrun.action_horizon)
118
+ )
119
+ num_inference_steps = (
120
+ int(cfg.eval_num_inference_steps)
121
+ if dryrun.num_inference_steps is None
122
+ else int(dryrun.num_inference_steps)
123
+ )
124
+ logger.info(
125
+ "action_horizon=%d num_inference_steps=%d warmup=%d iters=%d",
126
+ action_horizon,
127
+ num_inference_steps,
128
+ int(dryrun.warmup),
129
+ int(dryrun.iters),
130
+ )
131
+ torch.manual_seed(int(dryrun.seed))
132
+ input_image = torch.rand(1, 3, height, width) * 2 - 1
133
+ proprio_dim = int(cfg.data.train.processor.proprio_output_dim)
134
+ proprio = torch.randn(1, proprio_dim)
135
+ if bool(dryrun.use_random_context):
136
+ context = torch.randn(1, int(cfg.data.train.context_len), model.text_dim)
137
+ context_mask = torch.ones(context.shape[:2], dtype=torch.bool)
138
+ prompt = None
139
+ else:
140
+ context = None
141
+ context_mask = None
142
+ prompt = "pick up the object"
143
+
144
+ infer_kwargs = {
145
+ "prompt": prompt,
146
+ "input_image": input_image,
147
+ "action_horizon": action_horizon,
148
+ "proprio": proprio,
149
+ "context": context,
150
+ "context_mask": context_mask,
151
+ "num_inference_steps": num_inference_steps,
152
+ "seed": int(dryrun.seed),
153
+ "rand_device": str(cfg.EVALUATION.rand_device),
154
+ "compile_action_infer": bool(dryrun.compile_action_infer),
155
+ }
156
+ infer_parameters = inspect.signature(model.infer_action).parameters
157
+ if "num_video_frames" in infer_parameters:
158
+ infer_kwargs["num_video_frames"] = (
159
+ (int(cfg.data.train.num_frames) - 1)
160
+ // int(cfg.data.train.action_video_freq_ratio)
161
+ + 1
162
+ )
163
+ if "action_infer_mode" in infer_parameters:
164
+ infer_kwargs["action_infer_mode"] = str(dryrun.action_infer_mode)
165
+
166
+ warmup = int(dryrun.warmup)
167
+ iters = int(dryrun.iters)
168
+ if bool(dryrun.benchmark_both):
169
+ infer_kwargs["compile_action_infer"] = False
170
+ eager_times, eager_peak = _benchmark_mode(
171
+ model,
172
+ infer_kwargs,
173
+ device,
174
+ label="eager",
175
+ warmup=warmup,
176
+ iters=iters,
177
+ )
178
+
179
+ infer_kwargs["compile_action_infer"] = True
180
+ cold_compile_time = _timed_infer(model, infer_kwargs, device)
181
+ logger.info("compile cold start: %.4fs", cold_compile_time)
182
+ compile_times, compile_peak = _benchmark_mode(
183
+ model,
184
+ infer_kwargs,
185
+ device,
186
+ label="compile",
187
+ warmup=warmup,
188
+ iters=iters,
189
+ )
190
+ eager_mean = statistics.mean(eager_times)
191
+ compile_mean = statistics.mean(compile_times)
192
+ logger.info(
193
+ "comparison | eager_mean=%.4fs compile_mean=%.4fs speedup=%.3fx "
194
+ "compile_cold_start=%.4fs eager_peak=%.3fGiB compile_peak=%.3fGiB",
195
+ eager_mean,
196
+ compile_mean,
197
+ eager_mean / compile_mean,
198
+ cold_compile_time,
199
+ eager_peak,
200
+ compile_peak,
201
+ )
202
+ return
203
+
204
+ torch.cuda.reset_peak_memory_stats(device)
205
+ for index in range(warmup):
206
+ elapsed = _timed_infer(model, infer_kwargs, device)
207
+ logger.info("warmup %d/%d: %.4fs", index + 1, warmup, elapsed)
208
+
209
+ torch.cuda.reset_peak_memory_stats(device)
210
+ timings = []
211
+ for index in range(iters):
212
+ elapsed = _timed_infer(model, infer_kwargs, device)
213
+ timings.append(elapsed)
214
+ logger.info("iteration %d/%d: %.4fs", index + 1, iters, elapsed)
215
+ peak_gib = torch.cuda.max_memory_allocated(device) / 1024**3
216
+ logger.info(
217
+ "mean=%.4fs median=%.4fs min=%.4fs max=%.4fs peak_allocated=%.3fGiB",
218
+ statistics.mean(timings),
219
+ statistics.median(timings),
220
+ min(timings),
221
+ max(timings),
222
+ peak_gib,
223
+ )
224
+
225
+
226
+ if __name__ == "__main__":
227
+ main()
baseline/fastwam_zipper/source/scripts/ds_configs/ds_zero1_config.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "train_batch_size": "auto",
3
+ "train_micro_batch_size_per_gpu": "auto",
4
+ "gradient_accumulation_steps": "auto",
5
+ "zero_optimization": {
6
+ "stage": 1,
7
+ "offload_optimizer": {
8
+ "device": "none"
9
+ },
10
+ "offload_param": {
11
+ "device": "none"
12
+ },
13
+ "overlap_comm": false,
14
+ "contiguous_gradients": false,
15
+ "reduce_bucket_size": 2e8,
16
+ "allgather_bucket_size": 2e8
17
+ }
18
+ }
baseline/fastwam_zipper/source/scripts/ds_configs/ds_zero2_config.json ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "train_batch_size": "auto",
3
+ "train_micro_batch_size_per_gpu": "auto",
4
+ "gradient_accumulation_steps": "auto",
5
+ "gradient_clipping": 1.0,
6
+ "zero_optimization": {
7
+ "stage": 2,
8
+ "offload_optimizer": {
9
+ "device": "none"
10
+ },
11
+ "offload_param": {
12
+ "device": "none"
13
+ },
14
+ "overlap_comm": false,
15
+ "contiguous_gradients": false,
16
+ "reduce_bucket_size": 2e8,
17
+ "allgather_bucket_size": 2e8
18
+ }
19
+ }
baseline/fastwam_zipper/source/scripts/precompute_text_embeds.py ADDED
@@ -0,0 +1,389 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import hashlib
2
+ import json
3
+ import logging
4
+ import os
5
+ import re
6
+ import uuid
7
+ from pathlib import Path
8
+ from typing import Any
9
+
10
+ import hydra
11
+ import torch
12
+ import torch.distributed as dist
13
+ from omegaconf import DictConfig, ListConfig
14
+ from tqdm import tqdm
15
+
16
+ from fastwam.datasets.lerobot.robot_video_dataset import DEFAULT_PROMPT
17
+ from fastwam.models.wan22.helpers.loader import _load_registered_model, _resolve_configs
18
+ from fastwam.models.wan22.wan_video_text_encoder import HuggingfaceTokenizer
19
+ from fastwam.utils.config_resolvers import register_default_resolvers
20
+ from fastwam.utils.logging_config import get_logger, setup_logging
21
+
22
+ register_default_resolvers()
23
+ logger = get_logger(__name__)
24
+
25
+ DEFAULT_MODEL_ID = "Wan-AI/Wan2.2-TI2V-5B"
26
+ DEFAULT_TOKENIZER_MODEL_ID = "Wan-AI/Wan2.1-T2V-1.3B"
27
+ DEFAULT_CONTEXT_LEN = 128
28
+ DEFAULT_BATCH_SIZE = 16
29
+
30
+
31
+ def _init_distributed():
32
+ world_size = int(os.environ.get("WORLD_SIZE", "1"))
33
+ if world_size <= 1:
34
+ return False, 0, 1, 0
35
+
36
+ local_rank = int(os.environ.get("LOCAL_RANK", "0"))
37
+ backend = "nccl" if torch.cuda.is_available() else "gloo"
38
+ if torch.cuda.is_available():
39
+ torch.cuda.set_device(local_rank)
40
+
41
+ if not dist.is_initialized():
42
+ dist.init_process_group(backend=backend, init_method="env://")
43
+
44
+ return True, dist.get_rank(), dist.get_world_size(), local_rank
45
+
46
+
47
+ def _to_bool(value: Any) -> bool:
48
+ if isinstance(value, bool):
49
+ return value
50
+ if isinstance(value, str):
51
+ text = value.strip().lower()
52
+ if text in {"1", "true", "yes", "y"}:
53
+ return True
54
+ if text in {"0", "false", "no", "n"}:
55
+ return False
56
+ raise ValueError(f"Cannot parse bool value: {value}")
57
+
58
+
59
+ def _iter_dataset_nodes(node: Any, path: str = "data"):
60
+ if isinstance(node, DictConfig):
61
+ if "dataset_dirs" in node and node.get("dataset_dirs") is not None:
62
+ yield path, node
63
+ for key, value in node.items():
64
+ yield from _iter_dataset_nodes(value, f"{path}.{key}")
65
+ elif isinstance(node, ListConfig):
66
+ for idx, value in enumerate(node):
67
+ yield from _iter_dataset_nodes(value, f"{path}[{idx}]")
68
+
69
+
70
+ def _collect_dataset_settings(data_cfg: DictConfig):
71
+ dataset_dirs: list[str] = []
72
+ cache_dirs: list[Path] = []
73
+ context_lens = set()
74
+
75
+ for node_path, node in _iter_dataset_nodes(data_cfg, path="data"):
76
+ raw_dirs = node.get("dataset_dirs")
77
+ if raw_dirs is None:
78
+ continue
79
+
80
+ cache_dir = node.get("text_embedding_cache_dir")
81
+ if cache_dir is None or not str(cache_dir).strip():
82
+ raise ValueError(
83
+ f"Missing `text_embedding_cache_dir` for dataset node `{node_path}` "
84
+ "(this node defines `dataset_dirs`)."
85
+ )
86
+
87
+ for ds in raw_dirs:
88
+ ds_str = str(ds)
89
+ if ds_str not in dataset_dirs:
90
+ dataset_dirs.append(ds_str)
91
+
92
+ cache_dir_path = Path(str(cache_dir)).expanduser()
93
+ if cache_dir_path not in cache_dirs:
94
+ cache_dirs.append(cache_dir_path)
95
+
96
+ context_len = node.get("context_len")
97
+ if context_len is not None:
98
+ context_lens.add(int(context_len))
99
+
100
+ logger.info("Discovered dataset node `%s` with %d dataset_dirs.", node_path, len(raw_dirs))
101
+
102
+ return dataset_dirs, cache_dirs, context_lens
103
+
104
+
105
+ def _resolve_context_len(context_lens: set[int]) -> int:
106
+ if len(context_lens) != 1:
107
+ raise ValueError(
108
+ f"Found multiple context_len values in data config: {sorted(context_lens)}. "
109
+ "Please keep them consistent."
110
+ )
111
+ return next(iter(context_lens))
112
+
113
+
114
+ def _read_unique_prompts(dataset_dirs: list[str]) -> list[str]:
115
+ prompts: list[str] = []
116
+ seen = set()
117
+ total_task_rows = 0
118
+
119
+ for ds_dir in dataset_dirs:
120
+ tasks_path = Path(ds_dir) / "meta" / "tasks.jsonl"
121
+ if not tasks_path.exists():
122
+ raise FileNotFoundError(f"Missing tasks file: {tasks_path}")
123
+
124
+ with tasks_path.open("r", encoding="utf-8") as f:
125
+ for line_idx, line in enumerate(f, start=1):
126
+ line = line.strip()
127
+ if not line:
128
+ continue
129
+ record = json.loads(line)
130
+ if "task" not in record:
131
+ raise KeyError(f"Missing `task` field at {tasks_path}:{line_idx}")
132
+ task = str(record["task"])
133
+ prompt = DEFAULT_PROMPT.format(task=task)
134
+ total_task_rows += 1
135
+ if prompt not in seen:
136
+ seen.add(prompt)
137
+ prompts.append(prompt)
138
+
139
+ logger.info(
140
+ "Loaded %d task rows from %d datasets, deduplicated to %d prompts.",
141
+ total_task_rows,
142
+ len(dataset_dirs),
143
+ len(prompts),
144
+ )
145
+ return prompts
146
+
147
+
148
+ def _get_override_prompt(override_instruction: Any) -> str | None:
149
+ if override_instruction is None:
150
+ return None
151
+ task = str(override_instruction).strip()
152
+ if task == "":
153
+ return None
154
+ return DEFAULT_PROMPT.format(task=task)
155
+
156
+
157
+ def _model_id_to_enc_id(model_id: str) -> str:
158
+ base = str(model_id).split("/")[-1]
159
+ enc_id = re.sub(r"[^a-z0-9]+", "", base.lower())
160
+ return enc_id or "textenc"
161
+
162
+
163
+ def _atomic_torch_save(payload: dict[str, torch.Tensor], output_path: Path):
164
+ output_path.parent.mkdir(parents=True, exist_ok=True)
165
+ tmp_path = output_path.parent / f".{output_path.name}.tmp.{uuid.uuid4().hex}"
166
+ torch.save(payload, str(tmp_path))
167
+ os.replace(tmp_path, output_path)
168
+
169
+
170
+ @hydra.main(config_path="../configs", config_name="train", version_base="1.3")
171
+ def main(cfg: DictConfig):
172
+ setup_logging(log_level=logging.INFO)
173
+
174
+ is_distributed, rank, world_size, local_rank = _init_distributed()
175
+ if is_distributed and rank == 0:
176
+ logger.info("Distributed enabled: world_size=%d", world_size)
177
+ if (not is_distributed) and torch.cuda.is_available() and torch.cuda.device_count() > 1:
178
+ logger.info(
179
+ "Multi-GPU available. To use it, run: torchrun --standalone --nproc_per_node=%d scripts/precompute_text_embeds.py",
180
+ torch.cuda.device_count(),
181
+ )
182
+
183
+ overwrite = _to_bool(cfg.get("overwrite", True))
184
+ model_cfg = cfg.model
185
+ if model_cfg is None:
186
+ raise ValueError("`cfg.model` is required.")
187
+ if cfg.data is None:
188
+ raise ValueError("`cfg.data` is required.")
189
+
190
+ dataset_dirs, cache_dirs, context_lens = _collect_dataset_settings(cfg.data)
191
+ if not cache_dirs:
192
+ raise ValueError("No `text_embedding_cache_dir` found under `cfg.data`.")
193
+
194
+ context_len = _resolve_context_len(context_lens)
195
+ override_prompt = _get_override_prompt(cfg.get("override_instruction"))
196
+ if override_prompt is not None:
197
+ prompts = [override_prompt]
198
+ logger.info("Using override_instruction; skipping dataset scan and encoding exactly 1 prompt.")
199
+ else:
200
+ if not dataset_dirs:
201
+ raise ValueError("No `dataset_dirs` found under `cfg.data`.")
202
+ prompts = _read_unique_prompts(dataset_dirs)
203
+ if not prompts:
204
+ logger.warning("No prompts found from tasks.jsonl; nothing to do.")
205
+ return
206
+
207
+ if torch.cuda.is_available():
208
+ device = f"cuda:{local_rank}" if is_distributed else "cuda"
209
+ else:
210
+ device = "cpu"
211
+ torch_dtype = torch.bfloat16
212
+ model_id = str(model_cfg.get("model_id", DEFAULT_MODEL_ID))
213
+ tokenizer_model_id = str(model_cfg.get("tokenizer_model_id", DEFAULT_TOKENIZER_MODEL_ID))
214
+ redirect_common_files = bool(model_cfg.get("redirect_common_files", True))
215
+ enc_id = _model_id_to_enc_id(model_id)
216
+
217
+ logger.info(
218
+ "Preparing text encoder with model_id=%s tokenizer_model_id=%s device=%s dtype=%s context_len=%d overwrite=%s",
219
+ model_id,
220
+ tokenizer_model_id,
221
+ device,
222
+ torch_dtype,
223
+ context_len,
224
+ overwrite,
225
+ )
226
+
227
+ _, text_config, _, tokenizer_config = _resolve_configs(
228
+ model_id=model_id,
229
+ tokenizer_model_id=tokenizer_model_id,
230
+ redirect_common_files=redirect_common_files,
231
+ )
232
+ text_config.download_if_necessary()
233
+ tokenizer_config.download_if_necessary()
234
+
235
+ text_encoder = _load_registered_model(
236
+ text_config.path,
237
+ "wan_video_text_encoder",
238
+ torch_dtype=torch_dtype,
239
+ device=device,
240
+ ).eval()
241
+ tokenizer = HuggingfaceTokenizer(
242
+ name=tokenizer_config.path,
243
+ seq_len=context_len,
244
+ clean="whitespace",
245
+ )
246
+
247
+ stats = {
248
+ str(cache_dir): {"new": 0, "overwrite": 0, "skip": 0}
249
+ for cache_dir in cache_dirs
250
+ }
251
+
252
+ prompts = prompts[rank::world_size] if is_distributed else prompts
253
+
254
+ if not overwrite:
255
+ fully_cached_local = 0
256
+ prompts_to_encode: list[str] = []
257
+ for prompt in prompts:
258
+ hashed = hashlib.sha256(prompt.encode("utf-8")).hexdigest()
259
+ filename = f"{hashed}.t5_len{context_len}.{enc_id}.pt"
260
+ fully_cached = True
261
+ for cache_dir in cache_dirs:
262
+ cache_path = cache_dir / filename
263
+ if not cache_path.exists():
264
+ fully_cached = False
265
+ break
266
+ if fully_cached:
267
+ fully_cached_local += 1
268
+ for cache_dir in cache_dirs:
269
+ stats[str(cache_dir)]["skip"] += 1
270
+ else:
271
+ prompts_to_encode.append(prompt)
272
+
273
+ prompts = prompts_to_encode
274
+
275
+ fully_cached_global = fully_cached_local
276
+ to_encode_global = len(prompts)
277
+ if is_distributed:
278
+ reduce_device = torch.device(device) if device.startswith("cuda") else torch.device("cpu")
279
+ count_tensor = torch.tensor([fully_cached_local, len(prompts)], device=reduce_device, dtype=torch.long)
280
+ dist.all_reduce(count_tensor, op=dist.ReduceOp.SUM)
281
+ fully_cached_global = int(count_tensor[0].item())
282
+ to_encode_global = int(count_tensor[1].item())
283
+
284
+ if (not is_distributed) or rank == 0:
285
+ logger.info(
286
+ "overwrite=false: fully cached prompts=%d, prompts to encode=%d",
287
+ fully_cached_global,
288
+ to_encode_global,
289
+ )
290
+
291
+ logger.info("Writing caches to %d directories.", len(cache_dirs))
292
+ prompts_encoded_local = len(prompts)
293
+ prompts_encoded_global = prompts_encoded_local
294
+ if is_distributed:
295
+ reduce_device = torch.device(device) if device.startswith("cuda") else torch.device("cpu")
296
+ count_tensor = torch.tensor([prompts_encoded_local], device=reduce_device, dtype=torch.long)
297
+ dist.all_reduce(count_tensor, op=dist.ReduceOp.SUM)
298
+ prompts_encoded_global = int(count_tensor.item())
299
+
300
+ over_length_prompts = 0
301
+ with tqdm(
302
+ total=len(prompts),
303
+ desc=f"Encoding prompts (rank {rank}/{world_size})" if is_distributed else "Encoding prompts",
304
+ unit="prompt",
305
+ dynamic_ncols=True,
306
+ disable=is_distributed and rank != 0,
307
+ ) as pbar:
308
+ with torch.no_grad():
309
+ for start in range(0, len(prompts), DEFAULT_BATCH_SIZE):
310
+ batch_prompts = prompts[start : start + DEFAULT_BATCH_SIZE]
311
+ ids, mask = tokenizer(batch_prompts, return_mask=True, add_special_tokens=True)
312
+ ids = ids.to(device)
313
+ mask = mask.to(device=device, dtype=torch.bool)
314
+ over_length_prompts += int(mask.all(dim=1).sum().item())
315
+ context = text_encoder(ids, mask)
316
+
317
+ for i, prompt in enumerate(batch_prompts):
318
+ hashed = hashlib.sha256(prompt.encode("utf-8")).hexdigest()
319
+ context_i = context[i].detach().to(device="cpu", dtype=torch.bfloat16).contiguous()
320
+ mask_i = mask[i].detach().to(device="cpu", dtype=torch.bool).contiguous()
321
+ payload = {
322
+ "context": context_i,
323
+ "mask": mask_i,
324
+ }
325
+
326
+ for cache_dir in cache_dirs:
327
+ cache_path = cache_dir / f"{hashed}.t5_len{context_len}.{enc_id}.pt"
328
+ key = str(cache_dir)
329
+ if cache_path.exists() and not overwrite:
330
+ stats[key]["skip"] += 1
331
+ continue
332
+
333
+ if cache_path.exists():
334
+ stats[key]["overwrite"] += 1
335
+ else:
336
+ stats[key]["new"] += 1
337
+
338
+ _atomic_torch_save(payload, cache_path)
339
+
340
+ pbar.update(len(batch_prompts))
341
+
342
+ over_length_global = over_length_prompts
343
+ if is_distributed:
344
+ reduce_device = torch.device(device) if device.startswith("cuda") else torch.device("cpu")
345
+ over_tensor = torch.tensor([over_length_prompts], device=reduce_device, dtype=torch.long)
346
+ dist.all_reduce(over_tensor, op=dist.ReduceOp.SUM)
347
+ over_length_global = int(over_tensor.item())
348
+
349
+ counts_tensor = torch.tensor(
350
+ [
351
+ [stats[str(cache_dir)]["new"], stats[str(cache_dir)]["overwrite"], stats[str(cache_dir)]["skip"]]
352
+ for cache_dir in cache_dirs
353
+ ],
354
+ device=reduce_device,
355
+ dtype=torch.long,
356
+ )
357
+ dist.all_reduce(counts_tensor, op=dist.ReduceOp.SUM)
358
+ if rank == 0:
359
+ for idx, cache_dir in enumerate(cache_dirs):
360
+ key = str(cache_dir)
361
+ stats[key]["new"] = int(counts_tensor[idx, 0].item())
362
+ stats[key]["overwrite"] = int(counts_tensor[idx, 1].item())
363
+ stats[key]["skip"] = int(counts_tensor[idx, 2].item())
364
+
365
+ if (not is_distributed) or rank == 0:
366
+ logger.info("Finished precomputing text embeddings.")
367
+ logger.info(
368
+ "Over-length prompts (mask all True, i.e. no padding after truncation/max_length=%d): %d/%d",
369
+ context_len,
370
+ over_length_global,
371
+ prompts_encoded_global,
372
+ )
373
+ for cache_dir in cache_dirs:
374
+ key = str(cache_dir)
375
+ logger.info(
376
+ "Cache dir: %s | new=%d overwrite=%d skip=%d",
377
+ key,
378
+ stats[key]["new"],
379
+ stats[key]["overwrite"],
380
+ stats[key]["skip"],
381
+ )
382
+
383
+ if is_distributed and dist.is_initialized():
384
+ dist.barrier()
385
+ dist.destroy_process_group()
386
+
387
+
388
+ if __name__ == "__main__":
389
+ main()
baseline/fastwam_zipper/source/scripts/prepare_galaxea_r1_lite_overlay.py ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Build a metadata-only FastWAM view of a canonical Galaxea LeRobot-v2 pack.
3
+
4
+ FastWAM vendors a LeRobot v2.1 loader which requires every statistics ``count``
5
+ to have shape ``[1]``. The shared converter stores an equivalent count once per
6
+ vector component. This script collapses only that redundant field in a private
7
+ overlay; parquet and video payloads remain read-only symlinks to the canonical
8
+ pack.
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import argparse
14
+ import hashlib
15
+ import json
16
+ import os
17
+ import shutil
18
+ import tempfile
19
+ from pathlib import Path
20
+
21
+
22
+ MARKER_NAME = "fastwam_overlay.json"
23
+
24
+
25
+ def sha256(path: Path) -> str:
26
+ digest = hashlib.sha256()
27
+ with path.open("rb") as handle:
28
+ for chunk in iter(lambda: handle.read(1024 * 1024), b""):
29
+ digest.update(chunk)
30
+ return digest.hexdigest()
31
+
32
+
33
+ def metadata_hashes(meta: Path) -> dict[str, str]:
34
+ return {
35
+ path.relative_to(meta).as_posix(): sha256(path)
36
+ for path in sorted(meta.rglob("*"))
37
+ if path.is_file() and path.name != MARKER_NAME
38
+ }
39
+
40
+
41
+ def normalized_stats(source: Path) -> tuple[dict, list[str]]:
42
+ stats = json.loads(source.read_text(encoding="utf-8"))
43
+ changed: list[str] = []
44
+ for feature, values in stats.items():
45
+ if not isinstance(values, dict) or "count" not in values:
46
+ continue
47
+ count = values["count"]
48
+ if not isinstance(count, list) or not count:
49
+ raise ValueError(f"{source}: {feature}.count must be a nonempty list")
50
+ if len(count) == 1:
51
+ continue
52
+ if any(value != count[0] for value in count[1:]):
53
+ raise ValueError(f"{source}: {feature}.count is not redundant: {count}")
54
+ values["count"] = [count[0]]
55
+ changed.append(feature)
56
+ return stats, changed
57
+
58
+
59
+ def validate_existing(source: Path, destination: Path) -> bool:
60
+ marker_path = destination / "meta" / MARKER_NAME
61
+ if not marker_path.is_file():
62
+ return False
63
+ marker = json.loads(marker_path.read_text(encoding="utf-8"))
64
+ expected = {
65
+ "format": "fastwam-galaxea-metadata-overlay-v1",
66
+ "source": str(source.resolve()),
67
+ "source_meta_sha256": metadata_hashes(source / "meta"),
68
+ }
69
+ if any(marker.get(key) != value for key, value in expected.items()):
70
+ return False
71
+ for payload in ("data", "videos"):
72
+ link = destination / payload
73
+ if not link.is_symlink() or link.resolve() != (source / payload).resolve():
74
+ return False
75
+ return marker.get("overlay_meta_sha256") == metadata_hashes(destination / "meta")
76
+
77
+
78
+ def build_overlay(source: Path, destination: Path) -> dict:
79
+ source = source.resolve()
80
+ if not (source / "meta" / "stats.json").is_file():
81
+ raise FileNotFoundError(f"missing canonical metadata: {source / 'meta/stats.json'}")
82
+ for payload in ("data", "videos"):
83
+ if not (source / payload).exists():
84
+ raise FileNotFoundError(f"missing canonical payload: {source / payload}")
85
+
86
+ if destination.exists():
87
+ if validate_existing(source, destination):
88
+ return json.loads(
89
+ (destination / "meta" / MARKER_NAME).read_text(encoding="utf-8")
90
+ )
91
+ raise FileExistsError(
92
+ f"refusing stale or foreign overlay: {destination}; remove that exact "
93
+ "FastWAM overlay before rebuilding"
94
+ )
95
+
96
+ destination.parent.mkdir(parents=True, exist_ok=True)
97
+ staging = Path(
98
+ tempfile.mkdtemp(prefix=f".{destination.name}.tmp.", dir=destination.parent)
99
+ )
100
+ try:
101
+ shutil.copytree(source / "meta", staging / "meta")
102
+ stats, changed = normalized_stats(source / "meta" / "stats.json")
103
+ (staging / "meta" / "stats.json").write_text(
104
+ json.dumps(stats, indent=2, sort_keys=True) + "\n", encoding="utf-8"
105
+ )
106
+ for payload in ("data", "videos"):
107
+ os.symlink(source / payload, staging / payload, target_is_directory=True)
108
+
109
+ marker = {
110
+ "format": "fastwam-galaxea-metadata-overlay-v1",
111
+ "source": str(source),
112
+ "source_meta_sha256": metadata_hashes(source / "meta"),
113
+ "overlay_meta_sha256": metadata_hashes(staging / "meta"),
114
+ "transformation": {
115
+ "file": "meta/stats.json",
116
+ "operation": "collapse identical per-dimension count values to one scalar count",
117
+ "features": changed,
118
+ },
119
+ "payload_storage": "absolute read-only symlinks; no data or videos copied",
120
+ }
121
+ (staging / "meta" / MARKER_NAME).write_text(
122
+ json.dumps(marker, indent=2, sort_keys=True) + "\n", encoding="utf-8"
123
+ )
124
+ os.replace(staging, destination)
125
+ return marker
126
+ except BaseException:
127
+ shutil.rmtree(staging, ignore_errors=True)
128
+ raise
129
+
130
+
131
+ def main() -> None:
132
+ parser = argparse.ArgumentParser()
133
+ parser.add_argument("--source", type=Path, required=True)
134
+ parser.add_argument("--destination", type=Path, required=True)
135
+ args = parser.parse_args()
136
+ marker = build_overlay(args.source, args.destination)
137
+ print(json.dumps({"status": "fastwam_overlay_ok", **marker}, indent=2, sort_keys=True))
138
+
139
+
140
+ if __name__ == "__main__":
141
+ main()
baseline/fastwam_zipper/source/scripts/preprocess_action_dit_backbone.py ADDED
@@ -0,0 +1,240 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ from pathlib import Path
3
+ from typing import Any
4
+
5
+ import torch
6
+ import torch.nn.functional as F
7
+ from omegaconf import OmegaConf
8
+
9
+ from fastwam.models.wan22.action_dit import ActionDiT
10
+ from fastwam.models.wan22.helpers.loader import load_wan22_ti2v_5b_components
11
+
12
+
13
+ def _parse_dtype(name: str) -> torch.dtype:
14
+ value = str(name).strip().lower()
15
+ if value == "float32":
16
+ return torch.float32
17
+ if value == "float16":
18
+ return torch.float16
19
+ if value == "bfloat16":
20
+ return torch.bfloat16
21
+ raise ValueError(f"Unsupported dtype: {name}. Expected one of: float32, float16, bfloat16.")
22
+
23
+
24
+ def _parse_bool(name: str) -> bool:
25
+ value = str(name).strip().lower()
26
+ if value in {"1", "true", "yes", "y"}:
27
+ return True
28
+ if value in {"0", "false", "no", "n"}:
29
+ return False
30
+ raise ValueError(f"Cannot parse bool value: {name}")
31
+
32
+
33
+ def _is_unresolved_interpolation(value: Any) -> bool:
34
+ return isinstance(value, str) and "${" in value and "}" in value
35
+
36
+
37
+ def _resolve_from_video_cfg(value: Any, video_cfg: dict[str, Any]) -> Any:
38
+ if not _is_unresolved_interpolation(value):
39
+ return value
40
+ text = str(value).strip()
41
+ if not (text.startswith("${") and text.endswith("}")):
42
+ return value
43
+ expr = text[2:-1]
44
+ if not expr.startswith("video_dit_config."):
45
+ return value
46
+ key = expr.split(".", 1)[1]
47
+ if key not in video_cfg:
48
+ return value
49
+ resolved = video_cfg[key]
50
+ return value if _is_unresolved_interpolation(resolved) else resolved
51
+
52
+
53
+ def _interpolate_last_dim(tensor: torch.Tensor, new_size: int) -> torch.Tensor:
54
+ if tensor.shape[-1] == new_size:
55
+ return tensor
56
+ flat = tensor.reshape(-1, 1, tensor.shape[-1]).to(torch.float32)
57
+ flat = F.interpolate(flat, size=new_size, mode="linear", align_corners=True)
58
+ return flat.reshape(*tensor.shape[:-1], new_size)
59
+
60
+
61
+ def _resize_tensor_to_shape(src: torch.Tensor, target_shape: tuple[int, ...]) -> torch.Tensor:
62
+ if tuple(src.shape) == tuple(target_shape):
63
+ return src
64
+
65
+ out = src.to(torch.float32)
66
+ while out.ndim < len(target_shape):
67
+ out = out.unsqueeze(0)
68
+ while out.ndim > len(target_shape):
69
+ if out.shape[0] != 1:
70
+ raise ValueError(
71
+ f"Cannot reduce tensor rank for resize: src shape={tuple(src.shape)}, target={target_shape}"
72
+ )
73
+ out = out.squeeze(0)
74
+
75
+ for dim, new_size in enumerate(target_shape):
76
+ current_size = out.shape[dim]
77
+ if current_size == new_size:
78
+ continue
79
+ # Permute the target dimension to the end for interpolation
80
+ perm = [i for i in range(out.ndim) if i != dim] + [dim]
81
+ # Construct inverse permutation to restore original order
82
+ inv_perm = [0] * out.ndim
83
+ for i, p in enumerate(perm):
84
+ inv_perm[p] = i
85
+ # Permute, interpolate, and restore original order
86
+ out_perm = out.permute(*perm).contiguous()
87
+ prefix_shape = out_perm.shape[:-1]
88
+ out_perm = _interpolate_last_dim(out_perm, new_size)
89
+ out_perm = out_perm.reshape(*prefix_shape, new_size)
90
+ out = out_perm.permute(*inv_perm).contiguous()
91
+
92
+ if tuple(out.shape) != tuple(target_shape):
93
+ raise ValueError(
94
+ f"Resize produced wrong shape for tensor. src={tuple(src.shape)}, target={target_shape}, got={tuple(out.shape)}"
95
+ )
96
+ return out.to(dtype=src.dtype)
97
+
98
+
99
+ def _load_model_config(path: Path) -> tuple[dict[str, Any], dict[str, Any]]:
100
+ cfg = OmegaConf.load(str(path))
101
+ if "video_dit_config" not in cfg or "action_dit_config" not in cfg:
102
+ raise ValueError(
103
+ f"`{path}` must contain both `video_dit_config` and `action_dit_config` at top level."
104
+ )
105
+
106
+ video_cfg = OmegaConf.to_container(cfg.video_dit_config, resolve=False)
107
+ action_cfg = OmegaConf.to_container(cfg.action_dit_config, resolve=False)
108
+ if not isinstance(video_cfg, dict) or not isinstance(action_cfg, dict):
109
+ raise ValueError("`video_dit_config` and `action_dit_config` must resolve to dicts.")
110
+
111
+ if _is_unresolved_interpolation(video_cfg.get("action_dim")):
112
+ print("[WARN] `video_dit_config.action_dim` is unresolved; defaulting to 7 for preprocessing.")
113
+ video_cfg["action_dim"] = 7
114
+
115
+ if _is_unresolved_interpolation(action_cfg.get("action_dim")):
116
+ print("[WARN] `action_dit_config.action_dim` is unresolved; defaulting to 7 for preprocessing.")
117
+ action_cfg["action_dim"] = 7
118
+
119
+ for key in ["num_heads", "attn_head_dim", "num_layers", "text_dim", "freq_dim"]:
120
+ action_cfg[key] = _resolve_from_video_cfg(action_cfg.get(key), video_cfg)
121
+
122
+ return video_cfg, action_cfg, cfg
123
+
124
+
125
+ def _require_int_config(cfg: dict[str, Any], key: str) -> int:
126
+ value = cfg.get(key)
127
+ if _is_unresolved_interpolation(value):
128
+ raise ValueError(f"`{key}` is unresolved interpolation: {value}")
129
+ return int(value)
130
+
131
+
132
+ def _require_float_config(cfg: dict[str, Any], key: str) -> float:
133
+ value = cfg.get(key)
134
+ if _is_unresolved_interpolation(value):
135
+ raise ValueError(f"`{key}` is unresolved interpolation: {value}")
136
+ return float(value)
137
+
138
+
139
+ def main() -> None:
140
+ parser = argparse.ArgumentParser(
141
+ description="Preprocess ActionDiT backbone weights from WanVideoDiT and save as .pt payload."
142
+ )
143
+ parser.add_argument("--model-config", required=True, help="Path to model yaml, e.g. configs/model/fastwam.yaml")
144
+ parser.add_argument("--output", required=True, help="Output .pt path for preprocessed ActionDiT backbone.")
145
+ parser.add_argument("--device", default="cpu", help="Device for loading model and preprocessing.")
146
+ parser.add_argument("--dtype", default="float32", choices=["float32", "float16", "bfloat16"])
147
+ parser.add_argument(
148
+ "--apply-alpha-scaling",
149
+ default="true",
150
+ help="Whether to apply alpha=sqrt(dv/da) when the last dimension is resized (true/false). Default: true.",
151
+ )
152
+ args = parser.parse_args()
153
+
154
+ model_config_path = Path(args.model_config)
155
+ output_path = Path(args.output)
156
+ output_path.parent.mkdir(parents=True, exist_ok=True)
157
+ apply_alpha_scaling = _parse_bool(args.apply_alpha_scaling)
158
+
159
+ video_cfg, action_cfg, cfg = _load_model_config(model_config_path)
160
+ torch_dtype = _parse_dtype(args.dtype)
161
+ redirect_common_files = _parse_bool(cfg.get("redirect_common_files", False))
162
+
163
+ int_fields = ["hidden_dim", "action_dim", "ffn_dim", "num_layers", "num_heads", "attn_head_dim", "text_dim", "freq_dim"]
164
+ for key in int_fields:
165
+ action_cfg[key] = _require_int_config(action_cfg, key)
166
+ action_cfg["eps"] = _require_float_config(action_cfg, "eps")
167
+
168
+ print(f"[INFO] Loaded model config from {model_config_path}. "
169
+ f"Preprocessing ActionDiT backbone with dtype={torch_dtype} on device={args.device}, "
170
+ f"apply_alpha_scaling={apply_alpha_scaling}.")
171
+ components = load_wan22_ti2v_5b_components(
172
+ device=args.device,
173
+ torch_dtype=torch_dtype,
174
+ model_id=cfg.get("model_id", "Wan-AI/Wan2.2-TI2V-5B"),
175
+ tokenizer_model_id=cfg.get("tokenizer_model_id", "Wan-AI/Wan2.1-T2V-1.3B"),
176
+ redirect_common_files=redirect_common_files,
177
+ dit_config=video_cfg,
178
+ )
179
+ video_expert = components.dit
180
+
181
+ action_expert = ActionDiT(**action_cfg).to(device=args.device, dtype=torch_dtype)
182
+ if int(action_cfg["num_heads"]) != int(video_expert.num_heads):
183
+ raise ValueError("ActionDiT `num_heads` must match video expert for MoT mixed attention.")
184
+ if int(action_cfg["attn_head_dim"]) != int(video_expert.attn_head_dim):
185
+ raise ValueError("ActionDiT `attn_head_dim` must match video expert for MoT mixed attention.")
186
+ if int(action_cfg["num_layers"]) != int(len(video_expert.blocks)):
187
+ raise ValueError("ActionDiT `num_layers` must match video expert.")
188
+
189
+ action_state = action_expert.state_dict()
190
+ video_state = video_expert.state_dict()
191
+ backbone_keys = ActionDiT.backbone_key_set(action_state.keys())
192
+
193
+ backbone_state_dict: dict[str, torch.Tensor] = {}
194
+ copied = 0
195
+ interpolated = 0
196
+ for key in sorted(backbone_keys):
197
+ if key not in video_state:
198
+ raise ValueError(f"Key `{key}` not found in video expert state dict.")
199
+ src = video_state[key]
200
+ target = action_state[key]
201
+ if tuple(src.shape) == tuple(target.shape):
202
+ value = src
203
+ copied += 1
204
+ else:
205
+ value = _resize_tensor_to_shape(src, tuple(target.shape))
206
+ if apply_alpha_scaling and src.ndim >= 2 and src.shape[-1] != target.shape[-1]:
207
+ alpha = (float(src.shape[-1]) / float(target.shape[-1])) ** 0.5
208
+ value = value.to(torch.float32) * alpha
209
+ interpolated += 1
210
+ backbone_state_dict[key] = value.detach().to(dtype=target.dtype, device="cpu").contiguous()
211
+
212
+ payload = {
213
+ "policy": {
214
+ "skip_prefixes": list(ActionDiT.ACTION_BACKBONE_SKIP_PREFIXES),
215
+ "alpha_scaling": bool(apply_alpha_scaling),
216
+ "interpolation": "sequential_1d_linear_align_corners_true",
217
+ },
218
+ "backbone_state_dict": backbone_state_dict,
219
+ "meta": {
220
+ "hidden_dim": int(action_cfg["hidden_dim"]),
221
+ "ffn_dim": int(action_cfg["ffn_dim"]),
222
+ "num_layers": int(action_cfg["num_layers"]),
223
+ "num_heads": int(action_cfg["num_heads"]),
224
+ "attn_head_dim": int(action_cfg["attn_head_dim"]),
225
+ "text_dim": int(action_cfg["text_dim"]),
226
+ "freq_dim": int(action_cfg["freq_dim"]),
227
+ "eps": float(action_cfg["eps"]),
228
+ },
229
+ }
230
+ torch.save(payload, str(output_path))
231
+
232
+ skipped = len(action_state) - len(backbone_keys)
233
+ print(
234
+ "[INFO] Saved ActionDiT backbone payload to "
235
+ f"{output_path} (copied={copied}, interpolated={interpolated}, skipped={skipped})."
236
+ )
237
+
238
+
239
+ if __name__ == "__main__":
240
+ main()
baseline/fastwam_zipper/source/scripts/train.py ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import hydra
2
+ from omegaconf import DictConfig
3
+
4
+ from fastwam.runtime import run_training
5
+ from fastwam.utils.config_resolvers import register_default_resolvers
6
+
7
+ register_default_resolvers()
8
+
9
+
10
+ @hydra.main(config_path="../configs", config_name="train", version_base="1.3")
11
+ def main(cfg: DictConfig):
12
+ run_training(cfg)
13
+
14
+
15
+ if __name__ == "__main__":
16
+ main()
baseline/fastwam_zipper/source/scripts/train_galaxea_r1_lite.sh ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+
4
+ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
5
+ RUNTIME_ROOT="${FASTWAM_RUNTIME_ROOT:-/code/fastwam-runtime-v1}"
6
+ PYTHON="${RUNTIME_ROOT}/bin/python"
7
+ NPROC_PER_NODE="${1:?usage: scripts/train_galaxea_r1_lite.sh NPROC_PER_NODE RUN_NAME [hydra overrides...]}"
8
+ RUN_NAME="${2:?usage: scripts/train_galaxea_r1_lite.sh NPROC_PER_NODE RUN_NAME [hydra overrides...]}"
9
+ shift 2
10
+
11
+ CANONICAL_TRAIN_DATA="${GALAXEA_CANONICAL_TRAIN_DATA:-/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/pi05_lerobot_v2/train}"
12
+ CANONICAL_VAL_DATA="${GALAXEA_CANONICAL_VAL_DATA:-/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/pi05_lerobot_v2/validation}"
13
+ TRAIN_DATA="${GALAXEA_FASTWAM_TRAIN_DATA:-/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/fastwam_lerobot_v2/train}"
14
+ VAL_DATA="${GALAXEA_FASTWAM_VAL_DATA:-/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/fastwam_lerobot_v2/validation}"
15
+ MANIFEST="${GALAXEA_MANIFEST:-${ROOT}/../reports/galaxea_baselines/source_protocol/episodes_final.jsonl}"
16
+ BASE_CKPT="${FASTWAM_BASE_CHECKPOINT:-/dataset_rc_o1_p1/qshou@xiaopeng.com/weights/real_world_baselines/fastwam/robotwin_uncond_3cam_384.pt}"
17
+ CKPT_MARKER="$(dirname "${BASE_CKPT}")/DOWNLOAD_VERIFIED.json"
18
+ RUN_ROOT="${GALAXEA_FASTWAM_RUN_ROOT:-/dataset_rc_o1_p1/qshou@xiaopeng.com/training_runs/galaxea_real_baselines/fastwam}"
19
+ OUTPUT_DIR="${RUN_ROOT}/${RUN_NAME}"
20
+ EXPECTED_CKPT_SIZE=12041813092
21
+ EXPECTED_CKPT_SHA256="776475b22566a791854ecf31cf3b50f25e7d8d94c343132ec16eb94994aa9e63"
22
+ EXPECTED_TRAIN_EPISODES="${GALAXEA_EXPECTED_TRAIN_EPISODES:-511}"
23
+ EXPECTED_VAL_EPISODES="${GALAXEA_EXPECTED_VAL_EPISODES:-57}"
24
+ NUM_EPOCHS="${NUM_EPOCHS:-10}"
25
+ PER_DEVICE_BATCH_SIZE="${PER_DEVICE_BATCH_SIZE:-8}"
26
+ GRADIENT_ACCUMULATION_STEPS="${GRADIENT_ACCUMULATION_STEPS:-2}"
27
+ MIN_FREE_GIB="${GALAXEA_RUN_MIN_FREE_GIB:-2048}"
28
+
29
+ [[ "${NPROC_PER_NODE}" =~ ^[1-9][0-9]*$ ]] || { echo "invalid process count" >&2; exit 2; }
30
+ [[ "${RUN_NAME}" =~ ^[A-Za-z0-9._-]+$ ]] || { echo "unsafe run name" >&2; exit 2; }
31
+ [[ "${NUM_EPOCHS}" =~ ^[1-9][0-9]*$ ]] || { echo "invalid NUM_EPOCHS" >&2; exit 2; }
32
+ [[ "${PER_DEVICE_BATCH_SIZE}" =~ ^[1-9][0-9]*$ ]] || { echo "invalid PER_DEVICE_BATCH_SIZE" >&2; exit 2; }
33
+ [[ "${GRADIENT_ACCUMULATION_STEPS}" =~ ^[1-9][0-9]*$ ]] || { echo "invalid GRADIENT_ACCUMULATION_STEPS" >&2; exit 2; }
34
+ [[ -x "${PYTHON}" ]] || { echo "missing FastWAM runtime: ${PYTHON}" >&2; exit 2; }
35
+ [[ -f "${BASE_CKPT}" ]] || { echo "missing FastWAM checkpoint: ${BASE_CKPT}" >&2; exit 2; }
36
+ [[ "$(stat -c %s "${BASE_CKPT}")" = "${EXPECTED_CKPT_SIZE}" ]] || { echo "FastWAM checkpoint size mismatch" >&2; exit 2; }
37
+ [[ "$(sha256sum "${BASE_CKPT}" | awk '{print $1}')" = "${EXPECTED_CKPT_SHA256}" ]] || { echo "FastWAM checkpoint SHA256 mismatch" >&2; exit 2; }
38
+ [[ -f "${CKPT_MARKER}" ]] || { echo "missing FastWAM checkpoint verification marker" >&2; exit 2; }
39
+ "${PYTHON}" - "${BASE_CKPT}" "${CKPT_MARKER}" <<'PY'
40
+ import json
41
+ import pathlib
42
+ import sys
43
+
44
+ checkpoint = pathlib.Path(sys.argv[1])
45
+ marker = json.loads(pathlib.Path(sys.argv[2]).read_text(encoding="utf-8"))
46
+ if marker.get("file") != checkpoint.name:
47
+ raise SystemExit("FastWAM checkpoint marker filename mismatch")
48
+ if int(marker.get("size_bytes", -1)) != checkpoint.stat().st_size:
49
+ raise SystemExit("FastWAM checkpoint marker size mismatch")
50
+ if marker.get("sha256") != "776475b22566a791854ecf31cf3b50f25e7d8d94c343132ec16eb94994aa9e63":
51
+ raise SystemExit("FastWAM checkpoint marker SHA256 mismatch")
52
+ PY
53
+ mkdir -p "${RUN_ROOT}"
54
+ available_kib="$(df -Pk "${RUN_ROOT}" | awk 'NR==2 {print $4}')"
55
+ required_kib="$((MIN_FREE_GIB * 1024 * 1024))"
56
+ (( available_kib >= required_kib )) || {
57
+ echo "insufficient run space: available_kib=${available_kib} required_kib=${required_kib}" >&2; exit 4;
58
+ }
59
+
60
+ resume_path="${BASE_CKPT}"
61
+ if [[ -e "${OUTPUT_DIR}" ]]; then
62
+ latest_state="$(find "${OUTPUT_DIR}/checkpoints/state" -mindepth 1 -maxdepth 1 -type d -name 'step_[0-9]*' -printf '%f\n' 2>/dev/null | sort | tail -n 1)"
63
+ [[ -n "${latest_state}" ]] || {
64
+ echo "existing output has no resumable state: ${OUTPUT_DIR}" >&2; exit 3;
65
+ }
66
+ resume_path="${OUTPUT_DIR}/checkpoints/state/${latest_state}"
67
+ fi
68
+
69
+ export GALAXEA_FASTWAM_TRAIN_DATA="${TRAIN_DATA}"
70
+ export GALAXEA_FASTWAM_VAL_DATA="${VAL_DATA}"
71
+ export DIFFSYNTH_MODEL_BASE_PATH="${DIFFSYNTH_MODEL_BASE_PATH:-/workspace-wl/shouqx@xiaopeng.com/model}"
72
+ export DIFFSYNTH_SKIP_DOWNLOAD=true
73
+ export PATH="${RUNTIME_ROOT}/bin:${PATH}"
74
+ export WANDB_MODE="${WANDB_MODE:-online}"
75
+ [[ "${WANDB_MODE}" != online || -n "${WANDB_API_KEY:-}" ]] || {
76
+ echo "WANDB_MODE=online requires WANDB_API_KEY" >&2; exit 2;
77
+ }
78
+
79
+ cd "${ROOT}"
80
+ "${PYTHON}" scripts/prepare_galaxea_r1_lite_overlay.py \
81
+ --source "${CANONICAL_TRAIN_DATA}" --destination "${TRAIN_DATA}"
82
+ "${PYTHON}" scripts/prepare_galaxea_r1_lite_overlay.py \
83
+ --source "${CANONICAL_VAL_DATA}" --destination "${VAL_DATA}"
84
+ "${PYTHON}" scripts/check_galaxea_r1_lite_contract.py \
85
+ --train "${TRAIN_DATA}" --validation "${VAL_DATA}" --manifest "${MANIFEST}" \
86
+ --expected-train-episodes "${EXPECTED_TRAIN_EPISODES}" \
87
+ --expected-validation-episodes "${EXPECTED_VAL_EPISODES}"
88
+
89
+ effective_global_batch_size="$((PER_DEVICE_BATCH_SIZE * NPROC_PER_NODE * GRADIENT_ACCUMULATION_STEPS))"
90
+ steps_per_epoch="$("${PYTHON}" "${ROOT}/../scripts/plan_galaxea_epochs.py" \
91
+ "${TRAIN_DATA}" --epochs "${NUM_EPOCHS}" \
92
+ --effective-global-batch-size "${effective_global_batch_size}" \
93
+ --field steps_per_epoch)"
94
+
95
+ if [[ "${FASTWAM_PREFLIGHT_ONLY:-0}" = "1" ]]; then
96
+ echo "GALEXEA_FASTWAM_PREFLIGHT_OK"
97
+ exit 0
98
+ fi
99
+
100
+ echo "GALAXEA_FASTWAM_LAUNCH gpus=${NPROC_PER_NODE} effective_gbs=${effective_global_batch_size} epochs=${NUM_EPOCHS} steps_per_epoch=${steps_per_epoch} dataset=${TRAIN_DATA}"
101
+
102
+ if [[ "${FASTWAM_SKIP_TEXT_PRECOMPUTE:-0}" == "1" ]]; then
103
+ "${PYTHON}" scripts/check_galaxea_text_cache.py \
104
+ --train "${TRAIN_DATA}" --validation "${VAL_DATA}" \
105
+ --cache "${GALAXEA_FASTWAM_TEXT_CACHE:-/dataset_rc_o1_p1/qshou@xiaopeng.com/dataset/galaxea_real_baselines_derived/fastwam_text_cache}"
106
+ else
107
+ "${PYTHON}" -m torch.distributed.run --standalone --nproc_per_node="${NPROC_PER_NODE}" \
108
+ scripts/precompute_text_embeds.py task=galaxea_r1_lite_3cam_1e-4 +overwrite=false
109
+ fi
110
+
111
+ RUN_ID="${RUN_NAME}" bash scripts/train_zero1.sh "${NPROC_PER_NODE}" \
112
+ task=galaxea_r1_lite_3cam_1e-4 \
113
+ "output_dir=${OUTPUT_DIR}" \
114
+ "resume=${resume_path}" \
115
+ "num_epochs=${NUM_EPOCHS}" \
116
+ "batch_size=${PER_DEVICE_BATCH_SIZE}" \
117
+ "gradient_accumulation_steps=${GRADIENT_ACCUMULATION_STEPS}" \
118
+ "save_every=${steps_per_epoch}" \
119
+ "eval_every=${steps_per_epoch}" \
120
+ "wandb.enabled=true" \
121
+ "wandb.mode=online" \
122
+ "wandb.project=galaxea-real-baselines" \
123
+ "wandb.group=${GALAXEA_WANDB_GROUP:-galaxea-real-baselines-10ep}" \
124
+ "wandb.name=${RUN_NAME}" \
125
+ "$@"
baseline/fastwam_zipper/source/scripts/train_zero1.sh ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+
4
+ NPROC_PER_NODE="${1:?Usage: bash scripts/train_zero1.sh <nproc_per_node> [hydra_overrides...]}"
5
+ shift
6
+
7
+ EXTRA_ARGS=("$@")
8
+ NUM_MACHINES="${NNODES:-1}"
9
+ MACHINE_RANK="${NODE_RANK:-0}"
10
+ MAIN_PROCESS_IP="${MASTER_ADDR:-127.0.0.1}"
11
+ MAIN_PROCESS_PORT="${MASTER_PORT:-29500}"
12
+
13
+ is_integer() {
14
+ [[ "${1}" =~ ^[0-9]+$ ]]
15
+ }
16
+
17
+ if ! is_integer "${NUM_MACHINES}" || ! is_integer "${MACHINE_RANK}"; then
18
+ echo "Error: NUM_MACHINES (${NUM_MACHINES}) and MACHINE_RANK (${MACHINE_RANK}) must be integers." >&2
19
+ exit 1
20
+ fi
21
+
22
+ extract_task_basename() {
23
+ local cfg="$1"
24
+ if [[ "${cfg}" == task/* ]]; then
25
+ local name="${cfg#task/}"
26
+ name="${name%.yaml}"
27
+ echo "${name}"
28
+ return 0
29
+ fi
30
+ return 1
31
+ }
32
+
33
+ TASK_BASENAME="train"
34
+ for ((i = 0; i < ${#EXTRA_ARGS[@]}; i++)); do
35
+ arg="${EXTRA_ARGS[$i]}"
36
+ case "${arg}" in
37
+ --config-name)
38
+ if ((i + 1 < ${#EXTRA_ARGS[@]})); then
39
+ next="${EXTRA_ARGS[$((i + 1))]}"
40
+ if parsed="$(extract_task_basename "${next}")"; then
41
+ TASK_BASENAME="${parsed}"
42
+ fi
43
+ fi
44
+ ;;
45
+ --config-name=*)
46
+ cfg="${arg#--config-name=}"
47
+ if parsed="$(extract_task_basename "${cfg}")"; then
48
+ TASK_BASENAME="${parsed}"
49
+ fi
50
+ ;;
51
+ task=*)
52
+ cfg="${arg#task=}"
53
+ cfg="${cfg%.yaml}"
54
+ TASK_BASENAME="${cfg}"
55
+ ;;
56
+ esac
57
+ done
58
+
59
+ if [[ -z "${RUN_ID:-}" ]]; then
60
+ if (( NUM_MACHINES <= 1 )); then
61
+ RUN_ID="$(date +%Y-%m-%d_%H-%M-%S)"
62
+ else
63
+ RUN_ID_SYNC_TIMEOUT="${RUN_ID_SYNC_TIMEOUT:-180}"
64
+ RUN_ID_SYNC_PORT="${RUN_ID_SYNC_PORT:-$((MAIN_PROCESS_PORT + 11))}"
65
+
66
+ export RUN_ID_SYNC_HOST="${MAIN_PROCESS_IP}"
67
+ export RUN_ID_SYNC_PORT
68
+ export RUN_ID_SYNC_TIMEOUT
69
+ export RUN_ID_SYNC_MACHINE_RANK="${MACHINE_RANK}"
70
+ export RUN_ID_SYNC_NUM_MACHINES="${NUM_MACHINES}"
71
+ export RUN_ID_SYNC_TASK_BASENAME="${TASK_BASENAME}"
72
+
73
+ RUN_ID="$(
74
+ python - <<'PY'
75
+ import datetime
76
+ import os
77
+ from datetime import timedelta
78
+
79
+ import torch.distributed as dist
80
+
81
+ host = os.environ["RUN_ID_SYNC_HOST"]
82
+ port = int(os.environ["RUN_ID_SYNC_PORT"])
83
+ timeout_s = int(os.environ["RUN_ID_SYNC_TIMEOUT"])
84
+ machine_rank = int(os.environ["RUN_ID_SYNC_MACHINE_RANK"])
85
+ num_machines = int(os.environ["RUN_ID_SYNC_NUM_MACHINES"])
86
+ task_basename = os.environ.get("RUN_ID_SYNC_TASK_BASENAME", "train")
87
+
88
+ store = dist.TCPStore(
89
+ host_name=host,
90
+ port=port,
91
+ world_size=num_machines,
92
+ is_master=(machine_rank == 0),
93
+ timeout=timedelta(seconds=timeout_s),
94
+ )
95
+ key = f"run_id::{task_basename}"
96
+ if machine_rank == 0:
97
+ run_id = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
98
+ store.set(key, run_id)
99
+ run_id = store.get(key).decode("utf-8")
100
+ print(run_id)
101
+ PY
102
+ )"
103
+
104
+ echo "[run_id_sync] mode=tcpstore host=${RUN_ID_SYNC_HOST} port=${RUN_ID_SYNC_PORT} timeout_s=${RUN_ID_SYNC_TIMEOUT} run_id=${RUN_ID}"
105
+ fi
106
+ fi
107
+
108
+ echo "[launch] nproc_per_node=${NPROC_PER_NODE} num_machines=${NUM_MACHINES} machine_rank=${MACHINE_RANK} run_id=${RUN_ID}"
109
+
110
+ python -m accelerate.commands.launch \
111
+ --config_file scripts/accelerate_configs/accelerate_zero1_ds.yaml \
112
+ --num_processes "${NPROC_PER_NODE}" \
113
+ scripts/train.py \
114
+ "output_dir=./runs/${TASK_BASENAME}/${RUN_ID}" \
115
+ "wandb.name=${TASK_BASENAME}" \
116
+ "${EXTRA_ARGS[@]}"
baseline/fastwam_zipper/source/scripts/train_zero2.sh ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+
4
+ NPROC_PER_NODE="${1:?Usage: bash scripts/train.sh <nproc_per_node> [hydra_overrides...]}"
5
+ shift
6
+
7
+ EXTRA_ARGS=("$@")
8
+ NUM_MACHINES="${NNODES:-1}"
9
+ MACHINE_RANK="${NODE_RANK:-0}"
10
+ MAIN_PROCESS_IP="${MASTER_ADDR:-127.0.0.1}"
11
+ MAIN_PROCESS_PORT="${MASTER_PORT:-29500}"
12
+
13
+ is_integer() {
14
+ [[ "${1}" =~ ^[0-9]+$ ]]
15
+ }
16
+
17
+ if ! is_integer "${NUM_MACHINES}" || ! is_integer "${MACHINE_RANK}"; then
18
+ echo "Error: NUM_MACHINES (${NUM_MACHINES}) and MACHINE_RANK (${MACHINE_RANK}) must be integers." >&2
19
+ exit 1
20
+ fi
21
+
22
+ extract_task_basename() {
23
+ local cfg="$1"
24
+ if [[ "${cfg}" == task/* ]]; then
25
+ local name="${cfg#task/}"
26
+ name="${name%.yaml}"
27
+ echo "${name}"
28
+ return 0
29
+ fi
30
+ return 1
31
+ }
32
+
33
+ TASK_BASENAME="train"
34
+ for ((i = 0; i < ${#EXTRA_ARGS[@]}; i++)); do
35
+ arg="${EXTRA_ARGS[$i]}"
36
+ case "${arg}" in
37
+ --config-name)
38
+ if ((i + 1 < ${#EXTRA_ARGS[@]})); then
39
+ next="${EXTRA_ARGS[$((i + 1))]}"
40
+ if parsed="$(extract_task_basename "${next}")"; then
41
+ TASK_BASENAME="${parsed}"
42
+ fi
43
+ fi
44
+ ;;
45
+ --config-name=*)
46
+ cfg="${arg#--config-name=}"
47
+ if parsed="$(extract_task_basename "${cfg}")"; then
48
+ TASK_BASENAME="${parsed}"
49
+ fi
50
+ ;;
51
+ task=*)
52
+ cfg="${arg#task=}"
53
+ cfg="${cfg%.yaml}"
54
+ TASK_BASENAME="${cfg}"
55
+ ;;
56
+ esac
57
+ done
58
+
59
+ if [[ -z "${RUN_ID:-}" ]]; then
60
+ if (( NUM_MACHINES <= 1 )); then
61
+ RUN_ID="$(date +%Y-%m-%d_%H-%M-%S)"
62
+ else
63
+ RUN_ID_SYNC_TIMEOUT="${RUN_ID_SYNC_TIMEOUT:-180}"
64
+ RUN_ID_SYNC_PORT="${RUN_ID_SYNC_PORT:-$((MAIN_PROCESS_PORT + 11))}"
65
+
66
+ export RUN_ID_SYNC_HOST="${MAIN_PROCESS_IP}"
67
+ export RUN_ID_SYNC_PORT
68
+ export RUN_ID_SYNC_TIMEOUT
69
+ export RUN_ID_SYNC_MACHINE_RANK="${MACHINE_RANK}"
70
+ export RUN_ID_SYNC_NUM_MACHINES="${NUM_MACHINES}"
71
+ export RUN_ID_SYNC_TASK_BASENAME="${TASK_BASENAME}"
72
+
73
+ RUN_ID="$(
74
+ python - <<'PY'
75
+ import datetime
76
+ import os
77
+ from datetime import timedelta
78
+
79
+ import torch.distributed as dist
80
+
81
+ host = os.environ["RUN_ID_SYNC_HOST"]
82
+ port = int(os.environ["RUN_ID_SYNC_PORT"])
83
+ timeout_s = int(os.environ["RUN_ID_SYNC_TIMEOUT"])
84
+ machine_rank = int(os.environ["RUN_ID_SYNC_MACHINE_RANK"])
85
+ num_machines = int(os.environ["RUN_ID_SYNC_NUM_MACHINES"])
86
+ task_basename = os.environ.get("RUN_ID_SYNC_TASK_BASENAME", "train")
87
+
88
+ store = dist.TCPStore(
89
+ host_name=host,
90
+ port=port,
91
+ world_size=num_machines,
92
+ is_master=(machine_rank == 0),
93
+ timeout=timedelta(seconds=timeout_s),
94
+ )
95
+ key = f"run_id::{task_basename}"
96
+ if machine_rank == 0:
97
+ run_id = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
98
+ store.set(key, run_id)
99
+ run_id = store.get(key).decode("utf-8")
100
+ print(run_id)
101
+ PY
102
+ )"
103
+
104
+ echo "[run_id_sync] mode=tcpstore host=${RUN_ID_SYNC_HOST} port=${RUN_ID_SYNC_PORT} timeout_s=${RUN_ID_SYNC_TIMEOUT} run_id=${RUN_ID}"
105
+ fi
106
+ fi
107
+
108
+ echo "[launch] nproc_per_node=${NPROC_PER_NODE} num_machines=${NUM_MACHINES} machine_rank=${MACHINE_RANK} run_id=${RUN_ID}"
109
+
110
+ python -m accelerate.commands.launch \
111
+ --config_file scripts/accelerate_configs/accelerate_zero2_ds.yaml \
112
+ --num_processes "${NPROC_PER_NODE}" \
113
+ scripts/train.py \
114
+ "output_dir=./runs/${TASK_BASENAME}/${RUN_ID}" \
115
+ "wandb.name=${TASK_BASENAME}" \
116
+ "${EXTRA_ARGS[@]}"
baseline/fastwam_zipper/source/src/fastwam.egg-info/PKG-INFO ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Metadata-Version: 2.4
2
+ Name: fastwam
3
+ Version: 0.1.0
4
+ Summary: FastWAM: standalone Wan2.2-TI2V-5B training/inference package
5
+ Requires-Python: >=3.10
6
+ License-File: LICENSE
7
+ Requires-Dist: accelerate==1.12.0
8
+ Requires-Dist: av==15.1.0
9
+ Requires-Dist: boto3==1.35.99
10
+ Requires-Dist: datasets==4.8.5
11
+ Requires-Dist: deepspeed==0.18.7
12
+ Requires-Dist: einops==0.8.1
13
+ Requires-Dist: gitpython==3.1.45
14
+ Requires-Dist: huggingface-hub==0.29.2
15
+ Requires-Dist: hydra-core==1.3.2
16
+ Requires-Dist: imageio==2.37.0
17
+ Requires-Dist: imageio-ffmpeg==0.6.0
18
+ Requires-Dist: jsonlines==4.0.0
19
+ Requires-Dist: modelscope==1.34.0
20
+ Requires-Dist: numpy==2.2.6
21
+ Requires-Dist: omegaconf==2.3.0
22
+ Requires-Dist: packaging==25.0
23
+ Requires-Dist: pandas==2.2.3
24
+ Requires-Dist: pillow==12.0.0
25
+ Requires-Dist: pyarrow==24.0.0
26
+ Requires-Dist: regex==2025.11.3
27
+ Requires-Dist: rich==14.2.0
28
+ Requires-Dist: safetensors==0.5.3
29
+ Requires-Dist: termcolor==2.5.0
30
+ Requires-Dist: torch==2.7.1+cu128
31
+ Requires-Dist: torchcodec==0.4.0
32
+ Requires-Dist: torchvision==0.22.1+cu128
33
+ Requires-Dist: tqdm==4.68.3
34
+ Requires-Dist: transformers==4.49.0
35
+ Requires-Dist: typing-extensions==4.15.0
36
+ Requires-Dist: wandb==0.23.1
37
+ Dynamic: license-file