ZUO66 commited on
Commit
3e7c150
·
verified ·
1 Parent(s): bd52195

Add A20 batch-512 trim-v2 step-4000 checkpoint

Browse files
README.md CHANGED
@@ -42,3 +42,44 @@ Training uses the Humanoid Benchmark online data augmentation spec v2 (15 photom
42
  effects, one of them boosted to 1.5x per sample) with the model's own image augmentation turned off, plus joint-angle
43
  bias and noise on the proprioception (column "Proprio augmentation"; the earlier recipe also delayed the
44
  proprioception and dropped action-history rows).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  effects, one of them boosted to 1.5x per sample) with the model's own image augmentation turned off, plus joint-angle
43
  bias and noise on the proprioception (column "Proprio augmentation"; the earlier recipe also delayed the
44
  proprioception and dropped action-history rows).
45
+
46
+ ## A20 pre-reach trim v2, batch 512, step 4000 (2026-11-17)
47
+
48
+ Checkpoint: [`a20-basket-transfer/b512-trimv2/step-004000/`](a20-basket-transfer/b512-trimv2/step-004000).
49
+
50
+ Trained on 100 demonstrations (54,433 frames) from
51
+ `humanoid-benchmark/a20-basket-transfer-pre-reach-trim-expert-v2`, with batch size 512,
52
+ 4,000 optimizer steps, seed 0, EMA weights (decay 0.9999), and Humanoid Benchmark
53
+ augmentation v2. These weights use the `hb_baselines/1` format.
54
+
55
+ Inputs: Unitree G1 state44 and separate `ego_view` and `chest` RGB images, resized
56
+ to 168 x 224. Output: 16 actions of dimension 76 (64 SONIC motion-token values and
57
+ 6 measured values for each INSPIRE hand). Normalization is stored in the weights.
58
+ The ResNet18 visual backbone was initialized with ImageNet weights and fine-tuned.
59
+
60
+ The checkpoint contains `model.pt`, `config.json` and `protocol.json` (weight size
61
+ and SHA-256). `SHA256SUMS` covers the checkpoint and the accompanying inference
62
+ source snapshot. The snapshot is under `code/hb_baselines_v1/`; its requirements
63
+ record the training environment. Optimizer/resume state is excluded.
64
+
65
+ After downloading this repository and installing its inference dependencies,
66
+ load the checkpoint from the repository root:
67
+
68
+ ```python
69
+ import sys
70
+ sys.path.insert(0, 'code/hb_baselines_v1')
71
+ from baselines.runtime import BaselinePolicy
72
+
73
+ policy = BaselinePolicy(
74
+ 'a20-basket-transfer/b512-trimv2/step-004000',
75
+ device='cpu', policy='act', robot='g1',
76
+ )
77
+ print(policy.info())
78
+ ```
79
+
80
+ The saved weights passed SHA-256 verification and strict CPU loading before upload.
81
+ The offline metrics in `config.json` use training frames; they are not held-out
82
+ or closed-loop task success rates.
83
+
84
+ This release uses the accompanying benchmark baseline loader. The earlier
85
+ `epoch-XXX` releases retain their original Psi0 ACT loader and file format.
a20-basket-transfer/b512-trimv2/step-004000/SHA256SUMS ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ 1dbec854e53d5c9098c3c51b4df35f33c1916abc0016f702be64d4e279382b65 a20-basket-transfer/b512-trimv2/step-004000/config.json
2
+ 42235f4f164febb7136fafcd899c7fa2ac4a558fab9615a0565de74acf8b856c a20-basket-transfer/b512-trimv2/step-004000/model.pt
3
+ 63d34ce738d8866211aece1c1dbca648e4a8ffbb732a6b2bf5ffca45d5646a3a a20-basket-transfer/b512-trimv2/step-004000/protocol.json
4
+ 0a8ee3ccab6e183804c08066a9e7eb207d9b13ca38c0e596ce5d58c8302d7929 code/hb_baselines_v1/baselines/__init__.py
5
+ 3db95cd8e4e436875e11f1054a99d250ca7d94e0127909090b58730402340f19 code/hb_baselines_v1/baselines/act_nets.py
6
+ 38a66937bb92393a3f1ce0644b249da28d582a4fd03d134b7743e6037abd88b9 code/hb_baselines_v1/baselines/nets.py
7
+ 750d4c6193aabcacc0f53df26c2c51f64b3f836b940b32ccb5c69f5959419afa code/hb_baselines_v1/baselines/policies.py
8
+ bb211f66038e53efd08ee3ec03e133d8bb206ec36cc608d349b4fad3a61e874b code/hb_baselines_v1/baselines/requirements.txt
9
+ 5d680d59124ca83f9c2088f8e1738395b87ddc58798acf7815e163102738b7ae code/hb_baselines_v1/baselines/runtime.py
a20-basket-transfer/b512-trimv2/step-004000/config.json ADDED
@@ -0,0 +1,884 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "policy": {
3
+ "policy": "act",
4
+ "n_cameras": 2,
5
+ "state_dim": 44,
6
+ "action_dim": 76,
7
+ "horizon": 16,
8
+ "backbone": "resnet18",
9
+ "backbone_weights": "IMAGENET1K_V1",
10
+ "backbone_norm": "frozen_bn",
11
+ "hidden_dim": 384,
12
+ "num_layers": 4,
13
+ "num_heads": 8,
14
+ "mlp_ratio": 4.0,
15
+ "dropout": 0.1,
16
+ "time_embed_dim": 128,
17
+ "num_inference_steps": null,
18
+ "num_train_timesteps": 100,
19
+ "beta_schedule": "squaredcos_cap_v2",
20
+ "prediction_type": "epsilon",
21
+ "clip_sample": true,
22
+ "act_hidden_dim": 512,
23
+ "dim_feedforward": 3200,
24
+ "enc_layers": 4,
25
+ "dec_layers": 7,
26
+ "nheads": 8,
27
+ "latent_dim": 32,
28
+ "kl_weight": 10.0,
29
+ "extra": {}
30
+ },
31
+ "contract": {
32
+ "name": "g1_sonic_dfx6",
33
+ "robot": "g1",
34
+ "cameras": [
35
+ "ego_view",
36
+ "chest"
37
+ ],
38
+ "state": [
39
+ [
40
+ "observation.state",
41
+ 0,
42
+ 29
43
+ ],
44
+ [
45
+ "observation.inspire_left_hand_state",
46
+ 0,
47
+ 6
48
+ ],
49
+ [
50
+ "observation.inspire_right_hand_state",
51
+ 0,
52
+ 6
53
+ ],
54
+ [
55
+ "observation.projected_gravity",
56
+ 0,
57
+ 3
58
+ ]
59
+ ],
60
+ "action": [
61
+ [
62
+ "action.motion_token",
63
+ 0,
64
+ 64
65
+ ],
66
+ [
67
+ "observation.inspire_left_hand_state",
68
+ 0,
69
+ 6
70
+ ],
71
+ [
72
+ "observation.inspire_right_hand_state",
73
+ 0,
74
+ 6
75
+ ]
76
+ ],
77
+ "state_dim": 44,
78
+ "action_dim": 76,
79
+ "action_space": "sonic_dfx6",
80
+ "body_dims": 29
81
+ },
82
+ "image_hw": [
83
+ 168,
84
+ 224
85
+ ],
86
+ "task_text": "Grasp the basket handle with the right hand from the v10 round table, walk while raising it, place and release it on the long desk, then withdraw the open right hand diagonally backward and upward without an additional downward arm return.",
87
+ "dataset": {
88
+ "name": "a20-basket-transfer-pre-reach-trim-expert-v2",
89
+ "root": "/mnt/data0/zuo/Humanoid_benchmark/data/baselines/a20-basket-transfer-pre-reach-trim-expert-v2",
90
+ "robot_type": null,
91
+ "fps": 30.0,
92
+ "episodes": [
93
+ 0,
94
+ 1,
95
+ 2,
96
+ 3,
97
+ 4,
98
+ 5,
99
+ 6,
100
+ 7,
101
+ 8,
102
+ 9,
103
+ 10,
104
+ 11,
105
+ 12,
106
+ 13,
107
+ 14,
108
+ 15,
109
+ 16,
110
+ 17,
111
+ 18,
112
+ 19,
113
+ 20,
114
+ 21,
115
+ 22,
116
+ 23,
117
+ 24,
118
+ 25,
119
+ 26,
120
+ 27,
121
+ 28,
122
+ 29,
123
+ 30,
124
+ 31,
125
+ 32,
126
+ 33,
127
+ 34,
128
+ 35,
129
+ 36,
130
+ 37,
131
+ 38,
132
+ 39,
133
+ 40,
134
+ 41,
135
+ 42,
136
+ 43,
137
+ 44,
138
+ 45,
139
+ 46,
140
+ 47,
141
+ 48,
142
+ 49,
143
+ 50,
144
+ 51,
145
+ 52,
146
+ 53,
147
+ 54,
148
+ 55,
149
+ 56,
150
+ 57,
151
+ 58,
152
+ 59,
153
+ 60,
154
+ 61,
155
+ 62,
156
+ 63,
157
+ 64,
158
+ 65,
159
+ 66,
160
+ 67,
161
+ 68,
162
+ 69,
163
+ 70,
164
+ 71,
165
+ 72,
166
+ 73,
167
+ 74,
168
+ 75,
169
+ 76,
170
+ 77,
171
+ 78,
172
+ 79,
173
+ 80,
174
+ 81,
175
+ 82,
176
+ 83,
177
+ 84,
178
+ 85,
179
+ 86,
180
+ 87,
181
+ 88,
182
+ 89,
183
+ 90,
184
+ 91,
185
+ 92,
186
+ 93,
187
+ 94,
188
+ 95,
189
+ 96,
190
+ 97,
191
+ 98,
192
+ 99
193
+ ],
194
+ "train_episodes": [
195
+ 0,
196
+ 1,
197
+ 2,
198
+ 3,
199
+ 4,
200
+ 5,
201
+ 6,
202
+ 7,
203
+ 8,
204
+ 9,
205
+ 10,
206
+ 11,
207
+ 12,
208
+ 13,
209
+ 14,
210
+ 15,
211
+ 16,
212
+ 17,
213
+ 18,
214
+ 19,
215
+ 20,
216
+ 21,
217
+ 22,
218
+ 23,
219
+ 24,
220
+ 25,
221
+ 26,
222
+ 27,
223
+ 28,
224
+ 29,
225
+ 30,
226
+ 31,
227
+ 32,
228
+ 33,
229
+ 34,
230
+ 35,
231
+ 36,
232
+ 37,
233
+ 38,
234
+ 39,
235
+ 40,
236
+ 41,
237
+ 42,
238
+ 43,
239
+ 44,
240
+ 45,
241
+ 46,
242
+ 47,
243
+ 48,
244
+ 49,
245
+ 50,
246
+ 51,
247
+ 52,
248
+ 53,
249
+ 54,
250
+ 55,
251
+ 56,
252
+ 57,
253
+ 58,
254
+ 59,
255
+ 60,
256
+ 61,
257
+ 62,
258
+ 63,
259
+ 64,
260
+ 65,
261
+ 66,
262
+ 67,
263
+ 68,
264
+ 69,
265
+ 70,
266
+ 71,
267
+ 72,
268
+ 73,
269
+ 74,
270
+ 75,
271
+ 76,
272
+ 77,
273
+ 78,
274
+ 79,
275
+ 80,
276
+ 81,
277
+ 82,
278
+ 83,
279
+ 84,
280
+ 85,
281
+ 86,
282
+ 87,
283
+ 88,
284
+ 89,
285
+ 90,
286
+ 91,
287
+ 92,
288
+ 93,
289
+ 94,
290
+ 95,
291
+ 96,
292
+ 97,
293
+ 98,
294
+ 99
295
+ ],
296
+ "val_episodes": [],
297
+ "train_frames": 54433,
298
+ "git_commit": null
299
+ },
300
+ "training": {
301
+ "steps": 4000,
302
+ "batch_size": 512,
303
+ "lr": 0.0001,
304
+ "betas": [
305
+ 0.95,
306
+ 0.999
307
+ ],
308
+ "weight_decay": 1e-06,
309
+ "warmup": 500,
310
+ "schedule": "linear warm-up + cosine",
311
+ "backbone_lr_scale": 0.1,
312
+ "ema": 0.9999,
313
+ "shift_px": 0,
314
+ "amp": true,
315
+ "seed": 0,
316
+ "augmentation": {
317
+ "spec": "docs/augmentation_v2.md",
318
+ "version": "hb-aug-v2-20261109",
319
+ "implementation": "evaluation/augmentation/hb_aug_v2.py",
320
+ "visual": {
321
+ "effects": [
322
+ "exposure",
323
+ "contrast",
324
+ "saturation",
325
+ "hue",
326
+ "gamma",
327
+ "temperature",
328
+ "rotation_crop",
329
+ "noise",
330
+ "gaussian_blur",
331
+ "jpeg",
332
+ "motion_blur",
333
+ "sharpen",
334
+ "defocus",
335
+ "light_shadow",
336
+ "wb_channel"
337
+ ],
338
+ "boosted_per_sample": 1,
339
+ "boost": 1.5,
340
+ "pixel_scale": "H / 224",
341
+ "future_targets": "none (one observation frame)"
342
+ },
343
+ "proprio": {
344
+ "robot": "g1",
345
+ "body_dims": 29,
346
+ "bias_deg": 2.0,
347
+ "bias_scope": "per_sample",
348
+ "noise_deg": 0.25,
349
+ "noise_clip_sigma": 3,
350
+ "delay_probability": 0.0,
351
+ "history_drop_probability": 0.0,
352
+ "velocity": false
353
+ },
354
+ "native_image_augmentation": "off",
355
+ "evaluation": "off"
356
+ },
357
+ "preflight": {
358
+ "samples": 64,
359
+ "state_dim": 44,
360
+ "body_dims": 29,
361
+ "images_changed": 64,
362
+ "body_values_moved": 1.0,
363
+ "max_body_offset_deg": 2.5382,
364
+ "actions_bit_identical": true,
365
+ "non_body_state_bit_identical": true,
366
+ "eval_items_deterministic": 8
367
+ }
368
+ },
369
+ "stats": {
370
+ "state": {
371
+ "min": [
372
+ -0.6335107088088989,
373
+ -0.26151371002197266,
374
+ -0.09143047779798508,
375
+ 0.13051827251911163,
376
+ -0.41364213824272156,
377
+ -0.17374181747436523,
378
+ -0.5706103444099426,
379
+ -0.4567093253135681,
380
+ -0.5945078730583191,
381
+ 0.07693764567375183,
382
+ -0.479587197303772,
383
+ -0.23942920565605164,
384
+ -0.1249256581068039,
385
+ -0.061801809817552567,
386
+ 0.03293042629957199,
387
+ -0.15965673327445984,
388
+ 0.12444422394037247,
389
+ -1.0301204919815063,
390
+ 0.8110364079475403,
391
+ -0.42373916506767273,
392
+ -0.2393926978111267,
393
+ -0.1671462506055832,
394
+ -1.8559280633926392,
395
+ -2.1003689765930176,
396
+ -0.44611114263534546,
397
+ -0.86998051404953,
398
+ -1.5242679119110107,
399
+ -0.9486506581306458,
400
+ -0.4465007185935974,
401
+ 0.9997791647911072,
402
+ 0.9987788796424866,
403
+ 0.9989608526229858,
404
+ 0.9997321963310242,
405
+ 0.8891316056251526,
406
+ 0.9935978651046753,
407
+ 0.3987571895122528,
408
+ 0.39748135209083557,
409
+ 0.3998531103134155,
410
+ 0.47865480184555054,
411
+ 0.43634313344955444,
412
+ 0.0,
413
+ -0.1312161237001419,
414
+ -0.10554340481758118,
415
+ -1.0
416
+ ],
417
+ "max": [
418
+ 0.18651676177978516,
419
+ 0.2880381643772125,
420
+ 0.3991033434867859,
421
+ 1.1868021488189697,
422
+ 0.30963531136512756,
423
+ 0.25351187586784363,
424
+ 0.13313187658786774,
425
+ 0.14919418096542358,
426
+ 0.09004810452461243,
427
+ 1.1367807388305664,
428
+ 0.47997909784317017,
429
+ 0.1897573471069336,
430
+ 0.12687286734580994,
431
+ 0.0903186947107315,
432
+ 0.2110750675201416,
433
+ 0.1764557808637619,
434
+ 0.3972950279712677,
435
+ -0.03375900536775589,
436
+ 1.248238205909729,
437
+ 0.5087170004844666,
438
+ 0.16197925806045532,
439
+ 0.19902198016643524,
440
+ 0.7476708292961121,
441
+ -0.14924407005310059,
442
+ 1.3937729597091675,
443
+ 1.4777758121490479,
444
+ 1.449870228767395,
445
+ 0.9068933725357056,
446
+ 1.5030572414398193,
447
+ 1.0,
448
+ 1.0,
449
+ 1.0,
450
+ 1.0,
451
+ 1.0,
452
+ 1.0,
453
+ 1.0,
454
+ 1.0,
455
+ 1.0,
456
+ 1.0,
457
+ 1.0,
458
+ 1.0,
459
+ 0.1443370282649994,
460
+ 0.07133471965789795,
461
+ -0.9889927506446838
462
+ ],
463
+ "mean": [
464
+ -0.1197492680031697,
465
+ 0.07687621598947185,
466
+ 0.13623676943730387,
467
+ 0.27067508540983654,
468
+ -0.09364010490166123,
469
+ -0.029264338937400346,
470
+ -0.0877177892591824,
471
+ -0.09172362643767557,
472
+ -0.21603681153788365,
473
+ 0.27020221177162185,
474
+ -0.13314246080400155,
475
+ 0.01521604484593622,
476
+ -0.007014541196278249,
477
+ 0.023597627586334522,
478
+ 0.1362211304918607,
479
+ 0.0044590163081436405,
480
+ 0.24020097902181153,
481
+ -0.5506130059849405,
482
+ 0.986759154493458,
483
+ -0.11353825983460156,
484
+ -0.09008970046721287,
485
+ 0.00793797976488734,
486
+ -0.7956446111257796,
487
+ -0.6433268364566035,
488
+ 0.35943391980855405,
489
+ 0.4128302732079997,
490
+ -0.6341984046200151,
491
+ 0.17366417734106818,
492
+ 0.31817584543570415,
493
+ 0.9999920738393157,
494
+ 0.9999916277741105,
495
+ 0.9999916507452171,
496
+ 0.9999910946907014,
497
+ 0.9975380262153739,
498
+ 0.9997842168412904,
499
+ 0.7431049855982732,
500
+ 0.743061001937713,
501
+ 0.7431612987122399,
502
+ 0.7812160503121215,
503
+ 0.7544858488771042,
504
+ 0.1884228588481359,
505
+ -0.018600662889153487,
506
+ -0.001923185563505277,
507
+ -0.9992875029526394
508
+ ],
509
+ "std": [
510
+ 0.1225902736805545,
511
+ 0.0667315519538753,
512
+ 0.06966628355722553,
513
+ 0.1683343132651939,
514
+ 0.11676670853146011,
515
+ 0.06393707008498987,
516
+ 0.14262675997248406,
517
+ 0.07183651035230894,
518
+ 0.07077858012951907,
519
+ 0.1583747884788023,
520
+ 0.09609718753190293,
521
+ 0.06937572380392895,
522
+ 0.038858334068535276,
523
+ 0.01853271444550659,
524
+ 0.019156535689633315,
525
+ 0.04755913941308977,
526
+ 0.041089601727671066,
527
+ 0.14354888870689778,
528
+ 0.053966848435702465,
529
+ 0.13684428975351073,
530
+ 0.05453949581291963,
531
+ 0.06076597347825228,
532
+ 0.6370602259123855,
533
+ 0.4013113120474627,
534
+ 0.24111485368872668,
535
+ 0.5381053688248473,
536
+ 0.48085827035516315,
537
+ 0.1527991524681821,
538
+ 0.44871163018722837,
539
+ 1.2824078270394207e-05,
540
+ 2.1003630589963823e-05,
541
+ 1.8574155779181667e-05,
542
+ 1.5382720229173254e-05,
543
+ 0.0031309164055460465,
544
+ 0.0001241612656891375,
545
+ 0.2938442161025196,
546
+ 0.29381668166139846,
547
+ 0.29367033348300187,
548
+ 0.25040401472179635,
549
+ 0.2708879748387615,
550
+ 0.3732468841625382,
551
+ 0.027734659598288042,
552
+ 0.01745881392269428,
553
+ 0.0008838855519788744
554
+ ]
555
+ },
556
+ "action": {
557
+ "min": [
558
+ -0.125,
559
+ -0.5,
560
+ -0.375,
561
+ -0.375,
562
+ -0.625,
563
+ -0.5625,
564
+ -0.4375,
565
+ -0.25,
566
+ -0.375,
567
+ -0.1875,
568
+ -0.625,
569
+ -0.25,
570
+ -0.25,
571
+ -0.375,
572
+ -0.3125,
573
+ -0.5,
574
+ -0.1875,
575
+ -0.25,
576
+ -0.5,
577
+ -0.375,
578
+ -0.4375,
579
+ -0.5,
580
+ -0.3125,
581
+ -0.5,
582
+ -0.5,
583
+ 0.0,
584
+ -0.4375,
585
+ -0.5,
586
+ -0.5,
587
+ -0.375,
588
+ -0.0625,
589
+ -0.125,
590
+ -0.375,
591
+ -0.1875,
592
+ 0.0625,
593
+ -0.6875,
594
+ -0.1875,
595
+ -0.5,
596
+ -0.375,
597
+ -0.3125,
598
+ -0.4375,
599
+ -0.5625,
600
+ -0.375,
601
+ -0.125,
602
+ -0.5625,
603
+ -0.4375,
604
+ -0.375,
605
+ -0.25,
606
+ -0.1875,
607
+ -0.5625,
608
+ -0.375,
609
+ -0.1875,
610
+ -0.5,
611
+ -0.5625,
612
+ -0.3125,
613
+ -0.3125,
614
+ -0.5,
615
+ -0.125,
616
+ -0.625,
617
+ -0.3125,
618
+ -0.5625,
619
+ -0.1875,
620
+ -0.3125,
621
+ -0.125,
622
+ 0.9997791647911072,
623
+ 0.9987788796424866,
624
+ 0.9989608526229858,
625
+ 0.9997321963310242,
626
+ 0.8891316056251526,
627
+ 0.9935978651046753,
628
+ 0.3987571895122528,
629
+ 0.39748135209083557,
630
+ 0.3998531103134155,
631
+ 0.47865480184555054,
632
+ 0.43634313344955444,
633
+ 0.0
634
+ ],
635
+ "max": [
636
+ 0.5625,
637
+ 0.375,
638
+ 0.375,
639
+ 0.4375,
640
+ 0.25,
641
+ 0.25,
642
+ 0.1875,
643
+ 0.25,
644
+ 0.3125,
645
+ 0.1875,
646
+ 0.125,
647
+ 0.375,
648
+ 0.5,
649
+ 0.25,
650
+ 0.3125,
651
+ 0.25,
652
+ 0.375,
653
+ 0.5625,
654
+ 0.1875,
655
+ 0.25,
656
+ 0.25,
657
+ 0.1875,
658
+ 0.5,
659
+ 0.3125,
660
+ 0.0625,
661
+ 0.5,
662
+ 0.0,
663
+ 0.1875,
664
+ 0.3125,
665
+ 0.375,
666
+ 0.6875,
667
+ 0.5,
668
+ 0.25,
669
+ 0.5,
670
+ 0.375,
671
+ 0.125,
672
+ 0.6875,
673
+ 0.1875,
674
+ 0.375,
675
+ 0.25,
676
+ 0.125,
677
+ 0.375,
678
+ 0.1875,
679
+ 0.5,
680
+ 0.3125,
681
+ 0.5,
682
+ 0.5,
683
+ 0.4375,
684
+ 0.5,
685
+ 0.1875,
686
+ 0.5,
687
+ 0.3125,
688
+ 0.625,
689
+ 0.125,
690
+ 0.25,
691
+ 0.5625,
692
+ 0.5,
693
+ 0.3125,
694
+ 0.125,
695
+ 0.1875,
696
+ 0.25,
697
+ 0.625,
698
+ 0.4375,
699
+ 0.6875,
700
+ 1.0,
701
+ 1.0,
702
+ 1.0,
703
+ 1.0,
704
+ 1.0,
705
+ 1.0,
706
+ 1.0,
707
+ 1.0,
708
+ 1.0,
709
+ 1.0,
710
+ 1.0,
711
+ 1.0
712
+ ],
713
+ "mean": [
714
+ 0.28530257380633073,
715
+ 0.0012216853746808002,
716
+ 0.02519611265225139,
717
+ 0.0920742013117043,
718
+ -0.18735762313302592,
719
+ -0.18916948358532507,
720
+ -0.10082578582844966,
721
+ -0.04547218599011629,
722
+ 0.009826300222291624,
723
+ -0.09046442415446512,
724
+ -0.3906258611504051,
725
+ 0.07456643947605313,
726
+ 0.06761982620836625,
727
+ 0.016528346774934324,
728
+ 0.02783008469127184,
729
+ -0.15926115591644774,
730
+ 0.137793250417945,
731
+ 0.13872558925651718,
732
+ -0.17794123050355484,
733
+ -0.04058314809031286,
734
+ -0.1081088218543898,
735
+ -0.27256788161593154,
736
+ 0.0295489408998218,
737
+ -0.12704953796410265,
738
+ -0.2065647217680451,
739
+ 0.257109657744383,
740
+ -0.2610296143883306,
741
+ -0.11359147943343192,
742
+ -0.08978239303363768,
743
+ 0.042958775007807766,
744
+ 0.3583063123472893,
745
+ 0.2295861425973215,
746
+ 0.005530882001726894,
747
+ 0.18030882001726894,
748
+ 0.22869399077765326,
749
+ -0.3416688865210442,
750
+ 0.16757757242849006,
751
+ -0.24212104789374092,
752
+ -0.08830580713905167,
753
+ 0.03962554883985817,
754
+ -0.24867382837616886,
755
+ -0.16334300883655137,
756
+ -0.10593298183087466,
757
+ 0.29390833685448164,
758
+ -0.17902857641504236,
759
+ -0.02676455459004648,
760
+ -0.010649560009553029,
761
+ 0.006596412102952253,
762
+ 0.19015693604982273,
763
+ -0.28608220197306783,
764
+ 0.0342829717267099,
765
+ 0.05214208292763581,
766
+ 0.2372308617933974,
767
+ -0.27289282236878365,
768
+ -0.07418638509727556,
769
+ 0.21441496886080136,
770
+ -0.07936362133264747,
771
+ 0.07053166277809417,
772
+ -0.2592969797732993,
773
+ -0.04738279168886521,
774
+ -0.25777561405764887,
775
+ 0.3476016387118108,
776
+ 0.0341980048867415,
777
+ 0.4095493542520162,
778
+ 0.9999920738393157,
779
+ 0.9999916277741105,
780
+ 0.9999916507452171,
781
+ 0.9999910946907014,
782
+ 0.9975380262153739,
783
+ 0.9997842168412904,
784
+ 0.7431049855982732,
785
+ 0.743061001937713,
786
+ 0.7431612987122399,
787
+ 0.7812160503121215,
788
+ 0.7544858488771042,
789
+ 0.1884228588481359
790
+ ],
791
+ "std": [
792
+ 0.13976769962393334,
793
+ 0.2042750041966315,
794
+ 0.11249063031363368,
795
+ 0.11082072512754977,
796
+ 0.1564752219884858,
797
+ 0.17244514813212275,
798
+ 0.10524673666978752,
799
+ 0.09606056085676273,
800
+ 0.16248529891535804,
801
+ 0.06517700714473815,
802
+ 0.12267993314050843,
803
+ 0.09188274452085451,
804
+ 0.1817845505649146,
805
+ 0.13895894808224718,
806
+ 0.09471602567899642,
807
+ 0.143892040083983,
808
+ 0.13123846053152619,
809
+ 0.17169478968550717,
810
+ 0.099377781857111,
811
+ 0.12335366669054255,
812
+ 0.09243992450340514,
813
+ 0.12474594719512408,
814
+ 0.1211368699762344,
815
+ 0.17473771543792177,
816
+ 0.12139112019256279,
817
+ 0.08058301151688785,
818
+ 0.08181288643740524,
819
+ 0.12245444611527911,
820
+ 0.15642136427791328,
821
+ 0.16023434329274994,
822
+ 0.1749952692685118,
823
+ 0.10352440920325795,
824
+ 0.07368975064774456,
825
+ 0.13144910448634894,
826
+ 0.04419890783153379,
827
+ 0.18157602689607938,
828
+ 0.15703202636240324,
829
+ 0.1308473547559705,
830
+ 0.10877569321547828,
831
+ 0.06509046280547963,
832
+ 0.08347560937172435,
833
+ 0.19809736217235097,
834
+ 0.06719381956707862,
835
+ 0.09127596366580104,
836
+ 0.13233099033714146,
837
+ 0.25952623468134783,
838
+ 0.21598709847307782,
839
+ 0.10931842674892184,
840
+ 0.11294459446909082,
841
+ 0.1403015515249487,
842
+ 0.10231818070157317,
843
+ 0.10139944441893624,
844
+ 0.288021039372521,
845
+ 0.11780041132456433,
846
+ 0.11650297357712408,
847
+ 0.21262101617713844,
848
+ 0.28703233994958033,
849
+ 0.09814627765272926,
850
+ 0.21616444128412057,
851
+ 0.06262758949048293,
852
+ 0.1745534315389492,
853
+ 0.17455467068976818,
854
+ 0.10468632106020986,
855
+ 0.18477106893909007,
856
+ 1.2824078270394207e-05,
857
+ 2.1003630589963823e-05,
858
+ 1.8574155779181667e-05,
859
+ 1.5382720229173254e-05,
860
+ 0.0031309164055460465,
861
+ 0.0001241612656891375,
862
+ 0.2938442161025196,
863
+ 0.29381668166139846,
864
+ 0.29367033348300187,
865
+ 0.25040401472179635,
866
+ 0.2708879748387615,
867
+ 0.3732468841625382
868
+ ]
869
+ }
870
+ },
871
+ "code_commit": "028df1bb7024fb1e3d176de7e60c84d3333cd97f",
872
+ "step": 4000,
873
+ "weights": "ema",
874
+ "offline_eval": {
875
+ "mse": 0.005647905636578798,
876
+ "mse_row0": 0.003931598272174597,
877
+ "mse_motion_token": 0.0046808067709207535,
878
+ "mse_inspire_left_hand_state": 0.0018608566606417298,
879
+ "mse_inspire_right_hand_state": 0.01975069008767605,
880
+ "samples": 256,
881
+ "frames": "train"
882
+ },
883
+ "format": "hb_baselines/1"
884
+ }
a20-basket-transfer/b512-trimv2/step-004000/model.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42235f4f164febb7136fafcd899c7fa2ac4a558fab9615a0565de74acf8b856c
3
+ size 336137451
a20-basket-transfer/b512-trimv2/step-004000/protocol.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "format": "hb_baselines/1",
3
+ "step": 4000,
4
+ "policy": "act",
5
+ "files": {
6
+ "model.pt": {
7
+ "bytes": 336137451,
8
+ "sha256": "42235f4f164febb7136fafcd899c7fa2ac4a558fab9615a0565de74acf8b856c"
9
+ }
10
+ }
11
+ }
code/hb_baselines_v1/baselines/__init__.py ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ """FM-DiT, DP (Diffusion Policy, DDPM-DiT) and ACT visuomotor baselines of the Humanoid Benchmark.
2
+
3
+ Ported from DextraVerse's imitation-learning core (`dextraverse/algorithms/il/models/{fm_dit,ddpm_dit,act}.py` and
4
+ their networks), torch only. `train.py` trains one on a task's LeRobot v2.1 expert dataset through a robot contract
5
+ (`data.CONTRACTS`: G1 state44/action76 or H2 state78/action75); `runtime.BaselinePolicy` serves a checkpoint to the
6
+ G1 evaluation adapters (`evaluation/policy/{FMDiT,DP,ACT}/model.py`, via `adapters.py`) and to the H2 adapter
7
+ (`debug/newton/h2_policy_adapters/baseline.py`). Usage and the environment: evaluation/README.md#baselines.
8
+ """
code/hb_baselines_v1/baselines/act_nets.py ADDED
@@ -0,0 +1,615 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """ACT (Zhao et al. 2023, "Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware") network core: the
2
+ ResNet backbone with frozen BatchNorm and sine position embedding, the DETR encoder-decoder transformer and the CVAE
3
+ action-chunk decoder.
4
+
5
+ Ported unchanged from DextraVerse (`dextraverse/algorithms/il/models/nets/act/`), which vendors the ACT repository's
6
+ `detr/models/{backbone,position_encoding,transformer,detr_vae}.py` (Apache-2.0, (c) Facebook, Inc. and its
7
+ affiliates; see THIRD_PARTY_NOTICES.md) with the NestedTensor/argparse plumbing removed, explicit `qpos_dim` /
8
+ `action_dim` instead of the ALOHA 14-DoF constants, and one backbone shared by every camera (upstream behaviour).
9
+ The policy uses `return_intermediate_dec=False`: `Transformer.forward(...)[0]` is the final decoder layer.
10
+ """
11
+ from __future__ import annotations
12
+
13
+ import copy
14
+ import math
15
+ from typing import Optional
16
+
17
+ import numpy as np
18
+ import torch
19
+ import torch.nn.functional as F
20
+ import torchvision
21
+ from torch import Tensor, nn
22
+ from torchvision.models._utils import IntermediateLayerGetter
23
+
24
+
25
+ class PositionEmbeddingSine(nn.Module):
26
+ """Standard DETR sine embedding over a ``(B, C, H, W)`` feature map."""
27
+
28
+ def __init__(
29
+ self, num_pos_feats: int = 64, temperature: int = 10000, normalize: bool = False
30
+ ) -> None:
31
+ super().__init__()
32
+ self.num_pos_feats = num_pos_feats
33
+ self.temperature = temperature
34
+ self.normalize = normalize
35
+ self.scale = 2 * math.pi
36
+
37
+ def forward(self, x: torch.Tensor) -> torch.Tensor:
38
+ not_mask = torch.ones((1, x.shape[-2], x.shape[-1]), device=x.device)
39
+ y_embed = not_mask.cumsum(1, dtype=torch.float32)
40
+ x_embed = not_mask.cumsum(2, dtype=torch.float32)
41
+ if self.normalize:
42
+ eps = 1e-6
43
+ y_embed = y_embed / (y_embed[:, -1:, :] + eps) * self.scale
44
+ x_embed = x_embed / (x_embed[:, :, -1:] + eps) * self.scale
45
+
46
+ dim_t = torch.arange(self.num_pos_feats, dtype=torch.float32, device=x.device)
47
+ dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)
48
+
49
+ pos_x = x_embed[:, :, :, None] / dim_t
50
+ pos_y = y_embed[:, :, :, None] / dim_t
51
+ pos_x = torch.stack(
52
+ (pos_x[:, :, :, 0::2].sin(), pos_x[:, :, :, 1::2].cos()), dim=4
53
+ ).flatten(3)
54
+ pos_y = torch.stack(
55
+ (pos_y[:, :, :, 0::2].sin(), pos_y[:, :, :, 1::2].cos()), dim=4
56
+ ).flatten(3)
57
+ return torch.cat((pos_y, pos_x), dim=3).permute(0, 3, 1, 2)
58
+
59
+
60
+ class FrozenBatchNorm2d(nn.Module):
61
+ """BatchNorm2d with fixed statistics and affine parameters (the DETR/ACT backbone norm)."""
62
+
63
+ def __init__(self, n: int) -> None:
64
+ super().__init__()
65
+ self.register_buffer("weight", torch.ones(n))
66
+ self.register_buffer("bias", torch.zeros(n))
67
+ self.register_buffer("running_mean", torch.zeros(n))
68
+ self.register_buffer("running_var", torch.ones(n))
69
+
70
+ def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict, *args):
71
+ state_dict.pop(prefix + "num_batches_tracked", None)
72
+ super()._load_from_state_dict(state_dict, prefix, local_metadata, strict, *args)
73
+
74
+ def forward(self, x: torch.Tensor) -> torch.Tensor:
75
+ w = self.weight.reshape(1, -1, 1, 1)
76
+ b = self.bias.reshape(1, -1, 1, 1)
77
+ rv = self.running_var.reshape(1, -1, 1, 1)
78
+ rm = self.running_mean.reshape(1, -1, 1, 1)
79
+ scale = w * (rv + 1e-5).rsqrt()
80
+ return x * scale + (b - rm * scale)
81
+
82
+
83
+ class Backbone(nn.Module):
84
+ """ResNet trunk returning the layer4 feature map."""
85
+
86
+ def __init__(self, name: str = "resnet18", weights: str | None = None) -> None:
87
+ super().__init__()
88
+ trunk = getattr(torchvision.models, name)(
89
+ replace_stride_with_dilation=[False, False, False],
90
+ weights=weights,
91
+ norm_layer=FrozenBatchNorm2d,
92
+ )
93
+ self.body = IntermediateLayerGetter(trunk, return_layers={"layer4": "0"})
94
+ self.num_channels = 512 if name in ("resnet18", "resnet34") else 2048
95
+
96
+ def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
97
+ return self.body(x)
98
+
99
+
100
+ class Joiner(nn.Sequential):
101
+ """Backbone + position embedding, returning matched feature/position lists."""
102
+
103
+ def __init__(self, backbone: Backbone, position_embedding: PositionEmbeddingSine) -> None:
104
+ super().__init__(backbone, position_embedding)
105
+ self.num_channels = backbone.num_channels
106
+
107
+ def forward(self, x: torch.Tensor) -> tuple[list[torch.Tensor], list[torch.Tensor]]:
108
+ features = list(self[0](x).values())
109
+ pos = [self[1](f).to(f.dtype) for f in features]
110
+ return features, pos
111
+
112
+
113
+ def build_backbone(
114
+ name: str = "resnet18", hidden_dim: int = 512, weights: str | None = None
115
+ ) -> Joiner:
116
+ """The ACT camera backbone: ResNet + sine embedding sized to the transformer width."""
117
+ return Joiner(Backbone(name, weights=weights), PositionEmbeddingSine(hidden_dim // 2))
118
+
119
+
120
+ class Transformer(nn.Module):
121
+ def __init__(
122
+ self,
123
+ d_model=512,
124
+ nhead=8,
125
+ num_encoder_layers=6,
126
+ num_decoder_layers=6,
127
+ dim_feedforward=2048,
128
+ dropout=0.1,
129
+ activation="relu",
130
+ normalize_before=False,
131
+ return_intermediate_dec=False,
132
+ ):
133
+ super().__init__()
134
+
135
+ encoder_layer = TransformerEncoderLayer(
136
+ d_model, nhead, dim_feedforward, dropout, activation, normalize_before
137
+ )
138
+ encoder_norm = nn.LayerNorm(d_model) if normalize_before else None
139
+ self.encoder = TransformerEncoder(encoder_layer, num_encoder_layers, encoder_norm)
140
+
141
+ decoder_layer = TransformerDecoderLayer(
142
+ d_model, nhead, dim_feedforward, dropout, activation, normalize_before
143
+ )
144
+ decoder_norm = nn.LayerNorm(d_model)
145
+ self.decoder = TransformerDecoder(
146
+ decoder_layer,
147
+ num_decoder_layers,
148
+ decoder_norm,
149
+ return_intermediate=return_intermediate_dec,
150
+ )
151
+
152
+ self._reset_parameters()
153
+
154
+ self.d_model = d_model
155
+ self.nhead = nhead
156
+
157
+ def _reset_parameters(self):
158
+ for p in self.parameters():
159
+ if p.dim() > 1:
160
+ nn.init.xavier_uniform_(p)
161
+
162
+ def forward(
163
+ self,
164
+ src,
165
+ mask,
166
+ query_embed,
167
+ pos_embed,
168
+ latent_input=None,
169
+ proprio_input=None,
170
+ additional_pos_embed=None,
171
+ ):
172
+ # TODO flatten only when input has H and W
173
+ if len(src.shape) == 4: # has H and W
174
+ # flatten NxCxHxW to HWxNxC
175
+ bs, c, h, w = src.shape
176
+ src = src.flatten(2).permute(2, 0, 1)
177
+ pos_embed = pos_embed.flatten(2).permute(2, 0, 1).repeat(1, bs, 1)
178
+ query_embed = query_embed.unsqueeze(1).repeat(1, bs, 1)
179
+ # mask = mask.flatten(1)
180
+
181
+ additional_pos_embed = additional_pos_embed.unsqueeze(1).repeat(
182
+ 1, bs, 1
183
+ ) # seq, bs, dim
184
+ pos_embed = torch.cat([additional_pos_embed, pos_embed], axis=0)
185
+
186
+ addition_input = torch.stack([latent_input, proprio_input], axis=0)
187
+ src = torch.cat([addition_input, src], axis=0)
188
+ else:
189
+ assert len(src.shape) == 3
190
+ # flatten NxHWxC to HWxNxC
191
+ bs, hw, c = src.shape
192
+ src = src.permute(1, 0, 2)
193
+ pos_embed = pos_embed.unsqueeze(1).repeat(1, bs, 1)
194
+ query_embed = query_embed.unsqueeze(1).repeat(1, bs, 1)
195
+
196
+ tgt = torch.zeros_like(query_embed)
197
+ memory = self.encoder(src, src_key_padding_mask=mask, pos=pos_embed)
198
+ hs = self.decoder(
199
+ tgt, memory, memory_key_padding_mask=mask, pos=pos_embed, query_pos=query_embed
200
+ )
201
+ hs = hs.transpose(1, 2)
202
+ return hs
203
+
204
+
205
+ class TransformerEncoder(nn.Module):
206
+ def __init__(self, encoder_layer, num_layers, norm=None):
207
+ super().__init__()
208
+ self.layers = _get_clones(encoder_layer, num_layers)
209
+ self.num_layers = num_layers
210
+ self.norm = norm
211
+
212
+ def forward(
213
+ self,
214
+ src,
215
+ mask: Optional[Tensor] = None,
216
+ src_key_padding_mask: Optional[Tensor] = None,
217
+ pos: Optional[Tensor] = None,
218
+ ):
219
+ output = src
220
+
221
+ for layer in self.layers:
222
+ output = layer(
223
+ output, src_mask=mask, src_key_padding_mask=src_key_padding_mask, pos=pos
224
+ )
225
+
226
+ if self.norm is not None:
227
+ output = self.norm(output)
228
+
229
+ return output
230
+
231
+
232
+ class TransformerDecoder(nn.Module):
233
+ def __init__(self, decoder_layer, num_layers, norm=None, return_intermediate=False):
234
+ super().__init__()
235
+ self.layers = _get_clones(decoder_layer, num_layers)
236
+ self.num_layers = num_layers
237
+ self.norm = norm
238
+ self.return_intermediate = return_intermediate
239
+
240
+ def forward(
241
+ self,
242
+ tgt,
243
+ memory,
244
+ tgt_mask: Optional[Tensor] = None,
245
+ memory_mask: Optional[Tensor] = None,
246
+ tgt_key_padding_mask: Optional[Tensor] = None,
247
+ memory_key_padding_mask: Optional[Tensor] = None,
248
+ pos: Optional[Tensor] = None,
249
+ query_pos: Optional[Tensor] = None,
250
+ ):
251
+ output = tgt
252
+
253
+ intermediate = []
254
+
255
+ for layer in self.layers:
256
+ output = layer(
257
+ output,
258
+ memory,
259
+ tgt_mask=tgt_mask,
260
+ memory_mask=memory_mask,
261
+ tgt_key_padding_mask=tgt_key_padding_mask,
262
+ memory_key_padding_mask=memory_key_padding_mask,
263
+ pos=pos,
264
+ query_pos=query_pos,
265
+ )
266
+ if self.return_intermediate:
267
+ intermediate.append(self.norm(output))
268
+
269
+ if self.norm is not None:
270
+ output = self.norm(output)
271
+ if self.return_intermediate:
272
+ intermediate.pop()
273
+ intermediate.append(output)
274
+
275
+ if self.return_intermediate:
276
+ return torch.stack(intermediate)
277
+
278
+ return output.unsqueeze(0)
279
+
280
+
281
+ class TransformerEncoderLayer(nn.Module):
282
+ def __init__(
283
+ self,
284
+ d_model,
285
+ nhead,
286
+ dim_feedforward=2048,
287
+ dropout=0.1,
288
+ activation="relu",
289
+ normalize_before=False,
290
+ ):
291
+ super().__init__()
292
+ self.self_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)
293
+ # Implementation of Feedforward model
294
+ self.linear1 = nn.Linear(d_model, dim_feedforward)
295
+ self.dropout = nn.Dropout(dropout)
296
+ self.linear2 = nn.Linear(dim_feedforward, d_model)
297
+
298
+ self.norm1 = nn.LayerNorm(d_model)
299
+ self.norm2 = nn.LayerNorm(d_model)
300
+ self.dropout1 = nn.Dropout(dropout)
301
+ self.dropout2 = nn.Dropout(dropout)
302
+
303
+ self.activation = _get_activation_fn(activation)
304
+ self.normalize_before = normalize_before
305
+
306
+ def with_pos_embed(self, tensor, pos: Optional[Tensor]):
307
+ return tensor if pos is None else tensor + pos
308
+
309
+ def forward_post(
310
+ self,
311
+ src,
312
+ src_mask: Optional[Tensor] = None,
313
+ src_key_padding_mask: Optional[Tensor] = None,
314
+ pos: Optional[Tensor] = None,
315
+ ):
316
+ q = k = self.with_pos_embed(src, pos)
317
+ src2 = self.self_attn(
318
+ q, k, value=src, attn_mask=src_mask, key_padding_mask=src_key_padding_mask
319
+ )[0]
320
+ src = src + self.dropout1(src2)
321
+ src = self.norm1(src)
322
+ src2 = self.linear2(self.dropout(self.activation(self.linear1(src))))
323
+ src = src + self.dropout2(src2)
324
+ src = self.norm2(src)
325
+ return src
326
+
327
+ def forward_pre(
328
+ self,
329
+ src,
330
+ src_mask: Optional[Tensor] = None,
331
+ src_key_padding_mask: Optional[Tensor] = None,
332
+ pos: Optional[Tensor] = None,
333
+ ):
334
+ src2 = self.norm1(src)
335
+ q = k = self.with_pos_embed(src2, pos)
336
+ src2 = self.self_attn(
337
+ q, k, value=src2, attn_mask=src_mask, key_padding_mask=src_key_padding_mask
338
+ )[0]
339
+ src = src + self.dropout1(src2)
340
+ src2 = self.norm2(src)
341
+ src2 = self.linear2(self.dropout(self.activation(self.linear1(src2))))
342
+ src = src + self.dropout2(src2)
343
+ return src
344
+
345
+ def forward(
346
+ self,
347
+ src,
348
+ src_mask: Optional[Tensor] = None,
349
+ src_key_padding_mask: Optional[Tensor] = None,
350
+ pos: Optional[Tensor] = None,
351
+ ):
352
+ if self.normalize_before:
353
+ return self.forward_pre(src, src_mask, src_key_padding_mask, pos)
354
+ return self.forward_post(src, src_mask, src_key_padding_mask, pos)
355
+
356
+
357
+ class TransformerDecoderLayer(nn.Module):
358
+ def __init__(
359
+ self,
360
+ d_model,
361
+ nhead,
362
+ dim_feedforward=2048,
363
+ dropout=0.1,
364
+ activation="relu",
365
+ normalize_before=False,
366
+ ):
367
+ super().__init__()
368
+ self.self_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)
369
+ self.multihead_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)
370
+ # Implementation of Feedforward model
371
+ self.linear1 = nn.Linear(d_model, dim_feedforward)
372
+ self.dropout = nn.Dropout(dropout)
373
+ self.linear2 = nn.Linear(dim_feedforward, d_model)
374
+
375
+ self.norm1 = nn.LayerNorm(d_model)
376
+ self.norm2 = nn.LayerNorm(d_model)
377
+ self.norm3 = nn.LayerNorm(d_model)
378
+ self.dropout1 = nn.Dropout(dropout)
379
+ self.dropout2 = nn.Dropout(dropout)
380
+ self.dropout3 = nn.Dropout(dropout)
381
+
382
+ self.activation = _get_activation_fn(activation)
383
+ self.normalize_before = normalize_before
384
+
385
+ def with_pos_embed(self, tensor, pos: Optional[Tensor]):
386
+ return tensor if pos is None else tensor + pos
387
+
388
+ def forward_post(
389
+ self,
390
+ tgt,
391
+ memory,
392
+ tgt_mask: Optional[Tensor] = None,
393
+ memory_mask: Optional[Tensor] = None,
394
+ tgt_key_padding_mask: Optional[Tensor] = None,
395
+ memory_key_padding_mask: Optional[Tensor] = None,
396
+ pos: Optional[Tensor] = None,
397
+ query_pos: Optional[Tensor] = None,
398
+ ):
399
+ q = k = self.with_pos_embed(tgt, query_pos)
400
+ tgt2 = self.self_attn(
401
+ q, k, value=tgt, attn_mask=tgt_mask, key_padding_mask=tgt_key_padding_mask
402
+ )[0]
403
+ tgt = tgt + self.dropout1(tgt2)
404
+ tgt = self.norm1(tgt)
405
+ tgt2 = self.multihead_attn(
406
+ query=self.with_pos_embed(tgt, query_pos),
407
+ key=self.with_pos_embed(memory, pos),
408
+ value=memory,
409
+ attn_mask=memory_mask,
410
+ key_padding_mask=memory_key_padding_mask,
411
+ )[0]
412
+ tgt = tgt + self.dropout2(tgt2)
413
+ tgt = self.norm2(tgt)
414
+ tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt))))
415
+ tgt = tgt + self.dropout3(tgt2)
416
+ tgt = self.norm3(tgt)
417
+ return tgt
418
+
419
+ def forward_pre(
420
+ self,
421
+ tgt,
422
+ memory,
423
+ tgt_mask: Optional[Tensor] = None,
424
+ memory_mask: Optional[Tensor] = None,
425
+ tgt_key_padding_mask: Optional[Tensor] = None,
426
+ memory_key_padding_mask: Optional[Tensor] = None,
427
+ pos: Optional[Tensor] = None,
428
+ query_pos: Optional[Tensor] = None,
429
+ ):
430
+ tgt2 = self.norm1(tgt)
431
+ q = k = self.with_pos_embed(tgt2, query_pos)
432
+ tgt2 = self.self_attn(
433
+ q, k, value=tgt2, attn_mask=tgt_mask, key_padding_mask=tgt_key_padding_mask
434
+ )[0]
435
+ tgt = tgt + self.dropout1(tgt2)
436
+ tgt2 = self.norm2(tgt)
437
+ tgt2 = self.multihead_attn(
438
+ query=self.with_pos_embed(tgt2, query_pos),
439
+ key=self.with_pos_embed(memory, pos),
440
+ value=memory,
441
+ attn_mask=memory_mask,
442
+ key_padding_mask=memory_key_padding_mask,
443
+ )[0]
444
+ tgt = tgt + self.dropout2(tgt2)
445
+ tgt2 = self.norm3(tgt)
446
+ tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt2))))
447
+ tgt = tgt + self.dropout3(tgt2)
448
+ return tgt
449
+
450
+ def forward(
451
+ self,
452
+ tgt,
453
+ memory,
454
+ tgt_mask: Optional[Tensor] = None,
455
+ memory_mask: Optional[Tensor] = None,
456
+ tgt_key_padding_mask: Optional[Tensor] = None,
457
+ memory_key_padding_mask: Optional[Tensor] = None,
458
+ pos: Optional[Tensor] = None,
459
+ query_pos: Optional[Tensor] = None,
460
+ ):
461
+ if self.normalize_before:
462
+ return self.forward_pre(
463
+ tgt,
464
+ memory,
465
+ tgt_mask,
466
+ memory_mask,
467
+ tgt_key_padding_mask,
468
+ memory_key_padding_mask,
469
+ pos,
470
+ query_pos,
471
+ )
472
+ return self.forward_post(
473
+ tgt,
474
+ memory,
475
+ tgt_mask,
476
+ memory_mask,
477
+ tgt_key_padding_mask,
478
+ memory_key_padding_mask,
479
+ pos,
480
+ query_pos,
481
+ )
482
+
483
+
484
+ def _get_clones(module, N):
485
+ return nn.ModuleList([copy.deepcopy(module) for i in range(N)])
486
+
487
+
488
+ def _get_activation_fn(activation):
489
+ """Return an activation function given a string."""
490
+ if activation == "relu":
491
+ return F.relu
492
+ if activation == "gelu":
493
+ return F.gelu
494
+ raise RuntimeError(f"activation should be relu/gelu, not {activation}.")
495
+
496
+
497
+
498
+ def reparametrize(mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
499
+ std = logvar.div(2).exp()
500
+ return mu + std * torch.randn_like(std)
501
+
502
+
503
+ def get_sinusoid_encoding_table(n_position: int, d_hid: int) -> torch.Tensor:
504
+ def angle_vec(position: int) -> list[float]:
505
+ return [position / np.power(10000, 2 * (j // 2) / d_hid) for j in range(d_hid)]
506
+
507
+ table = np.array([angle_vec(i) for i in range(n_position)])
508
+ table[:, 0::2] = np.sin(table[:, 0::2])
509
+ table[:, 1::2] = np.cos(table[:, 1::2])
510
+ return torch.FloatTensor(table).unsqueeze(0)
511
+
512
+
513
+ def build_style_encoder(
514
+ hidden_dim: int, nheads: int, dim_feedforward: int, enc_layers: int, dropout: float
515
+ ) -> TransformerEncoder:
516
+ """The CVAE style encoder (a plain transformer encoder over [CLS, qpos, action seq])."""
517
+ layer = TransformerEncoderLayer(hidden_dim, nheads, dim_feedforward, dropout, "relu", False)
518
+ return TransformerEncoder(layer, enc_layers, norm=None)
519
+
520
+
521
+ class DETRVAE(nn.Module):
522
+ """CVAE decoder: (qpos, images[, action seq]) → ``num_queries`` future actions."""
523
+
524
+ def __init__(
525
+ self,
526
+ backbone: Joiner,
527
+ transformer: Transformer,
528
+ encoder: TransformerEncoder,
529
+ *,
530
+ qpos_dim: int,
531
+ action_dim: int,
532
+ num_queries: int,
533
+ latent_dim: int = 32,
534
+ ) -> None:
535
+ super().__init__()
536
+ self.num_queries = num_queries
537
+ self.transformer = transformer
538
+ self.encoder = encoder
539
+ hidden_dim = transformer.d_model
540
+ self.action_head = nn.Linear(hidden_dim, action_dim)
541
+ self.query_embed = nn.Embedding(num_queries, hidden_dim)
542
+ self.backbone = backbone # ONE backbone shared by every camera (upstream behavior)
543
+ self.input_proj = nn.Conv2d(backbone.num_channels, hidden_dim, kernel_size=1)
544
+ self.input_proj_robot_state = nn.Linear(qpos_dim, hidden_dim)
545
+
546
+ # CVAE style encoder inputs
547
+ self.latent_dim = latent_dim
548
+ self.cls_embed = nn.Embedding(1, hidden_dim)
549
+ self.encoder_action_proj = nn.Linear(action_dim, hidden_dim)
550
+ self.encoder_joint_proj = nn.Linear(qpos_dim, hidden_dim)
551
+ self.latent_proj = nn.Linear(hidden_dim, latent_dim * 2)
552
+ self.register_buffer(
553
+ "pos_table", get_sinusoid_encoding_table(1 + 1 + num_queries, hidden_dim)
554
+ )
555
+ # decoder side
556
+ self.latent_out_proj = nn.Linear(latent_dim, hidden_dim)
557
+ self.additional_pos_embed = nn.Embedding(2, hidden_dim) # latent + proprio slots
558
+
559
+ def forward(
560
+ self,
561
+ qpos: torch.Tensor,
562
+ images: torch.Tensor,
563
+ actions: torch.Tensor | None = None,
564
+ is_pad: torch.Tensor | None = None,
565
+ ) -> tuple[torch.Tensor, tuple[torch.Tensor | None, torch.Tensor | None]]:
566
+ """``qpos (B, Dq)``, ``images (B, ncam, C, H, W)``; train passes ``actions (B, T, Da)``."""
567
+ is_training = actions is not None
568
+ bs = qpos.shape[0]
569
+ if is_training:
570
+ action_embed = self.encoder_action_proj(actions)
571
+ qpos_embed = self.encoder_joint_proj(qpos).unsqueeze(1)
572
+ cls_embed = self.cls_embed.weight.unsqueeze(0).repeat(bs, 1, 1)
573
+ encoder_input = torch.cat([cls_embed, qpos_embed, action_embed], dim=1)
574
+ encoder_input = encoder_input.permute(1, 0, 2)
575
+ if is_pad is None:
576
+ is_pad = torch.zeros(bs, actions.shape[1], dtype=torch.bool, device=qpos.device)
577
+ cls_joint_is_pad = torch.full((bs, 2), False, device=qpos.device)
578
+ key_pad = torch.cat([cls_joint_is_pad, is_pad], dim=1)
579
+ pos_embed = self.pos_table.clone().detach().permute(1, 0, 2)
580
+ encoder_output = self.encoder(
581
+ encoder_input, pos=pos_embed, src_key_padding_mask=key_pad
582
+ )[0]
583
+ latent_info = self.latent_proj(encoder_output)
584
+ mu = latent_info[:, : self.latent_dim]
585
+ logvar = latent_info[:, self.latent_dim :]
586
+ latent_input = self.latent_out_proj(reparametrize(mu, logvar))
587
+ else:
588
+ mu = logvar = None
589
+ latent_sample = torch.zeros(bs, self.latent_dim, device=qpos.device)
590
+ latent_input = self.latent_out_proj(latent_sample)
591
+
592
+ all_cam_features, all_cam_pos = [], []
593
+ for cam_id in range(images.shape[1]):
594
+ features, pos = self.backbone(images[:, cam_id])
595
+ all_cam_features.append(self.input_proj(features[0]))
596
+ all_cam_pos.append(pos[0])
597
+ proprio_input = self.input_proj_robot_state(qpos)
598
+ src = torch.cat(all_cam_features, dim=3) # cameras fold into the width axis
599
+ pos = torch.cat(all_cam_pos, dim=3)
600
+ hs = self.transformer(
601
+ src,
602
+ None,
603
+ self.query_embed.weight,
604
+ pos,
605
+ latent_input,
606
+ proprio_input,
607
+ self.additional_pos_embed.weight,
608
+ )[0]
609
+ return self.action_head(hs), (mu, logvar)
610
+
611
+
612
+ def kl_divergence(mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
613
+ """Total KL(q(z|x) || N(0, I)), summed over latent dims, meaned over the batch."""
614
+ klds = -0.5 * (1 + logvar - mu.pow(2) - logvar.exp())
615
+ return klds.sum(1).mean()
code/hb_baselines_v1/baselines/nets.py ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Networks of the FM-DiT and DP (DDPM-DiT) baselines: the AdaLN DiT action backbone and the camera + state encoder.
2
+
3
+ Ported from DextraVerse's imitation-learning core (`dextraverse/algorithms/il/models/nets/dit.py`, `layers.py`,
4
+ `positional_embedding.py`, `vision/backbones.py`, `vision/encoder.py`), unchanged in architecture and initialization.
5
+ Left out: the ROVE improvement embedding, the spatial-softmax / R3M / tactile options and the inpainting path (the
6
+ baselines always condition through the global observation feature). Added: the ResNet's normalization choice
7
+ (`get_resnet`): DextraVerse's GroupNorm for a backbone trained from scratch, or BatchNorm frozen at the pretrained
8
+ statistics (the DETR / ACT practice) for an ImageNet-initialized backbone that is fine-tuned with the policy.
9
+ """
10
+ from __future__ import annotations
11
+
12
+ import math
13
+
14
+ import torch
15
+ import torch.nn as nn
16
+ import torch.nn.functional as F
17
+ import torchvision
18
+
19
+ IMAGENET_MEAN = (0.485, 0.456, 0.406)
20
+ IMAGENET_STD = (0.229, 0.224, 0.225)
21
+
22
+
23
+ class SinusoidalPosEmb(nn.Module):
24
+ def __init__(self, dim):
25
+ super().__init__()
26
+ self.dim = dim
27
+
28
+ def forward(self, x):
29
+ half_dim = self.dim // 2
30
+ emb = math.log(10000) / (half_dim - 1)
31
+ emb = torch.exp(torch.arange(half_dim, device=x.device) * -emb)
32
+ emb = x[:, None] * emb[None, :]
33
+ return torch.cat((emb.sin(), emb.cos()), dim=-1)
34
+
35
+
36
+ class RotaryPosEmb(nn.Module):
37
+ """Rotary position embedding over each attention head's (S, head_dim) slice."""
38
+
39
+ def __init__(self, dim):
40
+ super().__init__()
41
+ self.dim = dim
42
+ self.register_buffer('inv_freq', 1.0 / (10000 ** (torch.arange(0, dim, 2).float() / dim)))
43
+
44
+ def forward(self, x): # x: (B, heads, S, head_dim)
45
+ t = torch.arange(x.shape[-2], device=x.device, dtype=self.inv_freq.dtype)
46
+ freqs = torch.outer(t, self.inv_freq.to(x.device))
47
+ emb = torch.cat([freqs, freqs], dim=-1)
48
+ cos, sin = emb.cos()[None, None], emb.sin()[None, None]
49
+ x1, x2 = x[..., : x.shape[-1] // 2], x[..., x.shape[-1] // 2:]
50
+ return x * cos + torch.cat([-x2, x1], dim=-1) * sin
51
+
52
+
53
+ class Mlp(nn.Module):
54
+ """The ViT MLP (timm.layers.Mlp) as DextraVerse's nets/layers.py."""
55
+
56
+ def __init__(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.0):
57
+ super().__init__()
58
+ out_features = out_features or in_features
59
+ hidden_features = hidden_features or in_features
60
+ self.fc1 = nn.Linear(in_features, hidden_features)
61
+ self.act = act_layer()
62
+ self.drop1 = nn.Dropout(drop)
63
+ self.norm = nn.Identity()
64
+ self.fc2 = nn.Linear(hidden_features, out_features)
65
+ self.drop2 = nn.Dropout(drop)
66
+
67
+ def forward(self, x):
68
+ return self.drop2(self.fc2(self.norm(self.drop1(self.act(self.fc1(x))))))
69
+
70
+
71
+ class Attention(nn.Module):
72
+ def __init__(self, dim, num_heads=8, qkv_bias=True, attn_drop=0.0, proj_drop=0.0, qk_norm=True):
73
+ super().__init__()
74
+ assert dim % num_heads == 0, 'dim should be divisible by num_heads'
75
+ self.num_heads = num_heads
76
+ self.head_dim = dim // num_heads
77
+ self.attn_drop = attn_drop
78
+ self.proj_drop = nn.Dropout(proj_drop)
79
+ self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
80
+ self.q_norm = nn.LayerNorm(self.head_dim) if qk_norm else nn.Identity()
81
+ self.k_norm = nn.LayerNorm(self.head_dim) if qk_norm else nn.Identity()
82
+ self.proj = nn.Linear(dim, dim)
83
+ self.rope = RotaryPosEmb(self.head_dim)
84
+
85
+ def forward(self, x):
86
+ B, S, C = x.shape
87
+ qkv = self.qkv(x).reshape(B, S, 3, self.num_heads, self.head_dim).permute(2, 0, 3, 1, 4)
88
+ q, k, v = qkv.unbind(0)
89
+ q, k = self.rope(self.q_norm(q)), self.rope(self.k_norm(k))
90
+ x = F.scaled_dot_product_attention(q, k, v, dropout_p=self.attn_drop if self.training else 0.0)
91
+ return self.proj_drop(self.proj(x.transpose(1, 2).reshape(B, S, C)))
92
+
93
+
94
+ class AdaLNBlock(nn.Module):
95
+ """Adaptive-LayerNorm transformer block as in DiT (Peebles et al. 2022)."""
96
+
97
+ def __init__(self, dim, num_heads, mlp_ratio=4.0, dropout=0.0):
98
+ super().__init__()
99
+ self.norm1 = nn.LayerNorm(dim, elementwise_affine=False, eps=1e-6)
100
+ self.attn = Attention(dim, num_heads=num_heads, attn_drop=dropout, proj_drop=dropout)
101
+ self.norm2 = nn.LayerNorm(dim, elementwise_affine=False, eps=1e-6)
102
+ self.mlp = Mlp(in_features=dim, hidden_features=int(dim * mlp_ratio),
103
+ act_layer=lambda: nn.GELU(approximate='tanh'), drop=0)
104
+ self.ada_ln = nn.Sequential(nn.SiLU(), nn.Linear(dim, 6 * dim))
105
+ self.dim = dim
106
+ nn.init.constant_(self.ada_ln[-1].weight, 0)
107
+ nn.init.constant_(self.ada_ln[-1].bias, 0)
108
+
109
+ def forward(self, x, t, c):
110
+ B = x.shape[0]
111
+ gamma1, gamma2, scale1, scale2, shift1, shift2 = self.ada_ln(F.silu(t + c)).view(B, 6, 1, self.dim).unbind(1)
112
+ x = x + self.attn(self.norm1(x).mul(scale1.add(1)).add_(shift1)).mul_(gamma1)
113
+ return x + self.mlp(self.norm2(x).mul(scale2.add(1)).add_(shift2)).mul_(gamma2)
114
+
115
+
116
+ class FlowTransformer(nn.Module):
117
+ """DiT action backbone: AdaLN blocks over the H action tokens, conditioned on time_embed(t) + cond_embed(obs).
118
+
119
+ As DextraVerse, the generic Xavier initialization runs after the blocks' zero init of their AdaLN projection, so
120
+ it is not the full AdaLN-Zero recipe (kept: it is the reference's training behaviour)."""
121
+
122
+ def __init__(self, input_dim, condition_dim, hidden_dim, output_dim, num_layers, num_heads, mlp_ratio=4.0,
123
+ dropout=0.1, time_embed_dim=256):
124
+ super().__init__()
125
+ self.input_proj = nn.Linear(input_dim, hidden_dim)
126
+ self.time_embed = nn.Sequential(SinusoidalPosEmb(time_embed_dim), nn.Linear(time_embed_dim, time_embed_dim * 4),
127
+ nn.Mish(), nn.Linear(time_embed_dim * 4, hidden_dim))
128
+ self.cond_embed = nn.Linear(condition_dim, hidden_dim)
129
+ self.transformer_blocks = nn.ModuleList(
130
+ [AdaLNBlock(hidden_dim, num_heads, mlp_ratio=mlp_ratio, dropout=dropout) for _ in range(num_layers)])
131
+ self.norm = nn.LayerNorm(hidden_dim)
132
+ self.out_proj = nn.Linear(hidden_dim, output_dim)
133
+
134
+ def basic_init(module):
135
+ if isinstance(module, nn.Linear):
136
+ nn.init.xavier_uniform_(module.weight)
137
+ if module.bias is not None:
138
+ nn.init.constant_(module.bias, 0)
139
+
140
+ self.apply(basic_init)
141
+ nn.init.normal_(self.time_embed[1].weight, std=0.02)
142
+ nn.init.normal_(self.time_embed[3].weight, std=0.02)
143
+
144
+ def forward(self, x, t, global_cond):
145
+ if not torch.is_tensor(t):
146
+ t = torch.tensor(t, device=x.device)
147
+ if t.ndim == 0:
148
+ t = t.expand(x.shape[0])
149
+ x = self.input_proj(x)
150
+ t = self.time_embed(t)
151
+ c = self.cond_embed(global_cond)
152
+ for block in self.transformer_blocks:
153
+ x = block(x, t, c)
154
+ return self.out_proj(self.norm(x))
155
+
156
+
157
+ def replace_batchnorm_with_groupnorm(module: nn.Module) -> nn.Module:
158
+ """DextraVerse's use_group_norm: every BatchNorm2d becomes GroupNorm(num_features // 16 groups)."""
159
+ for name, child in module.named_children():
160
+ if isinstance(child, nn.BatchNorm2d):
161
+ setattr(module, name, nn.GroupNorm(num_groups=child.num_features // 16, num_channels=child.num_features))
162
+ else:
163
+ replace_batchnorm_with_groupnorm(child)
164
+ return module
165
+
166
+
167
+ NORMS = ('group', 'frozen_bn')
168
+
169
+
170
+ def get_resnet(name='resnet18', weights=None, norm='group') -> nn.Module:
171
+ """A torchvision ResNet with its classifier removed: (N, 3, H, W) -> (N, 512) globally pooled.
172
+
173
+ weights: torchvision weights name (e.g. 'IMAGENET1K_V1') to initialize from, or None (random initialization;
174
+ a checkpoint load supplies every weight). norm: 'group' replaces every BatchNorm with GroupNorm (DextraVerse's
175
+ use_group_norm, for training from scratch: pretrained BatchNorm statistics and affine are dropped); 'frozen_bn'
176
+ keeps BatchNorm frozen at its (pretrained) running statistics and affine, the convolutions fine-tune."""
177
+ if norm == 'frozen_bn':
178
+ from .act_nets import FrozenBatchNorm2d
179
+ resnet = getattr(torchvision.models, name)(weights=weights, norm_layer=FrozenBatchNorm2d)
180
+ elif norm == 'group':
181
+ resnet = replace_batchnorm_with_groupnorm(getattr(torchvision.models, name)(weights=weights))
182
+ else:
183
+ raise ValueError(f'norm must be one of {NORMS}, got {norm!r}')
184
+ resnet.fc = nn.Identity()
185
+ return resnet
186
+
187
+
188
+ class ImageStateEncoder(nn.Module):
189
+ """DextraVerse MultiImageObsEncoder as configured for FM-DiT / DDPM-DiT: one ResNet shared by every camera
190
+ (share_rgb_model; `get_resnet` weights and norm), ImageNet normalization of the [0, 1] images, the pooled features
191
+ of each camera in the contract's camera order, then the normalized state.
192
+ Output (B, n_cameras * 512 + state_dim)."""
193
+
194
+ def __init__(self, n_cameras, state_dim, backbone='resnet18', weights=None, norm='group'):
195
+ super().__init__()
196
+ self.rgb = get_resnet(backbone, weights=weights, norm=norm)
197
+ self.n_cameras, self.state_dim = int(n_cameras), int(state_dim)
198
+ self.register_buffer('mean', torch.tensor(IMAGENET_MEAN).view(1, 3, 1, 1), persistent=False)
199
+ self.register_buffer('std', torch.tensor(IMAGENET_STD).view(1, 3, 1, 1), persistent=False)
200
+ self.feature_dim = self.n_cameras * (512 if backbone in ('resnet18', 'resnet34') else 2048) + self.state_dim
201
+
202
+ def forward(self, images, state):
203
+ """images (B, n_cameras, 3, H, W) in [0, 1]; state (B, state_dim) normalized."""
204
+ B = images.shape[0]
205
+ x = (images.flatten(0, 1) - self.mean) / self.std
206
+ features = self.rgb(x).reshape(B, -1) # camera features side by side, in camera order
207
+ return torch.cat([features, state], dim=-1)
code/hb_baselines_v1/baselines/policies.py ADDED
@@ -0,0 +1,281 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The three baseline policies behind one contract, ported from DextraVerse's imitation-learning core:
2
+
3
+ - `fm_dit`, `FMDiTPolicy` (reference: dextraverse/algorithms/il/models/fm_dit.py + fm_base.py): flow matching, MSE on
4
+ the velocity `a - noise`; sampled with 10 midpoint steps from noise (20 DiT calls);
5
+ - `dp`, `DPPolicy` (ddpm_dit.py + ddpm_base.py: Diffusion Policy with a DiT backbone): DDPM epsilon MSE over 100
6
+ steps; sampled with the 100 DDPM steps;
7
+ - `act`, `ACTPolicy` (act.py + nets/act/): L1 + `kl_weight` x KL of the CVAE; decoded from the zero latent in one pass.
8
+
9
+ Contract (every policy): `compute_loss(images, state, action) -> (loss, metrics)` and
10
+ `predict(images, state, generator) -> (action, normalized action)`; images (B, n_cameras, 3, H, W) float in [0, 1],
11
+ state (B, state_dim) and action (B, horizon, action_dim) in physical units. Each policy owns its `Normalizer`
12
+ (DextraVerse's LinearNormalizer 'limits' mode: state and action to [-1, 1] per dimension, constant dimensions shifted
13
+ to 0), saved in its state dict. One observation frame (n_obs_steps 1: the current images and state); the chunk's row 0
14
+ is the action of the observation's own frame (the LeRobot rows t..t+horizon-1). The configuration that builds a
15
+ policy is a plain dict (`PolicyConfig` fields), stored in the checkpoint.
16
+
17
+ The ResNet18 starts from torchvision's ImageNet-1k weights (`backbone_weights` 'IMAGENET1K_V1', the default since
18
+ 2026-11-16) and is fine-tuned with the policy, its BatchNorm frozen at the pretrained statistics (`backbone_norm`
19
+ 'frozen_bn', as ACT always does); `backbone_weights` None trains from scratch with DextraVerse's GroupNorm encoder
20
+ (FM-DiT, DP) or ACT's frozen-BatchNorm one. A checkpoint load builds the structure without fetching any weights.
21
+ """
22
+ from __future__ import annotations
23
+
24
+ from dataclasses import asdict, dataclass, field, replace
25
+
26
+ import torch
27
+ import torch.nn as nn
28
+ import torch.nn.functional as F
29
+
30
+ from .act_nets import DETRVAE, Transformer, build_backbone, build_style_encoder, kl_divergence
31
+ from .nets import IMAGENET_MEAN, IMAGENET_STD, FlowTransformer, ImageStateEncoder
32
+
33
+
34
+ @dataclass
35
+ class PolicyConfig:
36
+ """Everything that builds a policy (stored in the checkpoint's config.json). Model defaults are DextraVerse's
37
+ configs/algorithm/il/model/{fm_dit,ddpm_dit,act}.yaml."""
38
+ policy: str
39
+ n_cameras: int
40
+ state_dim: int
41
+ action_dim: int
42
+ horizon: int = 16
43
+ backbone: str = 'resnet18'
44
+ backbone_weights: str | None = 'IMAGENET1K_V1' # torchvision weights to initialize from; None = from scratch
45
+ backbone_norm: str | None = None # 'frozen_bn' or 'group'; None: frozen_bn when pretrained or ACT, else group
46
+ # FM-DiT / DP (DiT backbone)
47
+ hidden_dim: int = 384
48
+ num_layers: int = 4
49
+ num_heads: int = 8
50
+ mlp_ratio: float = 4.0
51
+ dropout: float = 0.1
52
+ time_embed_dim: int = 128
53
+ num_inference_steps: int | None = None # None: FM-DiT 10 midpoint steps, DP every training timestep (100)
54
+ # DP noise schedule (diffusers DDPMScheduler)
55
+ num_train_timesteps: int = 100
56
+ beta_schedule: str = 'squaredcos_cap_v2'
57
+ prediction_type: str = 'epsilon'
58
+ clip_sample: bool = True
59
+ # ACT
60
+ act_hidden_dim: int = 512
61
+ dim_feedforward: int = 3200
62
+ enc_layers: int = 4
63
+ dec_layers: int = 7
64
+ nheads: int = 8
65
+ latent_dim: int = 32
66
+ kl_weight: float = 10.0
67
+ extra: dict = field(default_factory=dict)
68
+
69
+ def __post_init__(self):
70
+ if self.backbone_norm is None:
71
+ self.backbone_norm = 'frozen_bn' if self.backbone_weights or self.policy == 'act' else 'group'
72
+ if self.policy == 'act' and self.backbone_norm != 'frozen_bn':
73
+ raise ValueError("ACT's backbone keeps frozen BatchNorm (backbone_norm 'frozen_bn')")
74
+
75
+ @classmethod
76
+ def from_dict(cls, data: dict) -> PolicyConfig:
77
+ known = {k: v for k, v in data.items() if k in cls.__dataclass_fields__}
78
+ return cls(**known)
79
+
80
+ def to_dict(self) -> dict:
81
+ return asdict(self)
82
+
83
+
84
+ class Normalizer(nn.Module):
85
+ """Per-dimension affine map to [-1, 1] from the training data's min/max (DextraVerse LinearNormalizer, mode
86
+ 'limits', fit_offset, range_eps 1e-4): x_n = x * scale + offset; a dimension whose range is below range_eps gets
87
+ scale 1 and maps its value to 0."""
88
+
89
+ def __init__(self, dim: int):
90
+ super().__init__()
91
+ self.register_buffer('scale', torch.ones(dim))
92
+ self.register_buffer('offset', torch.zeros(dim))
93
+
94
+ @torch.no_grad()
95
+ def fit(self, minimum, maximum, range_eps=1e-4):
96
+ lo = torch.as_tensor(minimum, dtype=torch.float32)
97
+ hi = torch.as_tensor(maximum, dtype=torch.float32)
98
+ span = hi - lo
99
+ ignore = span < range_eps
100
+ span[ignore] = 2.0
101
+ scale = 2.0 / span
102
+ offset = -1.0 - scale * lo
103
+ offset[ignore] = -lo[ignore]
104
+ self.scale.copy_(scale)
105
+ self.offset.copy_(offset)
106
+
107
+ def normalize(self, x):
108
+ return x * self.scale + self.offset
109
+
110
+ def unnormalize(self, x):
111
+ return (x - self.offset) / self.scale
112
+
113
+
114
+ class BasePolicy(nn.Module):
115
+ name = ''
116
+ backbone_prefix = '' # state-dict prefix of the image backbone (its own learning rate in training)
117
+
118
+ def __init__(self, cfg: PolicyConfig, init_weights=None):
119
+ super().__init__()
120
+ self.cfg = cfg
121
+ self.state_norm = Normalizer(cfg.state_dim)
122
+ self.action_norm = Normalizer(cfg.action_dim)
123
+
124
+ def fit_normalizers(self, stats: dict):
125
+ self.state_norm.fit(stats['state']['min'], stats['state']['max'])
126
+ self.action_norm.fit(stats['action']['min'], stats['action']['max'])
127
+
128
+ @property
129
+ def sampling_steps(self) -> int:
130
+ return 1
131
+
132
+ def compute_loss(self, images, state, action):
133
+ raise NotImplementedError
134
+
135
+ def predict(self, images, state, generator=None):
136
+ raise NotImplementedError
137
+
138
+
139
+ class DiTPolicy(BasePolicy):
140
+ """Shared by FM-DiT and DP: the shared-ResNet image + state encoder and the AdaLN DiT over the action chunk
141
+ (global observation conditioning, DextraVerse obs_as_global_cond)."""
142
+ backbone_prefix = 'obs_encoder.rgb.'
143
+
144
+ def __init__(self, cfg: PolicyConfig, init_weights=None):
145
+ super().__init__(cfg)
146
+ self.obs_encoder = ImageStateEncoder(cfg.n_cameras, cfg.state_dim, backbone=cfg.backbone,
147
+ weights=init_weights, norm=cfg.backbone_norm)
148
+ self.model = FlowTransformer(input_dim=cfg.action_dim, condition_dim=self.obs_encoder.feature_dim,
149
+ hidden_dim=cfg.hidden_dim, output_dim=cfg.action_dim, num_layers=cfg.num_layers,
150
+ num_heads=cfg.num_heads, mlp_ratio=cfg.mlp_ratio, dropout=cfg.dropout,
151
+ time_embed_dim=cfg.time_embed_dim)
152
+
153
+ def condition(self, images, state):
154
+ return self.obs_encoder(images, self.state_norm.normalize(state))
155
+
156
+
157
+ class FMDiTPolicy(DiTPolicy):
158
+ """Flow-matching DiT (DextraVerse FlowMatchingDiTImagePolicy): x_t = (1 - t) noise + t a, t ~ U(0, 1), the DiT
159
+ regresses the velocity a - noise; sampling integrates t = 0 -> 1 with the midpoint rule."""
160
+ name = 'fm_dit'
161
+
162
+ @property
163
+ def sampling_steps(self) -> int:
164
+ return int(self.cfg.num_inference_steps or 10)
165
+
166
+ def compute_loss(self, images, state, action):
167
+ cond = self.condition(images, state)
168
+ target = self.action_norm.normalize(action)
169
+ noise = torch.randn_like(target)
170
+ t = torch.rand(target.shape[0], device=target.device)
171
+ te = t.view(-1, 1, 1)
172
+ x_t = (1 - te) * noise + te * target
173
+ loss = F.mse_loss(self.model(x_t, t, cond), target - noise)
174
+ return loss, dict(fm=float(loss.detach()))
175
+
176
+ @torch.no_grad()
177
+ def predict(self, images, state, generator=None):
178
+ cond = self.condition(images, state)
179
+ B, steps = images.shape[0], self.sampling_steps
180
+ x = torch.randn((B, self.cfg.horizon, self.cfg.action_dim), device=cond.device, dtype=cond.dtype,
181
+ generator=generator)
182
+ times = torch.linspace(0, 1.0, steps + 1, device=cond.device)
183
+ for i in range(steps):
184
+ t0 = times[i].view(1).expand(B)
185
+ t1 = times[i + 1].view(1).expand(B)
186
+ dt = (t1 - t0).view(-1, 1, 1)
187
+ x_mid = x + self.model(x, t0, cond) * dt / 2
188
+ x = x + dt * self.model(x_mid, t0 + (t1 - t0) / 2, cond)
189
+ return self.action_norm.unnormalize(x), x
190
+
191
+
192
+ class DPPolicy(DiTPolicy):
193
+ """Diffusion Policy with the DiT backbone (DextraVerse DiffusionDiTImagePolicy, ddpm_dit.yaml): DDPM with 100
194
+ training steps, squaredcos_cap_v2 betas, fixed_small variance, clipped samples, epsilon prediction; sampled with
195
+ every training step (100 DiT calls)."""
196
+ name = 'dp'
197
+
198
+ def __init__(self, cfg: PolicyConfig, init_weights=None):
199
+ super().__init__(cfg, init_weights)
200
+ from diffusers.schedulers.scheduling_ddpm import DDPMScheduler
201
+ self.noise_scheduler = DDPMScheduler(num_train_timesteps=cfg.num_train_timesteps, beta_start=0.0001,
202
+ beta_end=0.02, beta_schedule=cfg.beta_schedule,
203
+ variance_type='fixed_small', clip_sample=cfg.clip_sample,
204
+ prediction_type=cfg.prediction_type)
205
+
206
+ @property
207
+ def sampling_steps(self) -> int:
208
+ return int(self.cfg.num_inference_steps or self.cfg.num_train_timesteps)
209
+
210
+ def compute_loss(self, images, state, action):
211
+ cond = self.condition(images, state)
212
+ target = self.action_norm.normalize(action)
213
+ noise = torch.randn_like(target)
214
+ t = torch.randint(0, self.noise_scheduler.config.num_train_timesteps, (target.shape[0],),
215
+ device=target.device).long()
216
+ noisy = self.noise_scheduler.add_noise(target, noise, t)
217
+ pred = self.model(noisy, t, cond)
218
+ goal = noise if self.cfg.prediction_type == 'epsilon' else target
219
+ loss = F.mse_loss(pred, goal)
220
+ return loss, dict(ddpm=float(loss.detach()))
221
+
222
+ @torch.no_grad()
223
+ def predict(self, images, state, generator=None):
224
+ cond = self.condition(images, state)
225
+ scheduler = self.noise_scheduler
226
+ x = torch.randn((images.shape[0], self.cfg.horizon, self.cfg.action_dim), device=cond.device,
227
+ dtype=cond.dtype, generator=generator)
228
+ scheduler.set_timesteps(self.sampling_steps, device=cond.device)
229
+ for t in scheduler.timesteps:
230
+ x = scheduler.step(self.model(x, t.to(cond.device), cond), t, x, generator=generator).prev_sample
231
+ return self.action_norm.unnormalize(x), x
232
+
233
+
234
+ class ACTPolicy(BasePolicy):
235
+ """ACT (DextraVerse ACTImagePolicy, act.yaml): ImageNet-normalized cameras through one ResNet with frozen
236
+ BatchNorm (spatial tokens, no pooling), the normalized state as qpos, a CVAE style encoder over the action chunk in
237
+ training, the DETR decoder with `horizon` queries; loss L1 + kl_weight x KL; inference decodes from the zero
238
+ latent (deterministic)."""
239
+ name = 'act'
240
+ backbone_prefix = 'model.backbone.'
241
+
242
+ def __init__(self, cfg: PolicyConfig, init_weights=None):
243
+ super().__init__(cfg)
244
+ transformer = Transformer(d_model=cfg.act_hidden_dim, nhead=cfg.nheads, num_encoder_layers=cfg.enc_layers,
245
+ num_decoder_layers=cfg.dec_layers, dim_feedforward=cfg.dim_feedforward,
246
+ dropout=cfg.dropout, return_intermediate_dec=False)
247
+ self.model = DETRVAE(build_backbone(cfg.backbone, hidden_dim=cfg.act_hidden_dim, weights=init_weights),
248
+ transformer,
249
+ build_style_encoder(cfg.act_hidden_dim, cfg.nheads, cfg.dim_feedforward, cfg.enc_layers,
250
+ cfg.dropout),
251
+ qpos_dim=cfg.state_dim, action_dim=cfg.action_dim, num_queries=cfg.horizon,
252
+ latent_dim=cfg.latent_dim)
253
+ self.register_buffer('mean', torch.tensor(IMAGENET_MEAN).view(1, 1, 3, 1, 1), persistent=False)
254
+ self.register_buffer('std', torch.tensor(IMAGENET_STD).view(1, 1, 3, 1, 1), persistent=False)
255
+
256
+ def compute_loss(self, images, state, action):
257
+ target = self.action_norm.normalize(action)
258
+ a_hat, (mu, logvar) = self.model(self.state_norm.normalize(state), (images - self.mean) / self.std,
259
+ actions=target)
260
+ l1 = F.l1_loss(a_hat, target)
261
+ kl = kl_divergence(mu, logvar)
262
+ return l1 + self.cfg.kl_weight * kl, dict(l1=float(l1.detach()), kl=float(kl.detach()))
263
+
264
+ @torch.no_grad()
265
+ def predict(self, images, state, generator=None): # noqa: ARG002 - deterministic (zero latent)
266
+ a_hat, _ = self.model(self.state_norm.normalize(state), (images - self.mean) / self.std)
267
+ return self.action_norm.unnormalize(a_hat), a_hat
268
+
269
+
270
+ POLICIES = {cls.name: cls for cls in (FMDiTPolicy, DPPolicy, ACTPolicy)}
271
+ LABELS = {'fm_dit': 'FM-DiT', 'dp': 'DP (DDPM-DiT)', 'act': 'ACT'}
272
+
273
+
274
+ def build_policy(cfg: PolicyConfig | dict, pretrained: bool = True) -> BasePolicy:
275
+ """The policy of `cfg`; pretrained=True initializes the backbone from cfg.backbone_weights (training),
276
+ False builds the same structure with random weights (a checkpoint load, which must not download anything)."""
277
+ cfg = cfg if isinstance(cfg, PolicyConfig) else PolicyConfig.from_dict(cfg)
278
+ if cfg.policy not in POLICIES:
279
+ raise KeyError(f'unknown baseline {cfg.policy!r}; one of {sorted(POLICIES)}')
280
+ cfg = replace(cfg) # a copy (resolved backbone_norm)
281
+ return POLICIES[cfg.policy](cfg, init_weights=cfg.backbone_weights if pretrained else None)
code/hb_baselines_v1/baselines/requirements.txt ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Environment of the FM-DiT / DP / ACT baselines (training and the policy server), Python 3.12, CUDA 12.8 wheels:
2
+ # uv venv <env> --python 3.12
3
+ # uv pip install --python <env>/bin/python -r evaluation/baselines/requirements.txt \
4
+ # --extra-index-url https://download.pytorch.org/whl/cu128 --index-strategy unsafe-best-match
5
+ torch==2.9.0+cu128
6
+ torchvision==0.24.0+cu128
7
+ diffusers==0.35.1
8
+ av==15.1.0
9
+ pyarrow==23.0.1
10
+ numpy==2.2.6
11
+ opencv-python-headless==4.11.0.86
12
+ pytest==8.4.2
code/hb_baselines_v1/baselines/runtime.py ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Checkpoints and inference of the baselines.
2
+
3
+ A checkpoint is a directory written by train.py (`<run>/checkpoints/step-NNNNNN/`, `final` links the last one):
4
+
5
+ - `model.pt`: torch.save of {format, config (the config.json text), state_dict}: the EMA weights (or the raw ones when
6
+ EMA is off) with the policy's state and action normalizers, loaded strictly with `weights_only=True`;
7
+ - `config.json`: the policy configuration, the contract (robot, cameras, state and action columns), the image size,
8
+ the training task text, dataset, episodes, recipe and statistics;
9
+ - `protocol.json`: model.pt's size and SHA-256.
10
+
11
+ `BaselinePolicy.predict` takes the live frames of the checkpoint's cameras (any size; resized with data.resize_frame,
12
+ as the training cache), the physical state of the contract and a seed, and returns the physical (horizon,
13
+ action_dim) chunk with its normalized form; row 0 is the action of the observation's frame.
14
+ """
15
+ from __future__ import annotations
16
+
17
+ import hashlib
18
+ import json
19
+ from pathlib import Path
20
+
21
+ import numpy as np
22
+
23
+ FORMAT = 'hb_baselines/1'
24
+
25
+
26
+ def sha256(path) -> str:
27
+ digest = hashlib.sha256()
28
+ with Path(path).open('rb') as stream:
29
+ for block in iter(lambda: stream.read(16 << 20), b''):
30
+ digest.update(block)
31
+ return digest.hexdigest()
32
+
33
+
34
+ def save_checkpoint(folder, policy, config: dict):
35
+ """Write model.pt, config.json and protocol.json of `policy` (an nn.Module) into `folder`."""
36
+ import torch
37
+ folder = Path(folder)
38
+ folder.mkdir(parents=True, exist_ok=True)
39
+ text = json.dumps(dict(config, format=FORMAT), indent=1)
40
+ state = {k: v.detach().cpu().contiguous() for k, v in policy.state_dict().items()}
41
+ torch.save(dict(format=FORMAT, config=text, state_dict=state), folder / 'model.pt.tmp')
42
+ (folder / 'model.pt.tmp').replace(folder / 'model.pt')
43
+ (folder / 'config.json').write_text(text)
44
+ (folder / 'protocol.json').write_text(json.dumps(dict(
45
+ format=FORMAT, step=config.get('step'), policy=config['policy']['policy'],
46
+ files={'model.pt': dict(bytes=(folder / 'model.pt').stat().st_size, sha256=sha256(folder / 'model.pt'))}),
47
+ indent=1))
48
+
49
+
50
+ def load_checkpoint(folder, device='cpu', verify=True):
51
+ """(policy in eval mode on `device`, config dict) of a checkpoint directory."""
52
+ import torch
53
+ from .policies import build_policy
54
+ folder = Path(folder)
55
+ if verify:
56
+ protocol = json.loads((folder / 'protocol.json').read_text())
57
+ entry = protocol['files']['model.pt']
58
+ if (folder / 'model.pt').stat().st_size != entry['bytes']:
59
+ raise ValueError(f'{folder}/model.pt differs from protocol.json')
60
+ payload = torch.load(folder / 'model.pt', map_location='cpu', weights_only=True)
61
+ if payload.get('format') != FORMAT:
62
+ raise ValueError(f"{folder}: format {payload.get('format')!r}, expected {FORMAT}")
63
+ config = json.loads(payload['config'])
64
+ if json.loads((folder / 'config.json').read_text()) != config:
65
+ raise ValueError(f'{folder}: config.json differs from the one inside model.pt')
66
+ policy = build_policy(config['policy'], pretrained=False) # every weight comes from model.pt
67
+ policy.load_state_dict(payload['state_dict'], strict=True)
68
+ return policy.to(device).eval(), config
69
+
70
+
71
+ class BaselinePolicy:
72
+ """A loaded checkpoint for closed-loop inference (one request at a time)."""
73
+
74
+ def __init__(self, checkpoint, device='cuda:0', policy: str | None = None, robot: str | None = None):
75
+ import torch
76
+ self.torch = torch
77
+ self.device = torch.device(device)
78
+ self.model, self.config = load_checkpoint(checkpoint, device=self.device)
79
+ self.checkpoint = str(Path(checkpoint).resolve())
80
+ name, contract = self.config['policy']['policy'], self.config['contract']
81
+ if policy is not None and name != policy:
82
+ raise ValueError(f'{checkpoint} is a {name!r} checkpoint, not {policy!r}')
83
+ if robot is not None and contract['robot'] != robot:
84
+ raise ValueError(f"{checkpoint} was trained for {contract['robot']!r} ({contract['name']}), not {robot!r}")
85
+ self.name, self.contract = name, contract
86
+ self.cameras = tuple(contract['cameras'])
87
+ self.image_hw = tuple(self.config['image_hw'])
88
+ self.horizon = int(self.config['policy']['horizon'])
89
+ self.state_dim, self.action_dim = int(contract['state_dim']), int(contract['action_dim'])
90
+ self.sampling_steps = int(self.model.sampling_steps)
91
+ self.task_text = self.config.get('task_text')
92
+
93
+ def info(self) -> dict:
94
+ return dict(model=self.name, checkpoint=self.checkpoint, step=self.config.get('step'),
95
+ contract=self.contract['name'], views=list(self.cameras), image_hw=list(self.image_hw),
96
+ action_chunk=self.horizon, action_dim=self.action_dim, state_dim=self.state_dim,
97
+ sampling_steps=self.sampling_steps, task_text=self.task_text,
98
+ training_data=self.config.get('dataset', {}).get('name'))
99
+
100
+ def images_tensor(self, images: dict):
101
+ from .data import resize_frame
102
+ missing = [c for c in self.cameras if c not in images]
103
+ if missing:
104
+ raise KeyError(f'missing cameras {missing}; the checkpoint reads {list(self.cameras)}')
105
+ frames = np.stack([resize_frame(np.asarray(images[c]), self.image_hw) for c in self.cameras])
106
+ x = self.torch.from_numpy(frames).to(self.device).permute(0, 3, 1, 2).float().div_(255.)
107
+ return x.unsqueeze(0) # (1, n_cameras, 3, h, w)
108
+
109
+ def predict(self, images: dict, state, seed: int = 0):
110
+ """(action (horizon, action_dim) float32 physical, normalized (horizon, action_dim) float32)."""
111
+ torch = self.torch
112
+ state = np.asarray(state, np.float32).reshape(-1)
113
+ if state.shape != (self.state_dim,) or not np.isfinite(state).all():
114
+ raise ValueError(f'expected a finite state of {self.state_dim} values, got {state.shape}')
115
+ generator = torch.Generator(device=self.device)
116
+ generator.manual_seed(int(seed) % (2 ** 63 - 1))
117
+ with torch.inference_mode():
118
+ action, raw = self.model.predict(self.images_tensor(images),
119
+ torch.from_numpy(state)[None].to(self.device), generator=generator)
120
+ action = action[0].float().cpu().numpy()
121
+ if action.shape != (self.horizon, self.action_dim) or not np.isfinite(action).all():
122
+ raise ValueError(f'invalid prediction {action.shape}')
123
+ return np.ascontiguousarray(action, np.float32), raw[0].float().cpu().numpy().astype(np.float32)