Fix QK norm to apply before RoPE, retrain released checkpoints

README.md

@@ -22,9 +22,9 @@ Prediction quality, measured by applying downstream task heads to the predicted
 | Method | Horizon | VSPW mIoU (↑) | Cityscapes mIoU (↑) | KITTI RMSE (↓) |
 |--------|---------|---------------|---------------------|----------------|
 | *Copy last (lower bound)* | *Short (1 frame)* | *51.2* | *53.5* | *3.76* |
-| DeltaWorld | Short (1 frame) | 56.3 (54.2) | 66.2 (64.2) | 2.95 (3.32) |
+| DeltaWorld | Short (1 frame) | 56.1 (54.2) | 65.9 (64.3) | 2.97 (3.36) |
 | *Copy last (lower bound)* | *Mid (3 frames)* | *44.3* | *39.6* | *4.86* |
-| DeltaWorld | Mid (3 frames) | 51.5 (46.6) | 55.3 (49.5) | 3.71 (4.74) |
+| DeltaWorld | Mid (3 frames) | 51.0 (45.8) | 54.8 (49.8) | 3.76 (5.14) |
 
 ## Usage
 

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9a5ea08d30dba656dc5e43e394ea7fad80a4ef9df0660159251897ee021e2e22
-size 342723187
+oid sha256:1ef52aa091abac376f896142bcb84a15fd73fc87656653d6409cd316a3b49219
+size 342727291