DavidSunok commited on
Commit
c3d9af6
路
verified 路
1 Parent(s): 5e635ed

docs: update audited SR results

Browse files
Files changed (1) hide show
  1. README.md +20 -29
README.md CHANGED
@@ -18,28 +18,21 @@ Search-Free World Models**.
18
  [Code](https://github.com/zju3dv/INTACT-JEPA) 路
19
  [Project page](https://zju3dv.github.io/INTACT-JEPA/)
20
 
21
- ## Recommended releases
22
-
23
- | Release | Actor context | Evaluation contract | PushT | Cube | Reacher | TwoRoom | Macro SR |
24
- |---|---|---|---:|---:|---:|---:|---:|
25
- | [**INTACT**](https://huggingface.co/INTACT-JEPA/INTACT/tree/main/INTACT) | Causal previous action | Stateful continuation | 86.44 +/- 1.07 | 100.00 +/- 0.00 | 97.78 +/- 0.51 | 98.78 +/- 0.69 | **95.75 +/- 0.52** |
26
- | [**INTACT, no previous action**](https://huggingface.co/INTACT-JEPA/INTACT/tree/main/INTACT-no-previous-action) | None | History-free cold start | 85.33 +/- 0.58 | 99.44 +/- 0.38 | 93.44 +/- 0.38 | 98.78 +/- 0.69 | **94.25 +/- 0.08** |
27
- | [**INTACT, unified E5**](https://huggingface.co/INTACT-JEPA/INTACT/tree/main/INTACT-unified) | Shared encoder, task heads | Official Direct | 80.22 +/- 1.26 | 99.56 +/- 0.77 | 95.67 +/- 1.76 | 82.11 +/- 4.11 | **89.39 +/- 0.77** |
28
-
29
- The first two releases are task-specific, full-data, one-epoch models evaluated with
30
- Official Direct control, `H=5`, `RH=5`, and zero search. Each value averages
31
- evaluation seeds `0/1/42` within training seeds `0/42/3072`; `+/-` is the
32
- sample standard deviation across the three training-seed means.
33
-
34
- The unified E5 release is the paper's controlled four-task shared-encoder
35
- experiment. It uses one shared encoder with task-specific heads and is listed
36
- separately because its epoch count and model-sharing protocol differ from the
37
- two task-specific E1 releases.
38
-
39
- The first two variants isolate the previous-action question cleanly. Standard INTACT
40
- uses strictly causal pre-`t` history at an explicitly declared continuation
41
- point and only policy-executed actions afterward. The history-free actor omits
42
- the previous-action channel in both training and evaluation.
43
 
44
  ## Download
45
 
@@ -58,20 +51,18 @@ hf download INTACT-JEPA/INTACT --include "INTACT-unified/*" \
58
 
59
  Each directory contains a model-specific README, auditable manifests, and
60
  SHA-256 checksums. The unified directory additionally contains the complete
61
- six-cell, 72-checkpoint comparison matrix.
62
 
63
  ## Unified E5 comparison
64
 
65
  | Controlled cell | Native interface | Official macro SR |
66
  |---|---|---:|
67
  | LeWM | CEM 300x30 | 66.17 |
68
- | Inverse only | Direct | 56.67 |
69
- | Waypoint intent only | Direct | 74.22 |
70
- | Goal intent only | Direct | 80.61 |
71
- | Waypoint INTACT | Direct | 76.42 |
72
- | **Goal-displacement INTACT** | **Direct** | **89.39** |
73
 
74
- All six cells use the same E5 shared-encoder protocol. Task-level results,
75
  per-seed statistics, checkpoint hashes, and runtime fingerprints are provided
76
  inside [`INTACT-unified`](https://huggingface.co/INTACT-JEPA/INTACT/tree/main/INTACT-unified).
77
  The immutable original layout also remains available at the frozen
 
18
  [Code](https://github.com/zju3dv/INTACT-JEPA) 路
19
  [Project page](https://zju3dv.github.io/INTACT-JEPA/)
20
 
21
+ ## Audited results
22
+
23
+ | Setting | Inference | PushT | Cube | Reacher | TwoRoom | Macro SR |
24
+ |---|---|---:|---:|---:|---:|---:|
25
+ | **Task-specific E1 INTACT** | **Direct, zero search** | 87.44 +/- 1.26 | 100.00 +/- 0.00 | 97.33 +/- 0.00 | 97.67 +/- 1.20 | **95.61 +/- 0.59** |
26
+ | **Task-specific E1 INTACT** | **Guarded A 128x3** | 91.56 +/- 0.69 | 99.67 +/- 0.33 | 97.56 +/- 0.51 | 97.56 +/- 1.26 | **96.58 +/- 0.44** |
27
+ | **Shared-encoder E5 INTACT** | **Direct, zero search** | 86.11 +/- 0.96 | 100.00 +/- 0.00 | 97.22 +/- 0.84 | 81.56 +/- 3.67 | **91.22 +/- 0.51** |
28
+ | [History-free E1 ablation](https://huggingface.co/INTACT-JEPA/INTACT/tree/main/INTACT-no-previous-action) | Direct, zero search | 85.33 +/- 0.58 | 99.44 +/- 0.38 | 93.44 +/- 0.38 | 98.78 +/- 0.69 | **94.25 +/- 0.08** |
29
+
30
+ All values use the corrected Official evaluation implementation. Standard
31
+ INTACT uses strictly causal pre-`t` action history at a declared continuation
32
+ point and appends every policy-executed action exactly once. The history-free
33
+ ablation removes the previous-action channel from both training and evaluation.
34
+ Guarded A is an optional local verifier around the Direct plan; it evaluates
35
+ 384 sampled candidates rather than the 9,000 used by CEM 300x30.
 
 
 
 
 
 
 
36
 
37
  ## Download
38
 
 
51
 
52
  Each directory contains a model-specific README, auditable manifests, and
53
  SHA-256 checksums. The unified directory additionally contains the complete
54
+ shared-encoder comparison matrix.
55
 
56
  ## Unified E5 comparison
57
 
58
  | Controlled cell | Native interface | Official macro SR |
59
  |---|---|---:|
60
  | LeWM | CEM 300x30 | 66.17 |
61
+ | Inverse only | Direct | 68.19 |
62
+ | Goal intent only | Direct | 84.92 |
63
+ | **Goal-displacement INTACT** | **Direct** | **91.22** |
 
 
64
 
65
+ All cells use the same E5 shared-encoder protocol. Task-level results,
66
  per-seed statistics, checkpoint hashes, and runtime fingerprints are provided
67
  inside [`INTACT-unified`](https://huggingface.co/INTACT-JEPA/INTACT/tree/main/INTACT-unified).
68
  The immutable original layout also remains available at the frozen