Wayne-King commited on
Commit
2b312e4
·
verified ·
1 Parent(s): 886e247

Update model card: checkpoint table and usage

Browse files
Files changed (1) hide show
  1. README.md +43 -6
README.md CHANGED
@@ -13,29 +13,66 @@ library_name: diffsynth
13
 
14
  # Echo-Memory — Wan 2.1 1.3B memory baseline checkpoints
15
 
16
- Paper-aligned **epoch-0** fine-tunes for the Echo-Memory controlled memory study ([GitHub](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory) · [project page](https://echo-team-joy-future-academy-jd.github.io/Echo-Memory/)).
17
 
18
  **Backbone:** [Wan-AI/Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B)
19
- **Training:** static in-domain pool, 1 epoch, **30,000 steps**, 640×352, 81-frame chunks
20
- **File layout:** `{row_id}/epoch-0.safetensors` — see `checkpoints.json` for metadata.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  ## Download
23
 
24
  ```bash
25
  pip install -U "huggingface_hub[cli]"
 
 
26
  huggingface-cli download Echo-Team/Echo-Memory context_k1/epoch-0.safetensors --local-dir ./ckpts
 
 
 
27
  ```
28
 
29
- ## Usage with Echo-Memory eval
 
 
 
 
30
 
31
  ```bash
32
  export WAN_BASE_MODEL=/path/to/Wan2.1-T2V-1.3B
33
  export DATASET_BASE_PATH=data/Context-as-Memory-Dataset
34
- export CKPT=./ckpts/context_k1/epoch-0.safetensors
 
 
 
35
  bash eval/v2/run_static_consistency_loop_and_revisit.sh
 
 
 
 
 
36
  ```
37
 
38
- Runtime memory flags are inferred from the checkpoint path via `env/memory_baseline_runtime.py`.
 
 
39
 
40
  ## Citation
41
 
 
13
 
14
  # Echo-Memory — Wan 2.1 1.3B memory baseline checkpoints
15
 
16
+ Paper-aligned **epoch-0** fine-tunes for [Echo-Memory](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory) ([project page](https://echo-team-joy-future-academy-jd.github.io/Echo-Memory/)).
17
 
18
  **Backbone:** [Wan-AI/Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B)
19
+ **Training:** static in-domain pool · 1 epoch · **30,000 steps** · 640×352 · 81-frame chunks
20
+ **Layout:** `{row_id}/epoch-0.safetensors`
21
+
22
+ ## Checkpoint index
23
+
24
+ | Family | Paper row | HF path | Steps |
25
+ | --- | --- | --- | ---: |
26
+ | Raw context | Context K=1 | `context_k1/epoch-0.safetensors` | 30,000 |
27
+ | Raw context | Context K=20 | `context_k20/epoch-0.safetensors` | 30,000 |
28
+ | Spatial | Spatial Memory | `spatial_mem/epoch-0.safetensors` | 30,000 |
29
+ | State-space | Block-wise SSM | `block_wise_ssm_two_chunk/epoch-0.safetensors` | 30,000 |
30
+ | State-space | Legacy Hybrid (VideoSSM) | `videossm_hybrid/epoch-0.safetensors` | 30,000 |
31
+ | Spatial | concat text (ablation) | `spatial_concat_text_two_chunk/epoch-0.safetensors` | 30,000 |
32
+ | Spatial | inject none (ablation) | `spatial_inject_none_two_chunk/epoch-0.safetensors` | 30,000 |
33
+ | Spatial | cross-attn t32 (ablation) | `spatial_cross_attn_readout_t32_g4_two_chunk/epoch-0.safetensors` | 30,000 |
34
+ | State-space | SSM ctx1 / every4 / hint21 | `ssm_ablation_ctx1_every4_hint21/epoch-0.safetensors` | 30,000 |
35
+ | State-space | SSM ctx5 / every1 / hint21 | `ssm_ablation_ctx5_every1_hint21/epoch-0.safetensors` | 30,000 |
36
+ | State-space | SSM ctx5 / every4 / hint81 | `ssm_ablation_ctx5_every4_hint81/epoch-0.safetensors` | 30,000 |
37
+
38
+ Context K=5 and FramePack compression rows are not yet released as `epoch-0` weights.
39
 
40
  ## Download
41
 
42
  ```bash
43
  pip install -U "huggingface_hub[cli]"
44
+
45
+ # one row
46
  huggingface-cli download Echo-Team/Echo-Memory context_k1/epoch-0.safetensors --local-dir ./ckpts
47
+
48
+ # all rows
49
+ huggingface-cli download Echo-Team/Echo-Memory --local-dir ./ckpts
50
  ```
51
 
52
+ Keep the row subdirectory in the local path (e.g. `./ckpts/spatial_mem/epoch-0.safetensors`).
53
+
54
+ ## Use with Echo-Memory code
55
+
56
+ Clone [Echo-Memory](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory), install the environment, then:
57
 
58
  ```bash
59
  export WAN_BASE_MODEL=/path/to/Wan2.1-T2V-1.3B
60
  export DATASET_BASE_PATH=data/Context-as-Memory-Dataset
61
+ export PYTHONPATH=$PWD:${PYTHONPATH:-}
62
+ export CKPT=./ckpts/spatial_mem/epoch-0.safetensors
63
+
64
+ # in-domain replay + revisit
65
  bash eval/v2/run_static_consistency_loop_and_revisit.sh
66
+ bash eval/v2/run_basic_replay_gt.sh
67
+
68
+ # open-domain revisit (first frames in repo)
69
+ PHASE=stage1 OOD_DIR=assets/opendomain_revisit \
70
+ bash eval/v2/revisit_suite/run_one_click_revisit_eval.sh
71
  ```
72
 
73
+ Memory runtime flags are inferred from the checkpoint path via `env/memory_baseline_runtime.py` — use the HF folder names above.
74
+
75
+ Full docs: [doc/checkpoints.md](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory/blob/main/doc/checkpoints.md)
76
 
77
  ## Citation
78