Update model card: checkpoint table and usage
Browse files
README.md
CHANGED
|
@@ -13,29 +13,66 @@ library_name: diffsynth
|
|
| 13 |
|
| 14 |
# Echo-Memory — Wan 2.1 1.3B memory baseline checkpoints
|
| 15 |
|
| 16 |
-
Paper-aligned **epoch-0** fine-tunes for
|
| 17 |
|
| 18 |
**Backbone:** [Wan-AI/Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B)
|
| 19 |
-
**Training:** static in-domain pool
|
| 20 |
-
**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## Download
|
| 23 |
|
| 24 |
```bash
|
| 25 |
pip install -U "huggingface_hub[cli]"
|
|
|
|
|
|
|
| 26 |
huggingface-cli download Echo-Team/Echo-Memory context_k1/epoch-0.safetensors --local-dir ./ckpts
|
|
|
|
|
|
|
|
|
|
| 27 |
```
|
| 28 |
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
```bash
|
| 32 |
export WAN_BASE_MODEL=/path/to/Wan2.1-T2V-1.3B
|
| 33 |
export DATASET_BASE_PATH=data/Context-as-Memory-Dataset
|
| 34 |
-
export
|
|
|
|
|
|
|
|
|
|
| 35 |
bash eval/v2/run_static_consistency_loop_and_revisit.sh
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
```
|
| 37 |
|
| 38 |
-
|
|
|
|
|
|
|
| 39 |
|
| 40 |
## Citation
|
| 41 |
|
|
|
|
| 13 |
|
| 14 |
# Echo-Memory — Wan 2.1 1.3B memory baseline checkpoints
|
| 15 |
|
| 16 |
+
Paper-aligned **epoch-0** fine-tunes for [Echo-Memory](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory) ([project page](https://echo-team-joy-future-academy-jd.github.io/Echo-Memory/)).
|
| 17 |
|
| 18 |
**Backbone:** [Wan-AI/Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B)
|
| 19 |
+
**Training:** static in-domain pool · 1 epoch · **30,000 steps** · 640×352 · 81-frame chunks
|
| 20 |
+
**Layout:** `{row_id}/epoch-0.safetensors`
|
| 21 |
+
|
| 22 |
+
## Checkpoint index
|
| 23 |
+
|
| 24 |
+
| Family | Paper row | HF path | Steps |
|
| 25 |
+
| --- | --- | --- | ---: |
|
| 26 |
+
| Raw context | Context K=1 | `context_k1/epoch-0.safetensors` | 30,000 |
|
| 27 |
+
| Raw context | Context K=20 | `context_k20/epoch-0.safetensors` | 30,000 |
|
| 28 |
+
| Spatial | Spatial Memory | `spatial_mem/epoch-0.safetensors` | 30,000 |
|
| 29 |
+
| State-space | Block-wise SSM | `block_wise_ssm_two_chunk/epoch-0.safetensors` | 30,000 |
|
| 30 |
+
| State-space | Legacy Hybrid (VideoSSM) | `videossm_hybrid/epoch-0.safetensors` | 30,000 |
|
| 31 |
+
| Spatial | concat text (ablation) | `spatial_concat_text_two_chunk/epoch-0.safetensors` | 30,000 |
|
| 32 |
+
| Spatial | inject none (ablation) | `spatial_inject_none_two_chunk/epoch-0.safetensors` | 30,000 |
|
| 33 |
+
| Spatial | cross-attn t32 (ablation) | `spatial_cross_attn_readout_t32_g4_two_chunk/epoch-0.safetensors` | 30,000 |
|
| 34 |
+
| State-space | SSM ctx1 / every4 / hint21 | `ssm_ablation_ctx1_every4_hint21/epoch-0.safetensors` | 30,000 |
|
| 35 |
+
| State-space | SSM ctx5 / every1 / hint21 | `ssm_ablation_ctx5_every1_hint21/epoch-0.safetensors` | 30,000 |
|
| 36 |
+
| State-space | SSM ctx5 / every4 / hint81 | `ssm_ablation_ctx5_every4_hint81/epoch-0.safetensors` | 30,000 |
|
| 37 |
+
|
| 38 |
+
Context K=5 and FramePack compression rows are not yet released as `epoch-0` weights.
|
| 39 |
|
| 40 |
## Download
|
| 41 |
|
| 42 |
```bash
|
| 43 |
pip install -U "huggingface_hub[cli]"
|
| 44 |
+
|
| 45 |
+
# one row
|
| 46 |
huggingface-cli download Echo-Team/Echo-Memory context_k1/epoch-0.safetensors --local-dir ./ckpts
|
| 47 |
+
|
| 48 |
+
# all rows
|
| 49 |
+
huggingface-cli download Echo-Team/Echo-Memory --local-dir ./ckpts
|
| 50 |
```
|
| 51 |
|
| 52 |
+
Keep the row subdirectory in the local path (e.g. `./ckpts/spatial_mem/epoch-0.safetensors`).
|
| 53 |
+
|
| 54 |
+
## Use with Echo-Memory code
|
| 55 |
+
|
| 56 |
+
Clone [Echo-Memory](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory), install the environment, then:
|
| 57 |
|
| 58 |
```bash
|
| 59 |
export WAN_BASE_MODEL=/path/to/Wan2.1-T2V-1.3B
|
| 60 |
export DATASET_BASE_PATH=data/Context-as-Memory-Dataset
|
| 61 |
+
export PYTHONPATH=$PWD:${PYTHONPATH:-}
|
| 62 |
+
export CKPT=./ckpts/spatial_mem/epoch-0.safetensors
|
| 63 |
+
|
| 64 |
+
# in-domain replay + revisit
|
| 65 |
bash eval/v2/run_static_consistency_loop_and_revisit.sh
|
| 66 |
+
bash eval/v2/run_basic_replay_gt.sh
|
| 67 |
+
|
| 68 |
+
# open-domain revisit (first frames in repo)
|
| 69 |
+
PHASE=stage1 OOD_DIR=assets/opendomain_revisit \
|
| 70 |
+
bash eval/v2/revisit_suite/run_one_click_revisit_eval.sh
|
| 71 |
```
|
| 72 |
|
| 73 |
+
Memory runtime flags are inferred from the checkpoint path via `env/memory_baseline_runtime.py` — use the HF folder names above.
|
| 74 |
+
|
| 75 |
+
Full docs: [doc/checkpoints.md](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory/blob/main/doc/checkpoints.md)
|
| 76 |
|
| 77 |
## Citation
|
| 78 |
|