Wayne-King commited on
Commit
afd288e
·
verified ·
1 Parent(s): 920fe0f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +58 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: diffsynth
4
+ pipeline_tag: text-to-video
5
+ tags:
6
+ - wan
7
+ - wan2.1
8
+ - echo-memory
9
+ - text-to-video
10
+ base_model: Wan-AI/Wan2.1-T2V-1.3B
11
+ ---
12
+
13
+ # Echo-Memory `context_k1` — complete Wan 2.1 1.3B DiT
14
+
15
+ DiffSynth-facing **full DiT** for the released Echo-Memory `context_k1` row.
16
+
17
+ - Paper: [arXiv:2606.09803](https://arxiv.org/abs/2606.09803)
18
+ - Code: [Echo-Team-Joy-Future-Academy-JD/Echo-Memory](https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory)
19
+ - Research overlay (HF): [Echo-Team/Echo-Memory](https://huggingface.co/Echo-Team/Echo-Memory) `context_k1/epoch-0.safetensors`
20
+
21
+ ## What this file is
22
+
23
+ `diffusion_pytorch_model.safetensors` is a **complete Wan 2.1 T2V 1.3B DiT** (825 / 825 official keys).
24
+
25
+ It is official [Wan-AI/Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B) DiT keys with the Echo-Memory `context_k1` overlay already merged. Research extras (`action_mlp`, `self_attn_with_action`, SSM / spatial slots) are **not** included, so the file loads like a normal Wan transformer (`strict=True` on `pipe.dit`).
26
+
27
+ T5 and VAE stay the official Wan files:
28
+
29
+ - `Wan-AI/Wan2.1-T2V-1.3B` `models_t5_umt5-xxl-enc-bf16.pth`
30
+ - `Wan-AI/Wan2.1-T2V-1.3B` `Wan2.1_VAE.pth`
31
+
32
+ ## DiffSynth
33
+
34
+ ```python
35
+ import torch
36
+ from diffsynth.pipelines.wan_video import WanVideoPipeline, ModelConfig
37
+ from diffsynth.utils.data import save_video
38
+
39
+ pipe = WanVideoPipeline.from_pretrained(
40
+ torch_dtype=torch.bfloat16,
41
+ device="cuda",
42
+ model_configs=[
43
+ ModelConfig(model_id="Wayne-King/echo-memory-context-k1-full", origin_file_pattern="diffusion_pytorch_model.safetensors"),
44
+ ModelConfig(model_id="Wan-AI/Wan2.1-T2V-1.3B", origin_file_pattern="models_t5_umt5-xxl-enc-bf16.pth"),
45
+ ModelConfig(model_id="Wan-AI/Wan2.1-T2V-1.3B", origin_file_pattern="Wan2.1_VAE.pth"),
46
+ ],
47
+ )
48
+ video = pipe(prompt="A toy bear on a table, the camera rotates around it", seed=42)
49
+ save_video(video, "echo_memory_context_k1.mp4")
50
+ ```
51
+
52
+ After this repo is mirrored on ModelScope, replace the DiT `model_id` with that ModelScope id (DiffSynth downloads from ModelScope by default).
53
+
54
+ ## Notes
55
+
56
+ - 30,000-step `epoch-0` fine-tune, 640×352, 81-frame chunks.
57
+ - Camera-action / multi-chunk revisit protocol stays in the Echo-Memory repo; this file is the Wan DiT overlay only.
58
+ - Apache-2.0. Please cite the Echo-Memory paper if you use it.