Files changed (1) hide show
  1. README.md +64 -0
README.md CHANGED
@@ -4,11 +4,55 @@ emoji: 🎬
4
  colorFrom: red
5
  colorTo: purple
6
  sdk: gradio
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  sdk_version: 6.17.3
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
  short_description: Fast Image Edit to Video (IE2V) pipeline.
 
 
 
 
 
12
  ---
13
 
14
  # 🎬 IE2V Fast Studio: Edit-Then-Animate Pipeline
@@ -57,4 +101,24 @@ Wan-AI Team for the foundational Wan 2.2 model weights.
57
 
58
  FireRedTeam for the state-of-the-art Qwen-based editing backbone.
59
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
  cbensimon for the pre-compiled WanTransformer3D AoT modules.
 
4
  colorFrom: red
5
  colorTo: purple
6
  sdk: gradio
7
+ sdk_version: 6.18.0
8
+ app_file: app.py
9
+ pinned: false
10
+ license: apache-2.0
11
+ short_description: Fast Image Edit to Video (IE2V) pipeline.
12
+ ---
13
+
14
+ # 🎬 IE2V Fast Studio: Edit-Then-Animate Pipeline
15
+
16
+ Welcome to the **IE2V (Image Edit to Video) Fast Studio**. This space bridges the gap between static image manipulation and next-gen video synthesis by combining **FireRed-Image-Edit-1.1** and **Wan 2.2 14B (FP8 Dynamic Quantized & AoT Compiled)** into a unified, high-performance creative sequence.
17
+
18
+ Modify specific regions or styles of an image with precise natural language instructions, then immediately generate seamless, fluid cinematic motion from the modified canvas.
19
+
20
+ ---
21
+
22
+ ## 🚀 Pipeline Core Architecture
23
+
24
+ ### 🛠️ Phase 1: Local Image Canvas Editing (FireRed 1.1)
25
+ - **Granular Control:** Powered by Qwen-Image-Edit-Rapid, enabling complex workflows like object replacement, style changes, or clothing modification (*e.g., "Convert it to a dotted cartoon style"*).
26
+ - **Flash Attention 3 (FA3):** Specialized hardware acceleration for instantaneous multi-reference canvas modifications.
27
+
28
+ ### 🎥 Phase 2: Ultra-Fast Motion Generation (Wan 2.2 14B)
29
+ - **Lightning LoRA Integration:** Generates striking, stable video outputs in just **4 to 8 inference steps** instead of the traditional 30+.
30
+ - **FP8 Quantization & Graph Compilation:** Uses `torchao` and `spaces.aoti_load` to minimize VRAM latency, maximizing safety boundaries within ZeroGPU infrastructure.
31
+
32
+ ---
33
+
34
+ ## 🛠️ Infrastructure & Resource Management
35
+
36
+ To avoid memory fragmentation and `RuntimeError: CUDA out of memory` when switching between models, the application strictly isolates the execution contexts.
37
+
38
+ ### VRAM Recycler Pattern (`app.py`)
39
+ Memory is forcefully claimed and flushed back to the OS before and after each inference step:
40
+ ---
41
+ title: IE2V Fast Studio (FireRed 1.1 + Wan 2.2 14B)
42
+ emoji: 🎬
43
+ colorFrom: red
44
+ colorTo: purple
45
+ sdk: gradio
46
  sdk_version: 6.17.3
47
  app_file: app.py
48
  pinned: false
49
  license: apache-2.0
50
  short_description: Fast Image Edit to Video (IE2V) pipeline.
51
+ tags:
52
+ - track:backyard
53
+ - achievement:offgrid
54
+ - achievement:offbrand
55
+ - achievement:fieldnotes
56
  ---
57
 
58
  # 🎬 IE2V Fast Studio: Edit-Then-Animate Pipeline
 
101
 
102
  FireRedTeam for the state-of-the-art Qwen-based editing backbone.
103
 
104
+ cbensimon for the pre-compiled WanTransformer3D AoT modules.
105
+ ```python
106
+ def clear_vram():
107
+ import gc
108
+ import torch
109
+ gc.collect()
110
+ torch.cuda.empty_cache()
111
+
112
+ Installation Blueprint
113
+ Ensure your runtime environment satisfies the compiled requirements:
114
+
115
+ pre-requirements.txt: Enforces pip>=23.0.0 for safe wheel building.
116
+
117
+ packages.txt: Must include system-level ffmpeg for video multiplexing.
118
+
119
+ 🤝 Credits & Acknowledgments
120
+ Wan-AI Team for the foundational Wan 2.2 model weights.
121
+
122
+ FireRedTeam for the state-of-the-art Qwen-based editing backbone.
123
+
124
  cbensimon for the pre-compiled WanTransformer3D AoT modules.