docs: split Memory-efficient Inference and GGUF+SageAttention into sub READMEs

README.md

@@ -299,90 +299,31 @@ See `inference.py --help` for all available options.
 
 ### 🔋 Memory-efficient Inference
 
-By default, `pipe.to("cuda")` loads all components onto the GPU simultaneously, requiring **~30 GB VRAM**.
+For GPUs with 24 GB or less (e.g. RTX 4090, RTX 3090), CPU offloading and FP8 quantization can reduce peak VRAM from ~30 GB to ~15 GB with minimal speed impact.
 
-For GPUs with 24 GB or less (e.g. RTX 4090, RTX 3090), use `enable_model_cpu_offload()` with the `expandable_segments` allocator setting:
+| Mode | Peak VRAM | Recommended GPU |
+|------|-----------|-----------------|
+| `pipe.to("cuda")` | ~30 GB | A100, H100, H200 |
+| `enable_model_cpu_offload()` | ~19 GB | RTX 4090, RTX 3090 |
+| `+ FP8 quantization` | ~15 GB | RTX 4090, RTX 3090 |
 
-```bash
-export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
-```
+> **Full guide** → [docs/memory-efficient-inference.md](docs/memory-efficient-inference.md)
 
-```python
-pipe = MotifVideoPipeline.from_pretrained(
-    "Motif-Technologies/Motif-Video-2B",
-    revision="diffusers-integration",
-    torch_dtype=torch.bfloat16,
-    guider=guider,  # see T2V example above
-)
-pipe.scheduler = FlowDPMSolver(
-    num_train_timesteps=pipe.scheduler.config.get("num_train_timesteps", 1000),
-    algorithm_type="dpmsolver++",
-    solver_order=2,
-    prediction_type="flow_prediction",
-    use_flow_sigmas=True,
-    flow_shift=15.0,
-)
-pipe.enable_model_cpu_offload()  # replaces pipe.to("cuda")
-
-output = pipe(
-    prompt="...",
-    negative_prompt="...",
-    height=736, width=1280, num_frames=121, num_inference_steps=50,
-    frame_rate=24, use_linear_quadratic_schedule=False,
-)
-export_to_video(output.frames[0], "output.mp4", fps=24)
-```
-
-This moves each component (text encoder → transformer → VAE) to GPU only when needed. The `expandable_segments` setting allows the CUDA memory allocator to efficiently reuse memory released by earlier components, avoiding fragmentation-related OOM errors.
-
-| Mode | Peak VRAM | Speed | Recommended GPU |
-|------|-----------|-------|-----------------|
-| `pipe.to("cuda")` | ~30 GB | Fastest | A100, H100, H200 |
-| `enable_model_cpu_offload()` | ~19 GB | Similar | RTX 4090, RTX 3090 |
-
-#### FP8 Weight Quantization (Optional)
-
-For further VRAM reduction, you can quantize the transformer weights to FP8 using [torchao](https://github.com/pytorch/ao):
-
-```bash
-pip install torchao
-```
+---
 
-```python
-from torchao.quantization import quantize_, Float8WeightOnlyConfig
+### 🧊 GGUF + SageAttention
 
-pipe = MotifVideoPipeline.from_pretrained(
-    "Motif-Technologies/Motif-Video-2B",
-    revision="diffusers-integration",
-    torch_dtype=torch.bfloat16,
-    guider=guider,  # see T2V example above
-)
-pipe.scheduler = FlowDPMSolver(
-    num_train_timesteps=pipe.scheduler.config.get("num_train_timesteps", 1000),
-    algorithm_type="dpmsolver++",
-    solver_order=2,
-    prediction_type="flow_prediction",
-    use_flow_sigmas=True,
-    flow_shift=15.0,
-)
-quantize_(pipe.transformer, Float8WeightOnlyConfig())
-pipe.enable_model_cpu_offload()
+GGUF quantized weights at [Motif-Video-2B-GGUF](https://huggingface.co/Motif-Technologies/Motif-Video-2B-GGUF) — up to 2.7 GB VRAM savings with no speed penalty. Combined with [SageAttention](https://github.com/thu-ml/SageAttention) for ~1.6× faster inference.
 
-output = pipe(
-    prompt="...",
-    negative_prompt="...",
-    height=736, width=1280, num_frames=121, num_inference_steps=50,
-    frame_rate=24, use_linear_quadratic_schedule=False,
-)
-export_to_video(output.frames[0], "output.mp4", fps=24)
-```
+| Variant | Sage (s/it) | Speedup | Peak alloc (GB) |
+|---------|------------|---------|-----------------|
+| BF16    | 14.75      | 1.58x   | 15.12           |
+| Q8_0    | 14.49      | 1.60x   | 13.44           |
+| Q4_K_M  | 14.59      | 1.60x   | 12.53           |
 
-This stores the transformer weights in FP8 (8-bit) instead of BF16 (16-bit), reducing peak VRAM from ~19 GB to ~15 GB while keeping all computation in BF16 precision.
+> **Full guide** → [docs/gguf-sageattention.md](docs/gguf-sageattention.md)
 
-| Mode | Peak VRAM | Notes |
-|------|-----------|-------|
-| `enable_model_cpu_offload()` | ~19 GB | BF16 baseline |
-| `+ Float8WeightOnlyConfig` | ~15 GB | FP8 weights, BF16 compute |
+---
 
 ### 🖥️ ComfyUI
 
@@ -390,140 +331,6 @@ Official ComfyUI custom nodes: [ComfyUI-MotifVideo2B](https://github.com/MotifTe
 
 > **Note:** Currently requires **High VRAM** mode. GGUF quantized model loading in ComfyUI is in progress.
 
----
-
-### 🧊 GGUF + SageAttention
-
-GGUF quantized transformer weights are available at [Motif-Video-2B-GGUF](https://huggingface.co/Motif-Technologies/Motif-Video-2B-GGUF), reducing VRAM with minimal quality loss. Combined with [SageAttention](https://github.com/thu-ml/SageAttention) for ~2× faster attention computation.
-
-#### GGUF Inference
-
-```bash
-pip install gguf
-```
-
-```python
-import torch
-from diffusers import (
-    AdaptiveProjectedGuidance,
-    DPMSolverMultistepScheduler,
-    GGUFQuantizationConfig,
-    MotifVideoPipeline,
-    MotifVideoTransformer3DModel,
-)
-from diffusers.utils import export_to_video
-from huggingface_hub import hf_hub_download
-
-guider = AdaptiveProjectedGuidance(
-    guidance_scale=8.0,
-    adaptive_projected_guidance_rescale=12.0,
-    adaptive_projected_guidance_momentum=0.1,
-    use_original_formulation=True,
-    normalization_dims="spatial",
-)
-
-variant = "Q4_K_M"  # Options: Q4_0, Q4_1, Q4_K_M, Q5_0, Q5_1, Q5_K_M, Q6_K, Q8_0, BF16
-ckpt_path = hf_hub_download(
-    "Motif-Technologies/Motif-Video-2B-GGUF",
-    filename=f"motifv-2b-dev-{variant}.gguf",
-)
-
-transformer = MotifVideoTransformer3DModel.from_single_file(
-    ckpt_path,
-    quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
-    config="Motif-Technologies/Motif-Video-2B",
-    revision="diffusers-integration",
-    subfolder="transformer",
-    torch_dtype=torch.bfloat16,
-)
-
-pipe = MotifVideoPipeline.from_pretrained(
-    "Motif-Technologies/Motif-Video-2B",
-    revision="diffusers-integration",
-    torch_dtype=torch.bfloat16,
-    guider=guider,
-    transformer=transformer,
-)
-
-pipe.scheduler = FlowDPMSolver(
-    num_train_timesteps=pipe.scheduler.config.get("num_train_timesteps", 1000),
-    algorithm_type="dpmsolver++",
-    solver_order=2,
-    prediction_type="flow_prediction",
-    use_flow_sigmas=True,
-    flow_shift=15.0,
-)
-pipe.enable_model_cpu_offload()
-
-output = pipe(
-    prompt="A woman standing in a sunlit field as flower petals swirl around her in slow motion. Each petal floats gently through the golden light, casting tiny shadows. Her hair moves like water, and time seems to stand still.",
-    negative_prompt="text overlay, graphic overlay, watermark, logo, subtitles, timestamp, broadcast graphics, UI elements, random letters, frozen pose, rigid, static expression, jerky motion, mechanical motion, discontinuous motion, flat framing, depthless, dull lighting, monotone, crushed shadows, blown-out highlights, shifting background, fading background, poor continuity, identity drift, deformation, flickering, ghosting, smearing, duplication, mutated proportions, inconsistent clothing, flat colors, desaturated, tonally compressed, poor background separation, exposure shift, uneven brightness, color balance shift",
-    height=736,
-    width=1280,
-    num_frames=121,
-    num_inference_steps=50,
-    frame_rate=24,
-    use_linear_quadratic_schedule=False,
-)
-export_to_video(output.frames[0], "output.mp4", fps=24)
-```
-
-#### SageAttention (Optional, ~1.6× faster)
-
-Same prompt and seed, 1280x736, 121 frames, 50 steps. Left = SDPA, Right = SageAttention.
-
-![BF16](assets/sage_compare_BF16.webp)
-![Q8_0](assets/sage_compare_Q8_0.webp)
-![Q5_K_M](assets/sage_compare_Q5_K_M.webp)
-![Q4_K_M](assets/sage_compare_Q4_K_M.webp)
-
-[SageAttention](https://github.com/thu-ml/SageAttention) accelerates attention by quantizing Q/K to INT8 and V to FP8, reducing memory bandwidth. Works with all GGUF variants.
-
-**Install** (build from source — PyPI only has 1.x, need 2.x):
-
-```bash
-# Set TORCH_CUDA_ARCH_LIST to match your GPU: "8.0" for A100, "9.0" for H100/H200
-TORCH_CUDA_ARCH_LIST="9.0" pip install git+https://github.com/thu-ml/SageAttention.git --no-build-isolation
-```
-
-**Usage with `inference.py`:**
-
-```bash
-python inference.py --use-sage-attention --prompt "..."
-```
-
-**Notes:**
-- Requires NVIDIA GPU with SM70+
-- SM90+ (H100, H200) — FP8 kernels for maximum speedup
-- SM80-SM89 (A100, RTX 3090, RTX 4090) — FP16 kernels (still faster than SDPA)
-- SM70-SM75 (V100, RTX 2080 Ti) — FP16 kernels
-- Set `TORCH_CUDA_ARCH_LIST` to match your GPU when building (e.g., `"8.6"` for RTX 3090, `"8.9"` for RTX 4090)
-- No quality degradation observed across all GGUF variants
-
-#### Benchmark
-
-Measured on NVIDIA H200, 1280x736, 121 frames, 50 steps, DPMSolver++ (order=2, flow_shift=15.0):
-
-| Variant | SDPA (s/it) | Sage (s/it) | Speedup | Peak alloc (GB) | Peak rsv (GB) | Total SDPA (s) | Total Sage (s) |
-|---------|------------|------------|---------|-----------------|----------------|----------------|----------------|
-| BF16    | 23.36      | 14.75      | 1.58x   | 14.78 / 15.12   | 24.93 / 24.90  | 1184           | 754            |
-| Q8_0    | 23.16      | 14.49      | 1.60x   | 13.10 / 13.44   | 23.14 / 23.11  | 1178           | 744            |
-| Q6_K    | 23.21      | 14.55      | 1.60x   | 12.62 / 12.95   | 22.72 / 22.69  | 1178           | 747            |
-| Q5_K_M  | 23.33      | 14.69      | 1.59x   | 12.39 / 12.72   | 22.45 / 22.42  | 1184           | 754            |
-| Q5_1    | 23.54      | 14.96      | 1.57x   | 12.47 / 12.81   | 22.66 / 22.62  | 1193           | 764            |
-| Q5_0    | 23.26      | 14.67      | 1.59x   | 12.37 / 12.71   | 22.55 / 22.52  | 1179           | 750            |
-| Q4_K_M  | 23.25      | 14.59      | 1.60x   | 12.19 / 12.53   | 22.22 / 22.18  | 1178           | 747            |
-| Q4_1    | 23.31      | 14.68      | 1.59x   | 12.26 / 12.60   | 22.26 / 22.22  | 1181           | 750            |
-| Q4_0    | 23.33      | 14.75      | 1.58x   | 12.14 / 12.47   | 22.18 / 22.14  | 1188           | 760            |
-
-Peak alloc/rsv columns show SDPA / Sage values. Sage adds ~0.3 GB alloc overhead (INT8/FP8 quantization buffers) with no change in reserved memory.
-
-**Key findings:**
-- **~1.59x faster with SageAttention** — consistent across all quantization levels
-- **VRAM unchanged** — sage overhead is negligible (~0.3 GB alloc)
-- **GGUF + Sage stacks** — Q4_K_M + Sage achieves 14.59 s/it at 12.53 GB alloc (vs BF16 SDPA: 23.36 s/it at 14.78 GB)
-
-
 ---
 
 ## 📊 Performance

docs/gguf-sageattention.md

@@ -0,0 +1,132 @@
+# 🧊 GGUF + SageAttention
+
+> See the main [README](../README.md) for `FlowDPMSolver` and pipeline setup.
+
+GGUF quantized transformer weights are available at [Motif-Video-2B-GGUF](https://huggingface.co/Motif-Technologies/Motif-Video-2B-GGUF), reducing VRAM with minimal quality loss. Combined with [SageAttention](https://github.com/thu-ml/SageAttention) for ~2× faster attention computation.
+
+## GGUF Inference
+
+```bash
+pip install gguf
+```
+
+```python
+import torch
+from diffusers import (
+    AdaptiveProjectedGuidance,
+    DPMSolverMultistepScheduler,
+    GGUFQuantizationConfig,
+    MotifVideoPipeline,
+    MotifVideoTransformer3DModel,
+)
+from diffusers.utils import export_to_video
+from huggingface_hub import hf_hub_download
+
+guider = AdaptiveProjectedGuidance(
+    guidance_scale=8.0,
+    adaptive_projected_guidance_rescale=12.0,
+    adaptive_projected_guidance_momentum=0.1,
+    use_original_formulation=True,
+    normalization_dims="spatial",
+)
+
+variant = "Q4_K_M"  # Options: Q4_0, Q4_1, Q4_K_M, Q5_0, Q5_1, Q5_K_M, Q6_K, Q8_0, BF16
+ckpt_path = hf_hub_download(
+    "Motif-Technologies/Motif-Video-2B-GGUF",
+    filename=f"motifv-2b-dev-{variant}.gguf",
+)
+
+transformer = MotifVideoTransformer3DModel.from_single_file(
+    ckpt_path,
+    quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
+    config="Motif-Technologies/Motif-Video-2B",
+    revision="diffusers-integration",
+    subfolder="transformer",
+    torch_dtype=torch.bfloat16,
+)
+
+pipe = MotifVideoPipeline.from_pretrained(
+    "Motif-Technologies/Motif-Video-2B",
+    revision="diffusers-integration",
+    torch_dtype=torch.bfloat16,
+    guider=guider,
+    transformer=transformer,
+)
+
+pipe.scheduler = FlowDPMSolver(
+    num_train_timesteps=pipe.scheduler.config.get("num_train_timesteps", 1000),
+    algorithm_type="dpmsolver++",
+    solver_order=2,
+    prediction_type="flow_prediction",
+    use_flow_sigmas=True,
+    flow_shift=15.0,
+)
+pipe.enable_model_cpu_offload()
+
+output = pipe(
+    prompt="A woman standing in a sunlit field as flower petals swirl around her in slow motion. Each petal floats gently through the golden light, casting tiny shadows. Her hair moves like water, and time seems to stand still.",
+    negative_prompt="text overlay, graphic overlay, watermark, logo, subtitles, timestamp, broadcast graphics, UI elements, random letters, frozen pose, rigid, static expression, jerky motion, mechanical motion, discontinuous motion, flat framing, depthless, dull lighting, monotone, crushed shadows, blown-out highlights, shifting background, fading background, poor continuity, identity drift, deformation, flickering, ghosting, smearing, duplication, mutated proportions, inconsistent clothing, flat colors, desaturated, tonally compressed, poor background separation, exposure shift, uneven brightness, color balance shift",
+    height=736,
+    width=1280,
+    num_frames=121,
+    num_inference_steps=50,
+    frame_rate=24,
+    use_linear_quadratic_schedule=False,
+)
+export_to_video(output.frames[0], "output.mp4", fps=24)
+```
+
+## SageAttention (Optional, ~1.6× faster)
+
+Same prompt and seed, 1280x736, 121 frames, 50 steps. Left = SDPA, Right = SageAttention.
+
+![BF16](../assets/sage_compare_BF16.webp)
+![Q8_0](../assets/sage_compare_Q8_0.webp)
+![Q5_K_M](../assets/sage_compare_Q5_K_M.webp)
+![Q4_K_M](../assets/sage_compare_Q4_K_M.webp)
+
+[SageAttention](https://github.com/thu-ml/SageAttention) accelerates attention by quantizing Q/K to INT8 and V to FP8, reducing memory bandwidth. Works with all GGUF variants.
+
+**Install** (build from source — PyPI only has 1.x, need 2.x):
+
+```bash
+# Set TORCH_CUDA_ARCH_LIST to match your GPU: "8.0" for A100, "9.0" for H100/H200
+TORCH_CUDA_ARCH_LIST="9.0" pip install git+https://github.com/thu-ml/SageAttention.git --no-build-isolation
+```
+
+**Usage with `inference.py`:**
+
+```bash
+python inference.py --use-sage-attention --prompt "..."
+```
+
+**Notes:**
+- Requires NVIDIA GPU with SM70+
+- SM90+ (H100, H200) — FP8 kernels for maximum speedup
+- SM80-SM89 (A100, RTX 3090, RTX 4090) — FP16 kernels (still faster than SDPA)
+- SM70-SM75 (V100, RTX 2080 Ti) — FP16 kernels
+- Set `TORCH_CUDA_ARCH_LIST` to match your GPU when building (e.g., `"8.6"` for RTX 3090, `"8.9"` for RTX 4090)
+- No quality degradation observed across all GGUF variants
+
+## Benchmark
+
+Measured on NVIDIA H200, 1280x736, 121 frames, 50 steps, DPMSolver++ (order=2, flow_shift=15.0):
+
+| Variant | SDPA (s/it) | Sage (s/it) | Speedup | Peak alloc (GB) | Peak rsv (GB) | Total SDPA (s) | Total Sage (s) |
+|---------|------------|------------|---------|-----------------|----------------|----------------|----------------|
+| BF16    | 23.36      | 14.75      | 1.58x   | 14.78 / 15.12   | 24.93 / 24.90  | 1184           | 754            |
+| Q8_0    | 23.16      | 14.49      | 1.60x   | 13.10 / 13.44   | 23.14 / 23.11  | 1178           | 744            |
+| Q6_K    | 23.21      | 14.55      | 1.60x   | 12.62 / 12.95   | 22.72 / 22.69  | 1178           | 747            |
+| Q5_K_M  | 23.33      | 14.69      | 1.59x   | 12.39 / 12.72   | 22.45 / 22.42  | 1184           | 754            |
+| Q5_1    | 23.54      | 14.96      | 1.57x   | 12.47 / 12.81   | 22.66 / 22.62  | 1193           | 764            |
+| Q5_0    | 23.26      | 14.67      | 1.59x   | 12.37 / 12.71   | 22.55 / 22.52  | 1179           | 750            |
+| Q4_K_M  | 23.25      | 14.59      | 1.60x   | 12.19 / 12.53   | 22.22 / 22.18  | 1178           | 747            |
+| Q4_1    | 23.31      | 14.68      | 1.59x   | 12.26 / 12.60   | 22.26 / 22.22  | 1181           | 750            |
+| Q4_0    | 23.33      | 14.75      | 1.58x   | 12.14 / 12.47   | 22.18 / 22.14  | 1188           | 760            |
+
+Peak alloc/rsv columns show SDPA / Sage values. Sage adds ~0.3 GB alloc overhead (INT8/FP8 quantization buffers) with no change in reserved memory.
+
+**Key findings:**
+- **~1.59x faster with SageAttention** — consistent across all quantization levels
+- **VRAM unchanged** — sage overhead is negligible (~0.3 GB alloc)
+- **GGUF + Sage stacks** — Q4_K_M + Sage achieves 14.59 s/it at 12.53 GB alloc (vs BF16 SDPA: 23.36 s/it at 14.78 GB)

docs/memory-efficient-inference.md

@@ -0,0 +1,88 @@
+# Memory-efficient Inference
+
+> See the main [README](../README.md) for `FlowDPMSolver` and `guider` setup.
+
+By default, `pipe.to("cuda")` loads all components onto the GPU simultaneously, requiring **~30 GB VRAM**.
+
+For GPUs with 24 GB or less (e.g. RTX 4090, RTX 3090), use `enable_model_cpu_offload()` with the `expandable_segments` allocator setting:
+
+```bash
+export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
+```
+
+```python
+pipe = MotifVideoPipeline.from_pretrained(
+    "Motif-Technologies/Motif-Video-2B",
+    revision="diffusers-integration",
+    torch_dtype=torch.bfloat16,
+    guider=guider,  # see T2V example above
+)
+pipe.scheduler = FlowDPMSolver(
+    num_train_timesteps=pipe.scheduler.config.get("num_train_timesteps", 1000),
+    algorithm_type="dpmsolver++",
+    solver_order=2,
+    prediction_type="flow_prediction",
+    use_flow_sigmas=True,
+    flow_shift=15.0,
+)
+pipe.enable_model_cpu_offload()  # replaces pipe.to("cuda")
+
+output = pipe(
+    prompt="...",
+    negative_prompt="...",
+    height=736, width=1280, num_frames=121, num_inference_steps=50,
+    frame_rate=24, use_linear_quadratic_schedule=False,
+)
+export_to_video(output.frames[0], "output.mp4", fps=24)
+```
+
+This moves each component (text encoder → transformer → VAE) to GPU only when needed. The `expandable_segments` setting allows the CUDA memory allocator to efficiently reuse memory released by earlier components, avoiding fragmentation-related OOM errors.
+
+| Mode | Peak VRAM | Speed | Recommended GPU |
+|------|-----------|-------|-----------------|
+| `pipe.to("cuda")` | ~30 GB | Fastest | A100, H100, H200 |
+| `enable_model_cpu_offload()` | ~19 GB | Similar | RTX 4090, RTX 3090 |
+
+## FP8 Weight Quantization (Optional)
+
+For further VRAM reduction, you can quantize the transformer weights to FP8 using [torchao](https://github.com/pytorch/ao):
+
+```bash
+pip install torchao
+```
+
+```python
+from torchao.quantization import quantize_, Float8WeightOnlyConfig
+
+pipe = MotifVideoPipeline.from_pretrained(
+    "Motif-Technologies/Motif-Video-2B",
+    revision="diffusers-integration",
+    torch_dtype=torch.bfloat16,
+    guider=guider,  # see T2V example above
+)
+pipe.scheduler = FlowDPMSolver(
+    num_train_timesteps=pipe.scheduler.config.get("num_train_timesteps", 1000),
+    algorithm_type="dpmsolver++",
+    solver_order=2,
+    prediction_type="flow_prediction",
+    use_flow_sigmas=True,
+    flow_shift=15.0,
+)
+quantize_(pipe.transformer, Float8WeightOnlyConfig())
+pipe.enable_model_cpu_offload()
+
+output = pipe(
+    prompt="...",
+    negative_prompt="...",
+    height=736, width=1280, num_frames=121, num_inference_steps=50,
+    frame_rate=24, use_linear_quadratic_schedule=False,
+)
+export_to_video(output.frames[0], "output.mp4", fps=24)
+```
+
+This stores the transformer weights in FP8 (8-bit) instead of BF16 (16-bit), reducing peak VRAM from ~19 GB to ~15 GB while keeping all computation in BF16 precision.
+
+| Mode | Peak VRAM | Notes |
+|------|-----------|-------|
+| `enable_model_cpu_offload()` | ~19 GB | BF16 baseline |
+| `+ Float8WeightOnlyConfig` | ~15 GB | FP8 weights, BF16 compute |