Spaces:
Running on Zero
docs: fold in ZeroGPU recon; add pinned requirements
Browse filesPLAN.md rewritten against corrected recon. The substantive reversals:
- ZeroGPU is RTX PRO 6000 Blackwell via MIG, not an A10G and not an
H200 — large is 48 GB, xlarge is 96 GB, and the API's 'zero-a10g'
string is a stale identifier. VRAM was never the constraint.
- enable_model_cpu_offload() must NOT be used. It is counterproductive
on ZeroGPU: anything accelerate parks on CPU is never registered for
packing, so it stays in main-process RAM, is fork-inherited by every
worker, and is re-copied host to device on every hook fire. The
platform's own O_DIRECT disk packer replaces it. Section 8 rewritten
around import order, module-scope .to('cuda'), static @spaces.GPU,
and the never-return-CUDA-tensors rule.
- CPU RAM (~68 GB) and transient disk (~144 GB peak against a 150 GB
cap) are the real ceilings. bf16 ships as the default with a loud
preflight; quantization is a one-line fallback switch, not the
default, because ZeroGPU's packer handling of quantised tensor
subclasses is unverified. Recorded as technical risk #1.
- The classic pipelines decode with the conv VAE, not the diffusion
decoder, which is 2.5's native default — so a naive call silently
discards the headline quality feature. Stage graph now decodes
explicitly, with denormalize=False mandatory against double
denormalization.
- Multishot downgraded to honest: no API, and the enhancer system
prompts actively discourage scene cuts.
- The 11 UI modes are documented as a presentation layer over four
real workflows rather than eleven code paths.
Adds twelve API gotchas and seven operational lessons as explicit test
cases, resolves all six open questions, and adds a five-entry risk
register.
requirements.txt carries the full pin set: torch trio pinned together
against the torchvision 0.28 trap, diffusers at an exact SHA with a
startup version assertion, gradio 6 since 5.x is EOL, numpy 2. No
spaces line — the platform pins its own version and a conflicting pin
breaks build-time resolution.
Adds preflight and zerogpu module stubs, a dev extra for test tooling,
and a 20 GB output retention cap.
- README.md +1 -1
- docs/PLAN.md +669 -377
- ltx25_studio/preflight.py +17 -0
- ltx25_studio/zerogpu.py +35 -0
- pyproject.toml +11 -4
- requirements.txt +67 -0
|
@@ -7,4 +7,4 @@ video + audio generation model — covering every mode the model supports, with
|
|
| 7 |
Runs locally on Apple Silicon (MPS) / NVIDIA (CUDA) and deploys to a private
|
| 8 |
Hugging Face Space on ZeroGPU.
|
| 9 |
|
| 10 |
-
Status:
|
|
|
|
| 7 |
Runs locally on Apple Silicon (MPS) / NVIDIA (CUDA) and deploys to a private
|
| 8 |
Hugging Face Space on ZeroGPU.
|
| 9 |
|
| 10 |
+
Status: design locked, implementation starting. See [`docs/PLAN.md`](docs/PLAN.md).
|
|
@@ -4,19 +4,23 @@
|
|
| 4 |
**Branch:** `soma/ltx25-studio` · **Remote:** `git@github.com:techfreakworm/ltx2.5-studio.git`
|
| 5 |
**Target:** private Hugging Face Space `techfreakworm/LTX2.5-Studio` (gradio SDK, ZeroGPU).
|
| 6 |
|
| 7 |
-
This document is the contract for the build. Every section
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
|
| 12 |
**Prime directive:** this app must not repeat the failure modes of its predecessor
|
| 13 |
-
(`ltx2.3-AIO-generator`). Those
|
| 14 |
-
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
---
|
| 18 |
|
| 19 |
-
## 1. Decision 1 — Integration path: **Diffusers**
|
| 20 |
|
| 21 |
We build on `diffusers`, loading `Lightricks/LTX-2.5-Diffusers`.
|
| 22 |
|
|
@@ -30,102 +34,118 @@ We build on `diffusers`, loading `Lightricks/LTX-2.5-Diffusers`.
|
|
| 30 |
|
| 31 |
### 1.2 Why C
|
| 32 |
|
| 33 |
-
**
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
`pipeline_ltx2_latent_upsample.py`, plus `connectors.py`, `latent_upsampler.py`,
|
| 37 |
-
`vocoder.py`, `export_utils.py`, `utils.py`. Models `AutoencoderKLLTX2Video`,
|
| 38 |
-
`AutoencoderKLLTX2Audio`, `LTX2VideoTransformer3DModel` are exported from the top-level
|
| 39 |
-
`diffusers` namespace.
|
| 40 |
-
|
| 41 |
-
**Path A is a heavy, fragile install for a Space.** It requires `uv sync` of a git workspace,
|
| 42 |
-
CUDA >= 12.7 and torch ~= 2.7. A Space build that depends on resolving a git workspace at
|
| 43 |
-
image-build time is a build we cannot reproduce or pin.
|
| 44 |
|
| 45 |
**Path B is the documented cause of the fragility we are told to avoid.** The 2.3 project's
|
| 46 |
-
`requirements.txt` vendors an entire ComfyUI runtime —
|
| 47 |
-
|
| 48 |
-
`
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
`
|
| 52 |
-
|
| 53 |
-
|
| 54 |
|
| 55 |
**Decisive coupling to §2.** The `*-comfy-int8-convrot` quantised weights are **ComfyUI-only**
|
| 56 |
-
— the model card says so
|
| 57 |
-
|
| 58 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
-
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
-
|
| 63 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
`diffusers/loaders/single_file_utils.py` does contain an LTX-2 route:
|
| 66 |
`CHECKPOINT_KEY_NAMES["ltx2"]` → `model_type = "ltx2-dev"` → config repo
|
| 67 |
-
`{"pretrained_model_name_or_path": "Lightricks/LTX-2"}`.
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
- `convert_ltx2_vae_to_diffusers` (`LTX_2_0_VIDEO_VAE_RENAME_DICT`)
|
| 72 |
-
- `convert_ltx2_audio_vae_to_diffusers` (`LTX_2_0_AUDIO_VAE_RENAME_DICT`)
|
| 73 |
|
| 74 |
There is **no** single-file converter for 2.5's `connectors`, `diffusion_decoder`, `vocoder`,
|
| 75 |
-
`duration_head` or `latent_upsampler`. Even a successful
|
| 76 |
-
|
| 77 |
-
|
| 78 |
|
| 79 |
---
|
| 80 |
|
| 81 |
-
## 2. Decision 2 — Disk
|
| 82 |
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
|
| 87 |
### 2.1 Trap A — duplicate shard sets
|
| 88 |
|
| 89 |
-
Two folders
|
| 90 |
|
| 91 |
| folder | set 1 | set 2 | note |
|
| 92 |
|---|---|---|---|
|
| 93 |
| `transformer/` | 4 shards: 9.932 + 9.919 + 9.986 + 8.138 = **37.975 GB** | 8 shards: 4.991 + 4.944 + 4.994 + 4.928 + 4.977 + 4.995 + 4.994 + 3.163 = **37.986 GB** | only **one** `diffusion_pytorch_model.safetensors.index.json` |
|
| 94 |
| `connectors/` | single file: **6.344 GB** | 2 shards: 5.000 + 1.344 = **6.344 GB** | same |
|
| 95 |
|
| 96 |
-
A naive `snapshot_download` of those folders pulls **both** sets
|
| 97 |
-
|
| 98 |
-
the index JSON and fetches only the shards the index references.
|
| 99 |
-
|
| 100 |
-
**Two consequences that are binding on the build:**
|
| 101 |
|
| 102 |
-
1
|
| 103 |
-
2
|
| 104 |
-
|
| 105 |
-
72 GB in, which is the worst possible time to find out.
|
| 106 |
|
| 107 |
### 2.2 Trap B — `transformer_full/` is the DEV transformer
|
| 108 |
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
(§2.5), not deleted from our mental model.
|
| 112 |
|
| 113 |
-
### 2.3 The subset we ship
|
| 114 |
|
| 115 |
| component | GB | why |
|
| 116 |
|---|---:|---|
|
| 117 |
| `transformer/` (distilled, one shard set) | 37.975 | the model |
|
| 118 |
-
| `text_encoder/` (5 shards: 4.961 + 4.967 + 4.884 + 4.986 + 4.121) | 23.919 | Gemma-
|
| 119 |
| `connectors/` | 6.344 | text → transformer conditioning; also a LoRA target |
|
| 120 |
-
| `vae/` | 1.452 | video VAE |
|
| 121 |
| `audio_vae/` | 0.107 | audio VAE (mel ↔ latent) |
|
| 122 |
| `vocoder/` | 0.258 | mel → waveform |
|
| 123 |
| `latent_upsampler/` | 0.996 | stage-2 spatial upscale |
|
| 124 |
| `temporal_latent_upsampler/` | 0.262 | stage-2 temporal upscale |
|
| 125 |
-
| `diffusion_decoder/` | 0.834 |
|
| 126 |
-
| `duration_head/` | 0.004 | duration prediction
|
| 127 |
| `tokenizer/` | 0.032 | |
|
| 128 |
-
| **TOTAL** | **72.185** | **≈ 72.19 GB** |
|
| 129 |
|
| 130 |
### 2.4 What we exclude, and why
|
| 131 |
|
|
@@ -133,137 +153,197 @@ ship the distilled 8-step path by default, so we do not ship it. It is documente
|
|
| 133 |
|---|---:|---|
|
| 134 |
| `transformer_full/` | 37.975 | the DEV transformer; not needed for the distilled 8-step path |
|
| 135 |
| duplicate shard sets (transformer 8-shard + connectors 2-shard) | 44.330 | Trap A — same weights, different sharding |
|
| 136 |
-
| `ltx-2.5-22b-distilled-lora-450-bf16.safetensors` (repo root) | 9.699 | the distilled LoRA for **DEV** workflows; useless when we
|
| 137 |
|
| 138 |
-
**Arithmetic check:** 72.185 + 37.975 + 44.330 + 9.699 = **164.189 GB**
|
| 139 |
-
|
| 140 |
unexplained remainder.
|
| 141 |
|
| 142 |
-
### 2.5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
why it is an *opt-in* rather than an impossibility — but enabling it leaves ~40 GB, and outputs
|
| 147 |
-
must then be pruned aggressively.
|
| 148 |
|
| 149 |
-
|
| 150 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 151 |
|
| 152 |
---
|
| 153 |
|
| 154 |
-
## 3. Decision 3 — Mode coverage:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 155 |
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
|
|
|
|
|
|
|
|
|
| 159 |
|
| 160 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 161 |
|
| 162 |
**(a) One conditioning mechanism covers both image and video.**
|
| 163 |
-
`LTX2VideoCondition
|
| 164 |
`PIL.Image.Image | list[PIL.Image.Image] | np.ndarray | torch.Tensor` — "the image (**or
|
| 165 |
-
video**) to condition the video on"
|
| 166 |
-
conditioning
|
| 167 |
-
|
| 168 |
-
it that way: a list of conditions, each with a source, an index and a strength.
|
| 169 |
|
| 170 |
**(b) The model is jointly audio-visual. There is no video-off switch.**
|
| 171 |
-
The return type is `LTX2PipelineOutput(frames=..., audio=...)`
|
| 172 |
-
|
| 173 |
-
important cost fact in the app, and it is why the "audio" modes below carry a warning.
|
| 174 |
|
| 175 |
**(c) Audio input is not a first-class kwarg.**
|
| 176 |
-
There is no `audio=` argument. Audio
|
| 177 |
-
|
| 178 |
-
`
|
| 179 |
-
`
|
| 180 |
-
|
| 181 |
-
Audio frame count and video frame count are computed independently, so **they can desync**;
|
| 182 |
-
§5.5 makes validating that a hard requirement.
|
| 183 |
|
| 184 |
-
### 3.
|
| 185 |
|
| 186 |
-
| # | Mode | How it is reached | Cost note |
|
| 187 |
-
|---|---|---|---|
|
| 188 |
-
| 1 | text → video | `
|
| 189 |
-
| 2 | image → video |
|
| 190 |
-
| 3 | image + text → video | as #2 with a prompt | first-class |
|
| 191 |
-
| 4 | video → video |
|
| 192 |
-
| 5 | text → audio+video | #1 — the audio track is always produced |
|
| 193 |
-
| 6 | image → audio+video | #2 | first-class |
|
| 194 |
-
| 7 | image + text → audio+video | #3 | first-class |
|
| 195 |
-
| 8 | text → audio | #1, export only the audio track | ⚠️ **still pays full video compute** |
|
| 196 |
-
| 9 | video → audio | #4, export only the audio track | ⚠️ same |
|
| 197 |
-
| 10 | audio → audio | `audio_latents` + `noise_scale`, export only audio | ⚠️ same |
|
| 198 |
-
| 11 | audio → video | `audio_latents` + `noise_scale`, export only video | ⚠️ pays audio encode on top |
|
| 199 |
-
|
| 200 |
-
**
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
### 3.
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 227 |
|
| 228 |
---
|
| 229 |
|
| 230 |
## 4. Decision 4 — LoRA: the headline feature
|
| 231 |
|
| 232 |
-
User-uploadable LoRA at inference time is
|
| 233 |
-
|
| 234 |
|
| 235 |
### 4.1 Verified mechanics
|
| 236 |
|
| 237 |
-
- All
|
| 238 |
-
`LTX2ImageToVideoPipeline`) are declared as
|
| 239 |
`class LTX2…Pipeline(DiffusionPipeline, FromSingleFileMixin, LTX2LoraLoaderMixin)`.
|
| 240 |
-
- `LTX2LoraLoaderMixin
|
| 241 |
-
**`connectors`** — a LoRA may legitimately touch the text-conditioning path
|
| 242 |
-
transformer.
|
| 243 |
- **Both key conventions load.** `lora_state_dict()` detects
|
| 244 |
`is_non_diffusers_format = any(k.startswith("diffusion_model.") for k in state_dict)` and runs
|
| 245 |
`_convert_non_diffusers_ltx2_lora_to_diffusers(state_dict)`; separately it detects
|
| 246 |
`has_connector = any(k.startswith("text_embedding_projection.") for k in state_dict)` and
|
| 247 |
-
converts those into the connectors sub-dict.
|
| 248 |
-
both work.
|
| 249 |
- `lora_state_dict(..., return_lora_metadata=True)` returns the file's `__metadata__`.
|
| 250 |
|
| 251 |
-
### 4.2 The danger: validation is almost non-existent
|
| 252 |
|
| 253 |
-
The **only** built-in check is a substring test
|
| 254 |
-
`ValueError("Invalid LoRA checkpoint. Make sure all LoRA param names contain `'lora'` substring.")`
|
| 255 |
|
| 256 |
-
A LoRA
|
| 257 |
-
|
| 258 |
-
|
| 259 |
-
|
| 260 |
|
| 261 |
-
We
|
| 262 |
-
|
| 263 |
-
|
| 264 |
-
`adaln_single`, `audio_adaln_single`; and
|
| 265 |
`__metadata__ = {'lora_rank': '450', 'lora_alpha': '450', 'model_version': '2.5.0', 'license': …}`.
|
| 266 |
-
**`model_version` is machine-readable**
|
| 267 |
|
| 268 |
### 4.3 The validator (7 stages — each stage is a test unit)
|
| 269 |
|
|
@@ -271,94 +351,202 @@ Accepts either a **Hub repo-id** or a **direct file upload**.
|
|
| 271 |
|
| 272 |
| # | Stage | Rule | On failure |
|
| 273 |
|---|---|---|---|
|
| 274 |
-
| 1 | **Size bound** | reject above a configured max | refuse before any download
|
| 275 |
-
| 2 | **Header-only parse** | read the safetensors header, **never**
|
| 276 |
-
| 3 | **Format** |
|
| 277 |
-
| 4 | **Version** | read `__metadata__.model_version`
|
| 278 |
-
| 5 | **Convention** | classify
|
| 279 |
-
| 6 | **Pairing** |
|
| 280 |
-
| 7 | **Attach assertion** | **after** `load_lora_weights`, assert the adapter attached to **> 0** modules |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 281 |
|
| 282 |
-
|
| 283 |
-
just to find out it is the wrong architecture.
|
| 284 |
|
| 285 |
-
|
| 286 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 287 |
|
| 288 |
---
|
| 289 |
|
| 290 |
## 5. Decision 5 — Resilience doctrine
|
| 291 |
|
| 292 |
-
|
| 293 |
|
| 294 |
-
### 5.1 Pin every dependency, with upper bounds
|
| 295 |
|
| 296 |
-
Evidence from the predecessor
|
| 297 |
-
|
| 298 |
-
comment:
|
| 299 |
|
| 300 |
> *"HF Spaces ships torch==2.10.0; bare 'torchaudio' lets pip pick an older wheel whose C++ ABI
|
| 301 |
> doesn't match → import crashes the whole audio code path… Pin the matching version."*
|
| 302 |
|
| 303 |
-
That comment is a post-mortem
|
| 304 |
-
|
| 305 |
-
|
| 306 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 307 |
|
| 308 |
### 5.2 One central model-file indirection
|
| 309 |
|
| 310 |
-
Every Hub repo-id, revision, subfolder
|
| 311 |
-
`ltx25_studio/model_files.py` — as data, never inlined at a call site.
|
| 312 |
-
be a one-line change in one file.
|
| 313 |
|
| 314 |
-
A missing file must raise an error naming **both the file and its URL**.
|
| 315 |
-
|
| 316 |
-
|
| 317 |
-
go look at.
|
| 318 |
|
| 319 |
-
Pinning a **revision**
|
| 320 |
-
|
| 321 |
|
| 322 |
-
### 5.3 Never return an empty output
|
| 323 |
|
| 324 |
Any generation that completes without producing a file **must** raise a specific, actionable
|
| 325 |
error that the UI renders visibly.
|
| 326 |
|
| 327 |
-
The
|
| 328 |
-
and
|
| 329 |
-
|
| 330 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 331 |
|
| 332 |
-
### 5.4 Read
|
| 333 |
|
| 334 |
-
Mel/audio parameters come from `audio_vae.config` at runtime:
|
| 335 |
-
`
|
| 336 |
`audio_vae_temporal_compression_ratio`, `audio_latent_channels`.
|
| 337 |
|
| 338 |
-
This mirrors
|
| 339 |
-
`self.vae.
|
| 340 |
-
`self.audio_vae.
|
| 341 |
-
|
| 342 |
-
fallbacks when a component is absent. We follow the same discipline.
|
| 343 |
|
| 344 |
### 5.5 Enforce and surface the hard constraints
|
| 345 |
|
| 346 |
| constraint | source | handling |
|
| 347 |
|---|---|---|
|
| 348 |
-
| `height % 32 == 0`
|
| 349 |
-
| `num_frames % 8 == 1` | **not** hard-raised upstream
|
| 350 |
-
|
|
| 351 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 352 |
|
| 353 |
**Verified constants** (`diffusers/pipelines/ltx2/utils.py`):
|
| 354 |
|
| 355 |
-
```python
|
| 356 |
-
DISTILLED_SIGMA_VALUES = [1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875] # 8 steps, CFG=1
|
| 357 |
-
STAGE_2_DISTILLED_SIGMA_VALUES = [0.909375, 0.725, 0.421875] # 3 steps
|
| 358 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 359 |
|
| 360 |
-
|
| 361 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 362 |
|
| 363 |
---
|
| 364 |
|
|
@@ -367,19 +555,33 @@ schedule change upstream fails a test instead of silently changing output qualit
|
|
| 367 |
| | |
|
| 368 |
|---|---|
|
| 369 |
| **Space** | `techfreakworm/LTX2.5-Studio`, **private** |
|
| 370 |
-
| **SDK** | gradio |
|
| 371 |
-
| **
|
| 372 |
-
| **
|
| 373 |
-
| **
|
|
|
|
|
|
|
| 374 |
| **Git** | GitHub `git@github.com:techfreakworm/ltx2.5-studio.git`; Space and GitHub stay in sync |
|
| 375 |
|
| 376 |
-
Naming precedent
|
| 377 |
-
(private)
|
|
|
|
|
|
|
| 378 |
|
| 379 |
-
|
| 380 |
-
does not fail fast; it fails after tens of GB of download.
|
| 381 |
|
| 382 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 383 |
|
| 384 |
`Lightricks/LTX-2.5-Diffusers` is gated (`gated: auto`) and the operator's token has **not**
|
| 385 |
accepted that gate. Verified response:
|
|
@@ -391,227 +593,317 @@ click resolves it:
|
|
| 391 |
|
| 392 |
**https://huggingface.co/Lightricks/LTX-2.5-Diffusers**
|
| 393 |
|
| 394 |
-
Until
|
| 395 |
-
Escalated to the orchestrator.
|
| 396 |
|
| 397 |
---
|
| 398 |
|
| 399 |
## 7. Architecture
|
| 400 |
|
| 401 |
-
### 7.1 Module layout
|
| 402 |
|
| 403 |
```
|
| 404 |
-
app.py # entrypoint
|
| 405 |
ltx25_studio/
|
| 406 |
-
|
| 407 |
-
|
| 408 |
-
|
| 409 |
-
|
| 410 |
-
|
| 411 |
-
|
| 412 |
-
|
| 413 |
-
|
| 414 |
-
|
| 415 |
-
|
| 416 |
-
|
| 417 |
-
#
|
| 418 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 419 |
lora/
|
| 420 |
-
|
| 421 |
-
|
| 422 |
-
|
| 423 |
-
apply.py # load_lora_weights + the post-attach assertion (§4.3 stage 7)
|
| 424 |
ui/
|
| 425 |
-
__init__.py
|
| 426 |
blocks.py # Gradio Blocks assembly
|
| 427 |
components.py # reusable widgets (condition list, LoRA picker, constraint hints)
|
| 428 |
-
handlers.py # thin adapters: UI event ->
|
| 429 |
```
|
| 430 |
|
| 431 |
### 7.2 Layering rules
|
| 432 |
|
| 433 |
- **Dependencies point inward.** `ui/` may import anything; `generate.py` must not import `ui/`.
|
| 434 |
`model_files.py`, `errors.py` and `constraints.py` import nothing from the package.
|
| 435 |
-
- **`modes.py` is data, not branching.** Adding a mode
|
| 436 |
-
|
| 437 |
-
- **`ui/handlers.py` contains no business logic.**
|
| 438 |
-
|
| 439 |
-
|
| 440 |
-
|
| 441 |
-
exception into a string for a human. `generate.py` never returns an error-shaped success.
|
| 442 |
-
|
| 443 |
-
### 7.3 Rationale for the shape
|
| 444 |
-
|
| 445 |
-
The predecessor put backend, models, modes and workflow in four flat top-level modules with
|
| 446 |
-
`ui.py` importing all of them. That works until the failure modes get interesting: the empty-
|
| 447 |
-
output bug (§5.3) lived precisely in the seam between "workflow returned something" and "UI
|
| 448 |
-
rendered it", and there was no layer boundary at which to assert. Here, `generate.py` owns that
|
| 449 |
-
invariant and can be tested for it directly.
|
| 450 |
|
| 451 |
-
|
| 452 |
|
| 453 |
-
|
| 454 |
-
|
| 455 |
-
|
| 456 |
-
|
| 457 |
-
|
| 458 |
-
- **`enable_model_cpu_offload()` is mandatory**, not a tuning option. It is part of pipeline
|
| 459 |
-
construction in `pipelines.py`, not a user-facing toggle.
|
| 460 |
-
- **Peak GPU residency ≈ transformer (37.98 GB) + activations.** The transformer is the
|
| 461 |
-
irreducible resident component during denoising; everything else (text encoder, VAEs, vocoder,
|
| 462 |
-
upsamplers, decoder) can be paged in and out around it.
|
| 463 |
-
- **The text encoder is the second-largest component (23.9 GB) and is needed only once**, before
|
| 464 |
-
denoising. It should be evicted after prompt encoding rather than held.
|
| 465 |
-
- **Stage 2 (latent upsample) is a separate pipeline** — `LTX2LatentUpsamplePipeline` — which
|
| 466 |
-
means it is a separate residency window and can be skipped entirely for the audio-only modes
|
| 467 |
-
(§3.2 mitigation).
|
| 468 |
-
- **Exact ZeroGPU numbers pending** from the lead; the offload strategy above does not depend on
|
| 469 |
-
them, but the maximum supported resolution × frame count does.
|
| 470 |
-
|
| 471 |
-
**Local (Apple Silicon / MPS) is a different regime.** MPS has no `enable_model_cpu_offload()`
|
| 472 |
-
equivalent with the same semantics and unified memory changes the arithmetic entirely. Device
|
| 473 |
-
strategy is selected in `config.py` and the local path is expected to be capability-limited
|
| 474 |
-
rather than identical. Do not assume a code path that works on ZeroGPU works on MPS.
|
| 475 |
|
| 476 |
-
|
| 477 |
|
| 478 |
-
|
|
|
|
| 479 |
|
| 480 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 481 |
|
| 482 |
```
|
| 483 |
-
.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 484 |
```
|
| 485 |
|
| 486 |
-
**
|
| 487 |
-
|
| 488 |
-
|
| 489 |
|
| 490 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 491 |
|
| 492 |
-
|
| 493 |
-
|
|
|
|
| 494 |
|
| 495 |
-
|
| 496 |
-
|---|---|
|
| 497 |
-
| `constraints.py` | pure functions over ints — dimension snapping, frame-count validation, sigma schedule shape |
|
| 498 |
-
| `model_files.py` | data-only assertions: every entry has a repo, revision, subfolder, filename; no duplicates; the §2.3 subset sums to the stated total |
|
| 499 |
-
| `lora/inspect.py` | synthesised safetensors **headers** — small fixtures, no real LoRA needed (this is why stage 2 is header-only) |
|
| 500 |
-
| `lora/validate.py` | the 7 stages against crafted good/bad headers, including the wrong-architecture case |
|
| 501 |
-
| `modes.py` | registry completeness (all 11 present), and each mode's kwarg builder produces the expected kwargs for given inputs |
|
| 502 |
-
| `errors.py` | every error type produces a message containing the actionable detail (filename, URL, found-prefixes) |
|
| 503 |
-
| `outputs.py` retention | eviction logic over a temp dir |
|
| 504 |
-
| `generate.py` never-empty invariant | inject a stub pipeline that returns nothing → assert a specific exception, **not** an empty path |
|
| 505 |
|
| 506 |
-
##
|
| 507 |
|
| 508 |
-
|
| 509 |
-
|
| 510 |
-
run must be green with zero weights on disk.
|
| 511 |
|
| 512 |
-
###
|
|
|
|
|
|
|
| 513 |
|
| 514 |
-
|
|
|
|
|
|
|
|
|
|
| 515 |
|
| 516 |
-
|
| 517 |
-
|
| 518 |
-
|
| 519 |
-
3. **The `preload_from_hub` list contains only live shards** — assertable against the index JSON
|
| 520 |
-
without downloading anything (§2.1).
|
| 521 |
|
| 522 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 523 |
|
| 524 |
-
|
| 525 |
|
| 526 |
-
|
|
|
|
|
|
|
| 527 |
|
| 528 |
-
|
|
|
|
|
|
|
| 529 |
|
| 530 |
-
|
|
|
|
| 531 |
|
| 532 |
-
|
| 533 |
-
`LTX2Pipeline`, `LTX2ConditionPipeline`, `LTX2ImageToVideoPipeline`,
|
| 534 |
-
`LTX2LatentUpsamplePipeline`. Plus models `AutoencoderKLLTX2Video`,
|
| 535 |
-
`AutoencoderKLLTX2Audio`, `LTX2VideoTransformer3DModel`.
|
| 536 |
-
- **Not present at any version on disk:** `LTX2InContextPipeline`, `LTX2ReferenceCondition`,
|
| 537 |
-
`LTX2HDRPipeline`, `LTX2HDRReferenceCondition`, `LTX2VideoDiffusionDecodePipeline`,
|
| 538 |
-
`LTX2DurationHead`, `LTX2VocoderWithBWE`, `LTX2VideoHDRProcessor`.
|
| 539 |
-
- `LTX2ConditionPipeline.__init__` in 0.37.1 takes exactly:
|
| 540 |
-
`scheduler, vae, audio_vae, text_encoder, tokenizer, connectors, transformer, vocoder`.
|
| 541 |
-
There is **no `duration_head` parameter and no `prompt_enhancer` parameter**. Grepping the
|
| 542 |
-
whole `pipelines/ltx2/` tree for `duration_head`, `prompt_enhancer`, `min_seconds`,
|
| 543 |
-
`max_seconds`, `enable_prompt_enhancement` returns **zero hits**.
|
| 544 |
-
- `LTX2_5_IMAGE_CRF` also does not exist in 0.37.1 — only `DISTILLED_SIGMA_VALUES` and
|
| 545 |
-
`STAGE_2_DISTILLED_SIGMA_VALUES` are in `utils.py` (both matching §5.5 exactly).
|
| 546 |
|
| 547 |
-
|
| 548 |
-
|
| 549 |
-
|
| 550 |
-
*and* ship `duration_head` (§3.5), the diffusion decoder (§2.3, 0.834 GB) or `LTX2_5_IMAGE_CRF`
|
| 551 |
-
(§5.5).
|
| 552 |
|
| 553 |
-
**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 554 |
|
| 555 |
```
|
| 556 |
-
|
| 557 |
```
|
| 558 |
|
| 559 |
-
|
| 560 |
-
|
| 561 |
-
means we own upgrades manually — both acceptable. The alternative, `diffusers>=0.37,<0.38`, is a
|
| 562 |
-
smaller build but silently drops duration prediction, the diffusion decoder, HDR and in-context.
|
| 563 |
|
| 564 |
-
|
| 565 |
-
because it decides which components `model_files.py` lists and which modes `modes.py` can offer.
|
| 566 |
|
| 567 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 568 |
|
| 569 |
-
|
| 570 |
-
exist in the released package (see OQ-1). If OQ-1 resolves to a release pin, `diffusion_decoder`
|
| 571 |
-
and `duration_head` should come out of the §2.3 subset (−0.838 GB) and the plan should say so
|
| 572 |
-
rather than shipping weights nothing loads.
|
| 573 |
|
| 574 |
-
|
|
|
|
| 575 |
|
| 576 |
-
|
| 577 |
-
and **silently floors** a non-conforming value — there is no raise. Two implications:
|
| 578 |
|
| 579 |
-
|
| 580 |
-
|
| 581 |
-
|
| 582 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 583 |
|
| 584 |
-
|
| 585 |
|
| 586 |
-
|
| 587 |
-
`GemmaTokenizer | GemmaTokenizerFast`. The brief says "Gemma4-12B". Cosmetic for the disk budget
|
| 588 |
-
(the 23.92 GB figure is unaffected), but `model_files.py` and any user-facing label should not
|
| 589 |
-
claim a model family the loader does not use. Note this is *separate* from the prompt enhancer,
|
| 590 |
-
which genuinely is `google/gemma-4-E2B-it` (§3.4) — that one is a different model and is not in
|
| 591 |
-
the pack anyway.
|
| 592 |
|
| 593 |
-
|
|
|
|
| 594 |
|
| 595 |
-
|
| 596 |
-
|
| 597 |
-
|
| 598 |
-
|
| 599 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 600 |
|
| 601 |
-
###
|
| 602 |
|
| 603 |
-
|
| 604 |
-
|
| 605 |
-
retention with oldest-first eviction in `outputs.py`, configured in `config.py`. Needs a number.
|
| 606 |
|
| 607 |
---
|
| 608 |
|
| 609 |
## 11. Build order
|
| 610 |
|
| 611 |
1. ✅ **This document**, committed.
|
| 612 |
-
2. **Scaffold** — venv, `pyproject.toml`
|
| 613 |
-
|
| 614 |
-
|
| 615 |
-
|
| 616 |
-
|
| 617 |
-
|
|
|
|
| 4 |
**Branch:** `soma/ltx25-studio` · **Remote:** `git@github.com:techfreakworm/ltx2.5-studio.git`
|
| 5 |
**Target:** private Hugging Face Space `techfreakworm/LTX2.5-Studio` (gradio SDK, ZeroGPU).
|
| 6 |
|
| 7 |
+
This document is the contract for the build. Every section is either a **decision** (settled,
|
| 8 |
+
build to it) or a **risk** (§10, needs measurement not argument). Where a claim was verified
|
| 9 |
+
against real source or real repo metadata, the evidence is cited inline so a future reader can
|
| 10 |
+
re-check it instead of trusting it.
|
| 11 |
|
| 12 |
**Prime directive:** this app must not repeat the failure modes of its predecessor
|
| 13 |
+
(`ltx2.3-AIO-generator`). Those were not model failures. They were *packaging, pinning, and
|
| 14 |
+
silent-failure* failures. §5 and §8 are the sections the build will be judged on.
|
| 15 |
+
|
| 16 |
+
**A note on how to read this document.** Several statements here contradict things that are
|
| 17 |
+
true of *other* diffusion apps, or true of LTX-2.5 on a normal GPU box but false on ZeroGPU. The
|
| 18 |
+
biggest is §8: **`enable_model_cpu_offload()` must not be used.** If a future reader's instinct
|
| 19 |
+
says otherwise, §8.4 explains why the instinct is wrong here.
|
| 20 |
|
| 21 |
---
|
| 22 |
|
| 23 |
+
## 1. Decision 1 — Integration path: **Diffusers**, pinned to an exact SHA
|
| 24 |
|
| 25 |
We build on `diffusers`, loading `Lightricks/LTX-2.5-Diffusers`.
|
| 26 |
|
|
|
|
| 34 |
|
| 35 |
### 1.2 Why C
|
| 36 |
|
| 37 |
+
**Path A is a heavy, fragile install for a Space.** It needs `uv sync` of a git workspace,
|
| 38 |
+
CUDA >= 12.7, torch ~= 2.7. A Space build that resolves a git workspace at image-build time is
|
| 39 |
+
a build we cannot reproduce or pin.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
**Path B is the documented cause of the fragility we are told to avoid.** The 2.3 project's
|
| 42 |
+
`requirements.txt` vendors an entire ComfyUI runtime — ~50 dependencies, nearly all unpinned:
|
| 43 |
+
`spandrel`, `torchsde`, `kornia`, `PyOpenGL`, `glfw`, `SQLAlchemy`, `alembic`, `Mako`, `blake3`,
|
| 44 |
+
`comfyui-frontend-package`, `comfyui-workflow-templates`, `comfyui-embedded-docs`, plus a
|
| 45 |
+
hand-maintained block of *custom-node deps we discovered are required but aren't auto-installed
|
| 46 |
+
by their respective nodes* (`gguf`, `imageio_ffmpeg`, `opencv-python`, `matplotlib`,
|
| 47 |
+
`scikit-image`, `onnxruntime`, `yt-dlp`). That comment block is the evidence: the dependency
|
| 48 |
+
surface was discovered by breakage, not by design. Diffusers replaces all of it with ~16 pinned
|
| 49 |
+
direct deps and zero custom-node surface.
|
| 50 |
|
| 51 |
**Decisive coupling to §2.** The `*-comfy-int8-convrot` quantised weights are **ComfyUI-only**
|
| 52 |
+
— the model card says so. Choosing A or C means those 36.9 GB of int8 weights are unusable to
|
| 53 |
+
us. Path choice and disk budget are one decision, not two.
|
| 54 |
+
|
| 55 |
+
### 1.3 The pinned revision — and why a release tag will not do
|
| 56 |
+
|
| 57 |
+
```
|
| 58 |
+
diffusers @ git+https://github.com/huggingface/diffusers@ac56fa2f25c1c15b4cbaa1e0b3255e9bc38c06dc
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
HEAD at 2026-08-19T12:11:04Z. Contains the LTX-2.5 merge (PR #14447, 2026-08-11) plus #14453
|
| 62 |
+
and #14462. PEP 508 direct reference — **not** the legacy `#egg=` form.
|
| 63 |
+
|
| 64 |
+
**A SHA is more pinned than a version range, not less.** It is byte-for-byte reproducible and
|
| 65 |
+
cannot drift. It costs a git clone at build time and manual upgrades; both are acceptable.
|
| 66 |
|
| 67 |
+
**The latest release (0.39.0) does not have 2.5, and it fails LATE.** `LTX2Pipeline` imports
|
| 68 |
+
fine on 0.39.0 — so the app starts, the UI renders, weights begin downloading — and only then
|
| 69 |
+
does it fail on a missing `temporal_latent_upsampler`. A failure that happens after 70 GB of
|
| 70 |
+
download is the worst kind.
|
| 71 |
|
| 72 |
+
**Therefore: a startup fail-fast assertion.**
|
| 73 |
+
|
| 74 |
+
```python
|
| 75 |
+
assert diffusers.__version__ == "0.40.0.dev0", (
|
| 76 |
+
f"LTX-2.5 needs the pinned diffusers SHA (reports 0.40.0.dev0); got {diffusers.__version__}. "
|
| 77 |
+
"See docs/PLAN.md §1.3."
|
| 78 |
+
)
|
| 79 |
+
```
|
| 80 |
+
|
| 81 |
+
This converts a late, confusing, expensive failure into an immediate, obvious one. It lives in
|
| 82 |
+
`preflight.py` (§7.1) and it is a test case (§9.3).
|
| 83 |
+
|
| 84 |
+
### 1.4 The single-file escape hatch: investigated and REJECTED
|
| 85 |
|
| 86 |
`diffusers/loaders/single_file_utils.py` does contain an LTX-2 route:
|
| 87 |
`CHECKPOINT_KEY_NAMES["ltx2"]` → `model_type = "ltx2-dev"` → config repo
|
| 88 |
+
`{"pretrained_model_name_or_path": "Lightricks/LTX-2"}`. Note the repo: **`LTX-2`, i.e. 2.0,
|
| 89 |
+
not 2.5**. The converters it ships are only `convert_ltx2_transformer_to_diffusers`,
|
| 90 |
+
`convert_ltx2_vae_to_diffusers` and `convert_ltx2_audio_vae_to_diffusers` — all keyed on
|
| 91 |
+
`LTX_2_0_*` rename dicts.
|
|
|
|
|
|
|
| 92 |
|
| 93 |
There is **no** single-file converter for 2.5's `connectors`, `diffusion_decoder`, `vocoder`,
|
| 94 |
+
`duration_head` or `latent_upsampler`. Even a successful load would give us a pipeline missing
|
| 95 |
+
the components that make 2.5 what it is. Settled; do not revisit without new upstream
|
| 96 |
+
converters.
|
| 97 |
|
| 98 |
---
|
| 99 |
|
| 100 |
+
## 2. Decision 2 — Disk, RAM and dtype budget
|
| 101 |
|
| 102 |
+
Three separate resource ceilings, and the tightest one is not the obvious one.
|
| 103 |
+
|
| 104 |
+
| resource | ceiling | our load | margin |
|
| 105 |
+
|---|---|---|---|
|
| 106 |
+
| **VRAM** (`xlarge`) | ~95 GiB | 67.2 GiB weights | ~28 GiB for activations — **comfortable** |
|
| 107 |
+
| **CPU RAM** | **~68 GB** (community cgroup observation, *not* an official figure — HF publishes ZeroGPU RAM as `"-"` / "Dynamic resources") | ~72 GB | **marginal — risk #1** |
|
| 108 |
+
| **Ephemeral disk** | 150 GB, **wiped on every restart/rebuild** | ~144 GB transient peak | **marginal — risk #1** |
|
| 109 |
+
|
| 110 |
+
**VRAM was never the problem.** The problems are RAM and transient disk.
|
| 111 |
|
| 112 |
### 2.1 Trap A — duplicate shard sets
|
| 113 |
|
| 114 |
+
Two folders ship the *same weights twice*, in two different shardings:
|
| 115 |
|
| 116 |
| folder | set 1 | set 2 | note |
|
| 117 |
|---|---|---|---|
|
| 118 |
| `transformer/` | 4 shards: 9.932 + 9.919 + 9.986 + 8.138 = **37.975 GB** | 8 shards: 4.991 + 4.944 + 4.994 + 4.928 + 4.977 + 4.995 + 4.994 + 3.163 = **37.986 GB** | only **one** `diffusion_pytorch_model.safetensors.index.json` |
|
| 119 |
| `connectors/` | single file: **6.344 GB** | 2 shards: 5.000 + 1.344 = **6.344 GB** | same |
|
| 120 |
|
| 121 |
+
A naive `snapshot_download` of those folders pulls **both** sets (+44.33 GB of pure
|
| 122 |
+
duplication). `from_pretrained` reads the index JSON and fetches only the referenced shards.
|
|
|
|
|
|
|
|
|
|
| 123 |
|
| 124 |
+
**Binding consequences:** (1) we rely on the index, never hand-rolling a `snapshot_download` of
|
| 125 |
+
`transformer/`; (2) any `preload_from_hub:` entry must name **only the live shards**. A wrong
|
| 126 |
+
list here does not fail fast — it fails after tens of GB, and on a 150 GB cap it evicts us.
|
|
|
|
| 127 |
|
| 128 |
### 2.2 Trap B — `transformer_full/` is the DEV transformer
|
| 129 |
|
| 130 |
+
A further **37.975 GB**, the *dev* (non-distilled) transformer. We ship the distilled 8-step
|
| 131 |
+
path, so we do not ship it. Documented as opt-in (§2.5), not deleted from our mental model.
|
|
|
|
| 132 |
|
| 133 |
+
### 2.3 The bf16 subset we ship
|
| 134 |
|
| 135 |
| component | GB | why |
|
| 136 |
|---|---:|---|
|
| 137 |
| `transformer/` (distilled, one shard set) | 37.975 | the model |
|
| 138 |
+
| `text_encoder/` (5 shards: 4.961 + 4.967 + 4.884 + 4.986 + 4.121) | 23.919 | Gemma-4 unified (§10 R-4) |
|
| 139 |
| `connectors/` | 6.344 | text → transformer conditioning; also a LoRA target |
|
| 140 |
+
| `vae/` | 1.452 | video VAE (fallback decoder — §7.4) |
|
| 141 |
| `audio_vae/` | 0.107 | audio VAE (mel ↔ latent) |
|
| 142 |
| `vocoder/` | 0.258 | mel → waveform |
|
| 143 |
| `latent_upsampler/` | 0.996 | stage-2 spatial upscale |
|
| 144 |
| `temporal_latent_upsampler/` | 0.262 | stage-2 temporal upscale |
|
| 145 |
+
| `diffusion_decoder/` | 0.834 | **the native 2.5 decoder** — §7.4 |
|
| 146 |
+
| `duration_head/` | 0.004 | duration prediction |
|
| 147 |
| `tokenizer/` | 0.032 | |
|
| 148 |
+
| **TOTAL** | **72.185** | **≈ 72.19 GB = 67.2 GiB** |
|
| 149 |
|
| 150 |
### 2.4 What we exclude, and why
|
| 151 |
|
|
|
|
| 153 |
|---|---:|---|
|
| 154 |
| `transformer_full/` | 37.975 | the DEV transformer; not needed for the distilled 8-step path |
|
| 155 |
| duplicate shard sets (transformer 8-shard + connectors 2-shard) | 44.330 | Trap A — same weights, different sharding |
|
| 156 |
+
| `ltx-2.5-22b-distilled-lora-450-bf16.safetensors` (repo root) | 9.699 | the distilled LoRA for **DEV** workflows; useless when we ship the distilled transformer |
|
| 157 |
|
| 158 |
+
**Arithmetic check:** 72.185 + 37.975 + 44.330 + 9.699 = **164.189 GB**, closing exactly against
|
| 159 |
+
the repo's stated 164.19 GB across 56 files. The accounting is complete; there is no
|
| 160 |
unexplained remainder.
|
| 161 |
|
| 162 |
+
### 2.5 The transient disk peak
|
| 163 |
+
|
| 164 |
+
Startup holds **two** copies: the HF cache (~72 GB) *and* the packed copy ZeroGPU writes to
|
| 165 |
+
`~/.zerogpu/tensors` (~72 GB) ≈ **144 GB against a 150 GB cap**. The 2.3 project was evicted
|
| 166 |
+
with exactly this: `"Workload evicted, storage limit exceeded (150G)"`.
|
| 167 |
+
|
| 168 |
+
**Mitigating fact:** the packer auto-prunes the HF cache blobs after packing —
|
| 169 |
+
`ZEROGPU_MMAP_AUTOPRUNE_PATTERN` = `~/.cache/huggingface/hub/models--*/blobs/*` — logging
|
| 170 |
+
`"Cleaned NGB of tensor files ... after packing"`. So the 144 GB peak is **transient**. It is
|
| 171 |
+
still a peak, and 6 GB of margin is not much. This is why §5.6's preflight measures free disk
|
| 172 |
+
and says a number rather than letting the platform evict us anonymously.
|
| 173 |
+
|
| 174 |
+
Remaining headroom after packing: `150 − 72.19 ≈ 77.8 GB` for the pip environment, user LoRA
|
| 175 |
+
uploads and outputs. `transformer_full/` (37.98 GB) fits in that — which is why it is opt-in
|
| 176 |
+
rather than impossible — but enabling it leaves ~40 GB and makes output eviction urgent.
|
| 177 |
+
|
| 178 |
+
### 2.6 dtype and quantization — **default bf16**, quantization is a documented fallback
|
| 179 |
+
|
| 180 |
+
Packed weight size decides which ZeroGPU tier we land on, and the tier decides quota cost. The
|
| 181 |
+
threshold is `xlarge_threshold = 42.00 GiB`: **exceed it and ZeroGPU silently auto-promotes to
|
| 182 |
+
`xlarge` and doubles quota cost, with no code change.**
|
| 183 |
+
|
| 184 |
+
| config | approx total | packed vs 42.00 GiB | tier | quota | verdict |
|
| 185 |
+
|---|---:|---|---|---|---|
|
| 186 |
+
| **bf16 everything** | 72.19 GB (67.2 GiB) | over | `xlarge` | **2×** | **DEFAULT** — documented by the model card, matches the official Lightricks Space pattern; RAM/disk marginal |
|
| 187 |
+
| fp8 transformer (~19) + bf16 TE (~24) + rest (~10) | ~53 GB (~49.3 GiB) | over | `xlarge` | 2× | comfortable RAM/disk, no quota win |
|
| 188 |
+
| fp8 transformer + int8 TE | ~41 GB (~38.2 GiB) | **under** | `large` | **1×** | best quota economics, highest accuracy risk |
|
| 189 |
|
| 190 |
+
**Why bf16 is the default and not the quantised config.** Shipping quantised by default would
|
| 191 |
+
trade a *known* uncertainty for an *unverified* one:
|
|
|
|
|
|
|
| 192 |
|
| 193 |
+
- The RAM ceiling (~68 GB) is a community cgroup observation, not an official number. It is
|
| 194 |
+
uncertain, but it is uncertainty about a configuration that upstream documents and that a
|
| 195 |
+
production Space already runs.
|
| 196 |
+
- Quantization carries its own **unquantified** risk on this specific platform. bitsandbytes
|
| 197 |
+
`Int8Params` / `Params4bit` and torchao both use **custom tensor subclasses**, and ZeroGPU's
|
| 198 |
+
packer tracks aliased tensors in a `WeakTensorKeyDictionary`. Whether it packs subclassed
|
| 199 |
+
tensors correctly is **unverified**. A packer that mishandles them fails in exactly the way
|
| 200 |
+
we cannot tolerate: quietly, as a quality regression.
|
| 201 |
+
|
| 202 |
+
So bf16 ships, `size="xlarge"` is **declared explicitly** rather than being silently promoted
|
| 203 |
+
(§8.5), and dtype/quantization is **one switch in `model_files.py`** so moving to fp8 or fp8+int8
|
| 204 |
+
is a one-line change. This is **technical risk #1** (§10), and it is to be settled by
|
| 205 |
+
measurement on the Space, not by argument.
|
| 206 |
+
|
| 207 |
+
### 2.7 Output retention — 20 GB, evicted oldest-first
|
| 208 |
+
|
| 209 |
+
`outputs/` is capped at **20 GB**, bounded-bytes, oldest-first eviction, as a single constant in
|
| 210 |
+
`config.py` overridable by env. Swept **on startup as well as after each run**, logging what was
|
| 211 |
+
evicted.
|
| 212 |
+
|
| 213 |
+
Rationale: ~78 GB headroom minus the pip environment and a user LoRA or two. 20 GB is many
|
| 214 |
+
generations and leaves the margin the 2.3 Space did not have when it filled its disk and went
|
| 215 |
+
unhealthy.
|
| 216 |
|
| 217 |
---
|
| 218 |
|
| 219 |
+
## 3. Decision 3 — Mode coverage: 11 UI modes over 4 real workflows
|
| 220 |
+
|
| 221 |
+
All 11 requested modes are reachable. **They are not 11 code paths**, and the plan says so
|
| 222 |
+
plainly — describing them as independent implementations would be dishonest and would produce a
|
| 223 |
+
much worse registry.
|
| 224 |
+
|
| 225 |
+
### 3.1 The four real workflows
|
| 226 |
+
|
| 227 |
+
From `_workflow_map` in `diffusers/modular_pipelines/ltx2/modular_blocks_ltx25.py`:
|
| 228 |
|
| 229 |
+
```
|
| 230 |
+
text2video {prompt}
|
| 231 |
+
image2video {image, prompt}
|
| 232 |
+
condition {conditions, prompt}
|
| 233 |
+
in_context {reference_conditions, num_frames, prompt}
|
| 234 |
+
```
|
| 235 |
|
| 236 |
+
Our **11 UI modes are a presentation layer over those four**, plus the audio-latent input route
|
| 237 |
+
and the audio-only export. That is the honest description, and it is also the design that keeps
|
| 238 |
+
the registry small: `modes.py` is data over four builders, not eleven.
|
| 239 |
+
|
| 240 |
+
### 3.2 Three facts that shape everything
|
| 241 |
|
| 242 |
**(a) One conditioning mechanism covers both image and video.**
|
| 243 |
+
`LTX2VideoCondition.frames` is typed
|
| 244 |
`PIL.Image.Image | list[PIL.Image.Image] | np.ndarray | torch.Tensor` — "the image (**or
|
| 245 |
+
video**) to condition the video on" — and carries `index` and `strength`. Image- and
|
| 246 |
+
video-conditioning are *the same code path* with a different payload. See §5.7 for the
|
| 247 |
+
`index` trap, which is severe.
|
|
|
|
| 248 |
|
| 249 |
**(b) The model is jointly audio-visual. There is no video-off switch.**
|
| 250 |
+
The return type is `LTX2PipelineOutput(frames=..., audio=...)`, both non-optional tensors. Every
|
| 251 |
+
call spends video compute and returns a video. This is the most important cost fact in the app.
|
|
|
|
| 252 |
|
| 253 |
**(c) Audio input is not a first-class kwarg.**
|
| 254 |
+
There is no `audio=` argument. Audio enters as `audio_latents` — upstream documents it as
|
| 255 |
+
*"Optional pre-encoded audio latents; random noise is used when not provided"* — combined with
|
| 256 |
+
`noise_scale`. `prepare_audio_latents` routes through `_create_noised_state`, whose body is
|
| 257 |
+
`noise_scale * noise + (1 - noise_scale) * latents`: img2img-style strength, where `1.0`
|
| 258 |
+
discards the input entirely.
|
|
|
|
|
|
|
| 259 |
|
| 260 |
+
### 3.3 The mode registry
|
| 261 |
|
| 262 |
+
| # | Mode | Workflow | How it is reached | Cost note |
|
| 263 |
+
|---|---|---|---|---|
|
| 264 |
+
| 1 | text → video | text2video | `prompt` | first-class |
|
| 265 |
+
| 2 | image → video | condition | one `LTX2VideoCondition(frames=PIL, index=0)`, empty prompt | first-class |
|
| 266 |
+
| 3 | image + text → video | condition | as #2 with a prompt | first-class |
|
| 267 |
+
| 4 | video → video | condition | `LTX2VideoCondition(frames=<video>, strength<1.0)` | first-class |
|
| 268 |
+
| 5 | text → audio+video | text2video | #1 — the audio track is always produced | **the default shape** |
|
| 269 |
+
| 6 | image → audio+video | condition | #2 | first-class |
|
| 270 |
+
| 7 | image + text → audio+video | condition | #3 | first-class |
|
| 271 |
+
| 8 | text → audio | text2video | #1, export only the audio track | ⚠️ **still pays full video compute** |
|
| 272 |
+
| 9 | video → audio | condition | #4, export only the audio track | ⚠️ same |
|
| 273 |
+
| 10 | audio → audio | condition | `audio_latents` + `noise_scale`, export only audio | ⚠️ same |
|
| 274 |
+
| 11 | audio → video | condition | `audio_latents` + `noise_scale`, export only video | ⚠️ pays audio encode on top |
|
| 275 |
+
|
| 276 |
+
**Mandatory mitigation for modes 8–11:** run at the **minimum legal resolution**, **skip the
|
| 277 |
+
stage-2 spatial upscale**, export only the requested track. The UI must say in words that
|
| 278 |
+
audio-only modes still render video internally — a user who does not know that will read the
|
| 279 |
+
runtime as a bug.
|
| 280 |
+
|
| 281 |
+
### 3.4 Multishot — **downgraded to honest**
|
| 282 |
+
|
| 283 |
+
There is **no multishot or scene API**. Multishot is prompt-structure driven only.
|
| 284 |
+
|
| 285 |
+
Worse: the 2.5 enhancer system prompts **explicitly instruct the model not to describe scene
|
| 286 |
+
cuts**, so prompt-level multishot actively fights the enhancer. We present it as *prompt-structure
|
| 287 |
+
driven, unverified, no API support* — a documented technique in the UI copy, not a feature with a
|
| 288 |
+
button that implies support we cannot back.
|
| 289 |
+
|
| 290 |
+
(Superseding an earlier draft of this plan that called it a first-class affordance. It is not.)
|
| 291 |
+
|
| 292 |
+
### 3.5 Prompt enhancement: **unavailable by default; we ship no dead toggle**
|
| 293 |
+
|
| 294 |
+
`LTX25AutoBlocks` declares a `prompt_enhancer` component, and it appears in
|
| 295 |
+
`_optional_components`. But **the Diffusers pack has no `prompt_enhancer/` folder** — all 56
|
| 296 |
+
files enumerated. It would come from `google/gemma-4-E2B-it`, a separate model. So
|
| 297 |
+
`enable_prompt_enhancement=True` fails on our subset.
|
| 298 |
+
|
| 299 |
+
Shipping a toggle that cannot work is exactly the silent lie §5 exists to prevent. No UI
|
| 300 |
+
control until the component is fetchable and wired.
|
| 301 |
+
|
| 302 |
+
Recorded for whoever enables it later: `prompt_max_new_tokens=600`, greedy decoding,
|
| 303 |
+
`prompt_enhancement_seed` is **inert under LTX-2.5's greedy decoding**, and the system prompt is
|
| 304 |
+
auto-selected between `LTX2_5_T2V_DEFAULT_SYSTEM_PROMPT` and `LTX2_5_I2V_DEFAULT_SYSTEM_PROMPT`.
|
| 305 |
+
|
| 306 |
+
### 3.6 Duration prediction: ships
|
| 307 |
+
|
| 308 |
+
`duration_head` is in the pack (0.004 GB) and in `_optional_components`. Omit `num_frames`, pass
|
| 309 |
+
`min_seconds` / `max_seconds`. See §5.7 for the `num_frames` default trap and the >1-prompt
|
| 310 |
+
restriction.
|
| 311 |
|
| 312 |
---
|
| 313 |
|
| 314 |
## 4. Decision 4 — LoRA: the headline feature
|
| 315 |
|
| 316 |
+
User-uploadable LoRA at inference time is why this app exists. It must be the most robust thing
|
| 317 |
+
in it.
|
| 318 |
|
| 319 |
### 4.1 Verified mechanics
|
| 320 |
|
| 321 |
+
- All generation pipelines are declared
|
|
|
|
| 322 |
`class LTX2…Pipeline(DiffusionPipeline, FromSingleFileMixin, LTX2LoraLoaderMixin)`.
|
| 323 |
+
- `LTX2LoraLoaderMixin._lora_loadable_modules = ["transformer", "connectors"]`. Note
|
| 324 |
+
**`connectors`** — a LoRA may legitimately touch the text-conditioning path.
|
|
|
|
| 325 |
- **Both key conventions load.** `lora_state_dict()` detects
|
| 326 |
`is_non_diffusers_format = any(k.startswith("diffusion_model.") for k in state_dict)` and runs
|
| 327 |
`_convert_non_diffusers_ltx2_lora_to_diffusers(state_dict)`; separately it detects
|
| 328 |
`has_connector = any(k.startswith("text_embedding_projection.") for k in state_dict)` and
|
| 329 |
+
converts those into the connectors sub-dict.
|
|
|
|
| 330 |
- `lora_state_dict(..., return_lora_metadata=True)` returns the file's `__metadata__`.
|
| 331 |
|
| 332 |
+
### 4.2 The danger: upstream validation is almost non-existent
|
| 333 |
|
| 334 |
+
The **only** built-in check is a substring test:
|
| 335 |
+
`ValueError("Invalid LoRA checkpoint. Make sure all LoRA param names contain `'lora'` substring.")`
|
| 336 |
|
| 337 |
+
A LoRA for a *completely different architecture* passes that check — it has `lora` in its key
|
| 338 |
+
names — then attaches to **zero** modules. Generation proceeds. Output looks like the base
|
| 339 |
+
model. **No error is raised.** That is silent garbage, and it is precisely the class of failure
|
| 340 |
+
the operator has told us to eliminate.
|
| 341 |
|
| 342 |
+
We do have a real compatibility signal. Reading the official LoRA's safetensors header directly:
|
| 343 |
+
3320 tensors, all under prefix `diffusion_model.`, peft-style `lora_A.weight`/`lora_B.weight`,
|
| 344 |
+
targeting `to_q`/`to_k`/`to_v`/`to_gate_logits`, `adaln_single`, `audio_adaln_single`; and
|
|
|
|
| 345 |
`__metadata__ = {'lora_rank': '450', 'lora_alpha': '450', 'model_version': '2.5.0', 'license': …}`.
|
| 346 |
+
**`model_version` is machine-readable.**
|
| 347 |
|
| 348 |
### 4.3 The validator (7 stages — each stage is a test unit)
|
| 349 |
|
|
|
|
| 351 |
|
| 352 |
| # | Stage | Rule | On failure |
|
| 353 |
|---|---|---|---|
|
| 354 |
+
| 1 | **Size bound** | reject above a configured max | refuse before any download or parse |
|
| 355 |
+
| 2 | **Header-only parse** | read the safetensors header, **never** materialise the payload | — |
|
| 356 |
+
| 3 | **Format** | safetensors with a parseable header | distinct messages: not-safetensors vs unparseable-header |
|
| 357 |
+
| 4 | **Version** | read `__metadata__.model_version` | **warn** if absent, **reject** on explicit mismatch |
|
| 358 |
+
| 5 | **Convention** | classify: `diffusion_model.` / `transformer.` / `text_embedding_projection.` / unknown | on unknown, the error **names the prefixes actually found** |
|
| 359 |
+
| 6 | **Pairing** | `lora_A`/`lora_B` (or `lora_down`/`lora_up`) must pair | name the unpaired keys |
|
| 360 |
+
| 7 | **Attach assertion** | **after** `load_lora_weights`, assert the adapter attached to **> 0** modules | specific error if zero — *the anti-silent-garbage check* |
|
| 361 |
+
|
| 362 |
+
Stage 2 is a hard rule: a multi-GB upload must never be fully materialised just to discover it
|
| 363 |
+
is the wrong architecture. Stage 7 is the one upstream does not do for us and the one that
|
| 364 |
+
actually protects the user; it runs on **every** load, built-in LoRAs included.
|
| 365 |
|
| 366 |
+
### 4.4 Fusing — the connectors trap
|
|
|
|
| 367 |
|
| 368 |
+
`fuse_lora` / `unfuse_lora` default to `components=["transformer"]` **only**.
|
| 369 |
+
|
| 370 |
+
**Connector LoRA layers are silently not fused** unless we pass
|
| 371 |
+
`components=["transformer", "connectors"]`. A LoRA that trains the text-conditioning path would
|
| 372 |
+
partially apply — worse than not applying, because it looks like it worked. We always pass both
|
| 373 |
+
components explicitly, and this is a test case.
|
| 374 |
|
| 375 |
---
|
| 376 |
|
| 377 |
## 5. Decision 5 — Resilience doctrine
|
| 378 |
|
| 379 |
+
Binding on all code in the repo.
|
| 380 |
|
| 381 |
+
### 5.1 Pin every dependency, with upper bounds — and know what NOT to pin
|
| 382 |
|
| 383 |
+
Evidence from the predecessor: `spaces>=0.30.0` and `torch>=2.4.0` — floors with **no ceiling**
|
| 384 |
+
— alongside exactly one hard pin, `torchaudio==2.10.0`, carrying this comment:
|
|
|
|
| 385 |
|
| 386 |
> *"HF Spaces ships torch==2.10.0; bare 'torchaudio' lets pip pick an older wheel whose C++ ABI
|
| 387 |
> doesn't match → import crashes the whole audio code path… Pin the matching version."*
|
| 388 |
|
| 389 |
+
That comment is a post-mortem, applied to one line while ~50 stayed unbounded.
|
| 390 |
+
|
| 391 |
+
**But the inverse mistake is also real, and the 2.3 repo made it.** Its own `AGENTS.md:19` says
|
| 392 |
+
**do not pin `spaces`** — the platform pins its own version (live Spaces report 0.49.3 / 0.50.0 /
|
| 393 |
+
0.50.4 while PyPI is 0.51.1) and a conflicting pin breaks pip resolution at build time. Its
|
| 394 |
+
`requirements.txt:2` then pins `spaces>=0.30.0` anyway. We follow the rule the repo wrote and
|
| 395 |
+
did not keep: **no `spaces` line in `requirements.txt`**, and the import is guarded in code
|
| 396 |
+
(`try: import spaces / except ImportError: spaces = None`) so local dev works.
|
| 397 |
+
|
| 398 |
+
The full pin set:
|
| 399 |
+
|
| 400 |
+
| package | pin | why |
|
| 401 |
+
|---|---|---|
|
| 402 |
+
| `torch` | `==2.11.0` | ZeroGPU supports exactly 2.8.0 / 2.9.1 / 2.10.0 / **2.11.0**; ceiling is 2.11.0. CUDA 12.8, sm_120 needs cu128+ |
|
| 403 |
+
| `torchvision` | `==0.26.0` | **trap:** `torchvision 0.28.0` hard-declares `torch==2.13.0`, so a floating torchvision silently upgrades torch past the ceiling |
|
| 404 |
+
| `torchaudio` | `==2.11.0` | must match torch exactly (the ABI break above) |
|
| 405 |
+
| `gradio` | `>=6.24.0,<7` | **5.x is END OF LIFE** (last stable 5.50.0, 2025-11-21) |
|
| 406 |
+
| `diffusers` | `@ git+…@ac56fa2f…` | §1.3 |
|
| 407 |
+
| `transformers` | `>=5.15.1,<6` | |
|
| 408 |
+
| `accelerate` | `>=1.14.0,<2` | |
|
| 409 |
+
| `peft` | `>=0.20.0,<0.21` | LoRA backend |
|
| 410 |
+
| `safetensors` | `>=0.8.0,<0.9` | |
|
| 411 |
+
| `huggingface_hub` | `>=1.28.0,<2` | |
|
| 412 |
+
| `hf_xet` | `>=1.6.0,<2` | |
|
| 413 |
+
| `imageio` | `>=2.37.4,<3` | |
|
| 414 |
+
| `imageio-ffmpeg` | `>=0.6.0,<0.7` | |
|
| 415 |
+
| `av` | `>=18.1.0,<19` | PyAV — required by CRF re-compression (§5.7) |
|
| 416 |
+
| `numpy` | `>=2.5.2,<3` | **numpy 2.** Do NOT pin `<2`: 1.26.4 is 2 years EOL and nothing in the stack caps it |
|
| 417 |
+
| `pillow` | `>=12.3.0,<13` | |
|
| 418 |
+
| `spaces` | **absent, deliberately** | platform-pinned; see above |
|
| 419 |
+
|
| 420 |
+
**gradio 6 breaking changes that hit us:** `gr.Video` subtitle-tuple return removed;
|
| 421 |
+
`show_download_button` / `show_share_button` → `buttons=[...]`; `gr.Audio` `show_controls` →
|
| 422 |
+
`show_recording_waveform`; `min_length` / `max_length` → event-listener validators.
|
| 423 |
+
|
| 424 |
+
**Python 3.12 is required, not preferred:** numpy 2.5.2 needs ≥3.12 and `av` 18 needs ≥3.11. The
|
| 425 |
+
default 3.10.13 image silently resolves both *downward* rather than failing.
|
| 426 |
|
| 427 |
### 5.2 One central model-file indirection
|
| 428 |
|
| 429 |
+
Every Hub repo-id, revision, subfolder, filename **and dtype/quantization choice** (§2.6) lives
|
| 430 |
+
in **one** module — `ltx25_studio/model_files.py` — as data, never inlined at a call site.
|
|
|
|
| 431 |
|
| 432 |
+
A missing file must raise an error naming **both the file and its URL**. Not hypothetical:
|
| 433 |
+
Lightricks deleted a 2.3 checkpoint from the Hub with no warning and broke a live Space. When it
|
| 434 |
+
happens again, the message should tell the operator which URL to go look at.
|
|
|
|
| 435 |
|
| 436 |
+
Pinning a **revision** per repo is part of this module's job — a repo whose content can change
|
| 437 |
+
under us is not pinned just because its name is stable.
|
| 438 |
|
| 439 |
+
### 5.3 Never return an empty output — and the four layers that let it happen
|
| 440 |
|
| 441 |
Any generation that completes without producing a file **must** raise a specific, actionable
|
| 442 |
error that the UI renders visibly.
|
| 443 |
|
| 444 |
+
**The full anatomy of the 2.3 silent reset**, because each layer was individually "reasonable"
|
| 445 |
+
and that is the lesson:
|
| 446 |
+
|
| 447 |
+
1. `models.ensure_models` hit a missing file and did `logger.warning(...) + continue`.
|
| 448 |
+
2. ComfyUI's executor caught every node exception and returned a FAILURE **tuple**, without
|
| 449 |
+
raising.
|
| 450 |
+
3. An `OutputEvent(video_path="")` was pushed **unconditionally**.
|
| 451 |
+
4. The UI did `event.video_path if event.video_path else gr.update()` — rendering an empty path
|
| 452 |
+
as a silent form reset.
|
| 453 |
+
|
| 454 |
+
The user saw "nothing happened". No error, no log, no signal. It hid for a **week**.
|
| 455 |
+
|
| 456 |
+
**Our rules, both halves mandatory:**
|
| 457 |
+
- **An empty path becomes an error AT THE SOURCE**, and every layer propagates it. A success
|
| 458 |
+
path that can return a falsy artifact is a bug regardless of current reachability.
|
| 459 |
+
- **No "newest file in the output directory" fallback.** The 2.3 one had no run correlation and
|
| 460 |
+
would show the *previous* run's video for a failed run — a worse failure than showing nothing.
|
| 461 |
+
**Each run is stamped into its own output subdirectory**, and a run's artifact is only ever
|
| 462 |
+
looked up inside that run's directory.
|
| 463 |
|
| 464 |
+
### 5.4 Read parameters from config, never hardcode
|
| 465 |
|
| 466 |
+
Mel/audio parameters come from `audio_vae.config` at runtime: `audio_sampling_rate`,
|
| 467 |
+
`audio_hop_length`, `audio_mel_bins`, `audio_vae_mel_compression_ratio`,
|
| 468 |
`audio_vae_temporal_compression_ratio`, `audio_latent_channels`.
|
| 469 |
|
| 470 |
+
This mirrors the pipeline itself, whose constructor reads `self.vae.spatial_compression_ratio`,
|
| 471 |
+
`self.vae.temporal_compression_ratio`, `self.audio_vae.mel_compression_ratio`,
|
| 472 |
+
`self.audio_vae.temporal_compression_ratio` and `self.transformer.config.patch_size` from the
|
| 473 |
+
loaded components, using literals only as fallbacks. Same discipline. §5.7 extends this to CRF.
|
|
|
|
| 474 |
|
| 475 |
### 5.5 Enforce and surface the hard constraints
|
| 476 |
|
| 477 |
| constraint | source | handling |
|
| 478 |
|---|---|---|
|
| 479 |
+
| `height % 32 == 0`, `width % 32 == 0` | enforced upstream (raises) in all generation pipelines *and* the latent-upsample pipeline | validate before the call, snap in the UI, **show** the snapped value |
|
| 480 |
+
| `num_frames % 8 == 1` | **not** hard-raised upstream. Falls out of `latent_num_frames = (num_frames - 1) // vae.temporal_compression_ratio + 1`, which **silently floors** a bad value | we raise. **The `8` is `vae.temporal_compression_ratio`** — see below |
|
| 481 |
+
| audio/video length agreement | audio and video lengths are computed independently, and a supplied `audio_latents` **overrides** the duration-derived audio length (§5.7) | validate they agree; raise rather than emit a desynced file |
|
| 482 |
+
|
| 483 |
+
**The `num_frames` modulus is a parameter, not a literal.** Hardcoding `8` would violate §5.4.
|
| 484 |
+
Resolution:
|
| 485 |
+
|
| 486 |
+
- `constraints.py` takes the ratio as a **parameter with a documented fallback of `8`**, because
|
| 487 |
+
the UI must validate input *before* any VAE exists.
|
| 488 |
+
- `pipelines.py` **asserts the loaded `vae.temporal_compression_ratio` equals that fallback** at
|
| 489 |
+
load time, raising loudly if upstream ever changes it.
|
| 490 |
+
|
| 491 |
+
That gives a working pre-load validator *and* a tripwire. Both are test cases.
|
| 492 |
|
| 493 |
**Verified constants** (`diffusers/pipelines/ltx2/utils.py`):
|
| 494 |
|
|
|
|
|
|
|
|
|
|
| 495 |
```
|
| 496 |
+
DISTILLED_SIGMA_VALUES = [1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875] # 8 steps, CFG=1
|
| 497 |
+
STAGE_2_DISTILLED_SIGMA_VALUES = [0.909375, 0.725, 0.421875] # 3 steps
|
| 498 |
+
```
|
| 499 |
+
|
| 500 |
+
Re-exported and asserted in `constraints.py` against the installed diffusers, so a schedule
|
| 501 |
+
change upstream fails a test instead of silently changing output quality.
|
| 502 |
+
|
| 503 |
+
### 5.6 Preflight — fail loudly on resources, before the platform kills us
|
| 504 |
|
| 505 |
+
At startup, `preflight.py` measures **free RAM and free disk** and compares them against the
|
| 506 |
+
requirement computed from the `model_files.py` table. If short, it raises a specific error
|
| 507 |
+
naming the numbers:
|
| 508 |
+
|
| 509 |
+
> `need ~67.2 GiB RAM for the bf16 load, saw 41.3 GiB` (see §2.6 for the quantised fallbacks)
|
| 510 |
+
|
| 511 |
+
This is the fail-loudly principle applied to the resource budget. It turns an anonymous OOM or a
|
| 512 |
+
`"Workload evicted, storage limit exceeded (150G)"` into a diagnosable message that names the
|
| 513 |
+
fix. It also runs the §1.3 diffusers-version assertion. Test case.
|
| 514 |
+
|
| 515 |
+
### 5.7 API gotchas that cause silent wrongness
|
| 516 |
+
|
| 517 |
+
Every row here is a **test case**, not a note. These are the ways this API is wrong-by-default.
|
| 518 |
+
|
| 519 |
+
| # | Gotcha | Consequence | Our handling |
|
| 520 |
+
|---|---|---|---|
|
| 521 |
+
| 1 | **`LTX2VideoCondition.index` is a LATENT index, not a pixel frame index.** `start_idx = max((latent_start_idx - 1) * vae_temporal_compression_ratio + 1, 0)` | **off-by-8 guaranteed** if a pixel frame number is passed | convert at the boundary in `conditioning.py`; UI speaks pixel frames, the pipeline gets latent indices |
|
| 522 |
+
| 2 | **Out-of-range condition indices are silently SKIPPED** with only a `logger.warning` | a whole conditioning image does nothing, silently | validate range ourselves and **raise**; this is a silent-failure path we convert into a loud one |
|
| 523 |
+
| 3 | **`noise_scale` default differs by class:** `0.0` on `LTX2Pipeline` / `LTX2ImageToVideoPipeline`, `None` on `LTX2ConditionPipeline` (resolving to `sigmas[0]` or 1.0) | copying a call between classes changes behaviour silently | always pass `noise_scale` explicitly; never rely on the default |
|
| 524 |
+
| 4 | **`num_frames` default differs:** `None` (auto-duration) on `LTX2Pipeline`/`ImageToVideo`/`Condition`; hard `121` on `LTX2InContextPipeline` and `LTX2HDRPipeline`, which have **no `duration_head` at all** | auto-duration silently unavailable on two pipelines | registry records per-workflow duration support; auto-duration also **raises** with >1 prompt and `num_frames=None` — validate before the call |
|
| 525 |
+
| 5 | `latents` must be **UNPACKED 5D** `[B,C,F,H,W]`; `audio_latents` **UNPACKED 4D** `[B,C,L,M]` | shape errors, or worse, silent misinterpretation | assert shapes at our boundary |
|
| 526 |
+
| 6 | **`output_type="latent"` returns ALREADY-DENORMALIZED latents** | re-feeding them re-normalizes → corrupted output | drives the mandatory `denormalize=False` in §7.4 |
|
| 527 |
+
| 7 | **Image-conditioning CRF re-compression is applied BY DEFAULT**, needs PyAV + uint8/PIL input; a float tensor **raises** unless `crf=0` | crashes on float input | normalise to uint8/PIL in `conditioning.py`; `av` is a pinned dep for this reason |
|
| 528 |
+
| 8 | **CRF is 18 when `text_encoder.config.model_type in ("gemma4_unified", "gemma4")`, else 33** | hardcoding either number is wrong for the other | **derive from `text_encoder.config.model_type` at runtime** (§5.4) |
|
| 529 |
+
| 9 | `from diffusers.pipelines.ltx2.export_utils import encode_video` is **DEPRECATED**, removal in 0.40.0 — and our pin *is* 0.40.0.dev0 | breaks on the very revision we pin | use `from diffusers.utils import encode_video` |
|
| 530 |
+
| 10 | `spatio_temporal_guidance_blocks` defaults to `[28]` for 2.3/2.5 (`[29]` was 2.0) and is a **mutable shared default** | in-place mutation corrupts every later call in the process | never mutate in place; copy before modifying |
|
| 531 |
+
| 11 | A supplied `audio_latents` **overrides** the duration-derived `audio_num_frames`: upstream logs `"Got audio_latents of shape […], audio_num_frames will be inferred"` then does `_, _, audio_num_frames, _ = audio_latents.shape` | supplying audio **silently redefines audio length** while `num_frames` still governs video → desync | §5.5 validation, with a concrete desync test case |
|
| 532 |
+
| 12 | `fuse_lora`/`unfuse_lora` default `components=["transformer"]` | connector LoRA layers silently unfused | §4.4 — always pass both |
|
| 533 |
+
|
| 534 |
+
Reference: `_optional_components = ["processor", "prompt_enhancer", "duration_head"]` (plus
|
| 535 |
+
`audio_scheduler` on Condition), and
|
| 536 |
+
`model_cpu_offload_seq = "prompt_enhancer->text_encoder->connectors->duration_head->transformer->vae->audio_vae->vocoder"`.
|
| 537 |
+
Recorded for completeness — **but see §8.4: we do not use accelerate's offload at all.**
|
| 538 |
+
|
| 539 |
+
### 5.8 Operational lessons from the 2.3 post-mortem
|
| 540 |
+
|
| 541 |
+
| # | Lesson | Rule |
|
| 542 |
+
|---|---|---|
|
| 543 |
+
| 1 | **ZeroGPU quota attribution rides a contextvar** (`gradio.context.LocalContext.request`). It does **not** cross a plain `threading.Thread` — without it every user is bucketed as Space-anonymous and you get *"Space app has reached its GPU limit"* at **0 minutes used** | any thread we spawn must carry `contextvars.copy_context()`; better, do not spawn plain threads across the GPU boundary |
|
| 544 |
+
| 2 | The `duration=` callable must accept **the exact same positional signature** as the decorated function, trailing kwargs included | otherwise TypeError at schedule time; test the signature match |
|
| 545 |
+
| 3 | **Hidden 1.5× duration multiplier on `large`** (`duration_factor`): `duration=60` is scheduled as 90 s. On `xlarge` the factor is skipped but the request is **doubled** instead | never reason about wall-clock quota from the `duration=` number alone |
|
| 546 |
+
| 4 | Default duration is 60 s. The **max is a server-side, tier-dependent, unpublished cap** | **discover it, never hardcode it.** Empirically ~120 s is the safe ceiling; the official Lightricks LTX Space ships `@spaces.GPU(duration=75, size="xlarge")` |
|
| 547 |
+
| 5 | **Do NOT auto-retry a ZeroGPU timeout** using the captured Gradio request: the `X-IP-Token` ages past TTL during attempt 1, so attempt 2 always 401s with *"Expired ZeroGPU proxy token"* — **masquerading as the root cause** | surface the original timeout; never retry behind the user's back |
|
| 548 |
+
| 6 | `preload_from_hub` is **capped at 10 entries**, and preloaded files are owned by the *build* user and **read-only to runtime uid 1000** — a later lazy `hf_hub_download` fails with `Permission denied (os error 13)` unless the cache is mirrored | §6.2 |
|
| 549 |
+
| 7 | Accumulated generations filled the 2.3 Space's disk and it went unhealthy | §2.7 — sweep on startup and after each run |
|
| 550 |
|
| 551 |
---
|
| 552 |
|
|
|
|
| 555 |
| | |
|
| 556 |
|---|---|
|
| 557 |
| **Space** | `techfreakworm/LTX2.5-Studio`, **private** |
|
| 558 |
+
| **SDK** | `sdk: gradio`, `sdk_version: 6.24.0` — **a custom Dockerfile forfeits ZeroGPU entirely** |
|
| 559 |
+
| **Python** | `python_version: "3.12"` — required, not optional (§5.1) |
|
| 560 |
+
| **Hardware** | ZeroGPU. `suggested_hardware: zero-a10g` is the **legacy id string the Spaces API still reports — it is NOT an A10G** (§8.1) |
|
| 561 |
+
| **Size** | `size="xlarge"` declared explicitly (§8.5) |
|
| 562 |
+
| **Weights** | README frontmatter `preload_from_hub:` |
|
| 563 |
+
| **Secret** | `HF_TOKEN` — required; `Lightricks/LTX-2.5-Diffusers` is gated |
|
| 564 |
| **Git** | GitHub `git@github.com:techfreakworm/ltx2.5-studio.git`; Space and GitHub stay in sync |
|
| 565 |
|
| 566 |
+
Naming precedent: `techfreakworm/LTX2.3-Studio`, `wan-studio` (private), `SAM3-cutout-studio`
|
| 567 |
+
(private).
|
| 568 |
+
|
| 569 |
+
### 6.2 `preload_from_hub` hazards
|
| 570 |
|
| 571 |
+
Two, and they interact:
|
|
|
|
| 572 |
|
| 573 |
+
1. **Only live shards** (§2.1 Trap A). A wrong list fails after tens of GB, not fast.
|
| 574 |
+
2. **Capped at 10 entries.** Our subset spans 11 component folders and ~20 individual shard
|
| 575 |
+
files, so a naive one-file-per-entry list **cannot fit**. Entries take the form
|
| 576 |
+
`repo_id file1,file2,…`, so one entry can name many files from one repo — the list must be
|
| 577 |
+
written that way. **Verification item before first deploy:** confirm there is no separate
|
| 578 |
+
per-entry file-count or line-length limit.
|
| 579 |
+
3. **Ownership.** Preloaded files belong to the *build* user and are read-only to runtime uid
|
| 580 |
+
1000. Any later lazy `hf_hub_download` into the same cache fails with
|
| 581 |
+
`Permission denied (os error 13)` unless the cache is mirrored. Design implication: **preload
|
| 582 |
+
everything we need, or mirror the cache at startup — do not mix the two strategies.**
|
| 583 |
+
|
| 584 |
+
### 6.3 🚨 BLOCKER — repo gate not accepted
|
| 585 |
|
| 586 |
`Lightricks/LTX-2.5-Diffusers` is gated (`gated: auto`) and the operator's token has **not**
|
| 587 |
accepted that gate. Verified response:
|
|
|
|
| 593 |
|
| 594 |
**https://huggingface.co/Lightricks/LTX-2.5-Diffusers**
|
| 595 |
|
| 596 |
+
Until accepted, nothing touching real weights can be built or tested. Everything that does not
|
| 597 |
+
need real weights proceeds. Escalated to the orchestrator.
|
| 598 |
|
| 599 |
---
|
| 600 |
|
| 601 |
## 7. Architecture
|
| 602 |
|
| 603 |
+
### 7.1 Module layout
|
| 604 |
|
| 605 |
```
|
| 606 |
+
app.py # entrypoint. `import spaces` FIRST (§8.2), then build + launch.
|
| 607 |
ltx25_studio/
|
| 608 |
+
config.py # device/dtype selection, paths, retention cap (§2.7), env overrides
|
| 609 |
+
model_files.py # §5.2 — THE single source of truth: repo ids, revisions, subfolders,
|
| 610 |
+
# filenames, sizes, AND the dtype/quantization switch (§2.6)
|
| 611 |
+
errors.py # typed exception hierarchy; every error carries an actionable detail
|
| 612 |
+
preflight.py # §5.6 — RAM/disk headroom vs the model_files requirement;
|
| 613 |
+
# §1.3 — the diffusers-version fail-fast assertion
|
| 614 |
+
zerogpu.py # §8 — the `spaces` shim, the static @spaces.GPU policy, the
|
| 615 |
+
# duration callable, contextvar propagation, and the
|
| 616 |
+
# "never return CUDA tensors" boundary
|
| 617 |
+
constraints.py # §5.5 — dims/frames/CRF/sigmas. Pure functions over numbers.
|
| 618 |
+
pipelines.py # construction on CPU, module-scope .to("cuda"), component wiring,
|
| 619 |
+
# and the vae.temporal_compression_ratio tripwire (§5.5)
|
| 620 |
+
conditioning.py # UI inputs -> list[LTX2VideoCondition]; pixel->latent index
|
| 621 |
+
# conversion (§5.7 #1); uint8/PIL normalisation (§5.7 #7)
|
| 622 |
+
audio.py # waveform <-> mel <-> audio_vae latents; params from config (§5.4);
|
| 623 |
+
# vocoder decode
|
| 624 |
+
modes.py # §3.3 registry: 11 UI modes as data over the 4 real workflows
|
| 625 |
+
generate.py # §7.4 stage graph. OWNS the never-empty invariant (§5.3) and the
|
| 626 |
+
# per-run output directory
|
| 627 |
+
outputs.py # mux video+audio, single-track export, 20 GB oldest-first eviction
|
| 628 |
lora/
|
| 629 |
+
inspect.py # header-only safetensors parse; metadata + convention detection
|
| 630 |
+
validate.py # validator stages 1-6 (§4.3)
|
| 631 |
+
apply.py # load_lora_weights, stage-7 attach assertion, both-component fusing
|
|
|
|
| 632 |
ui/
|
|
|
|
| 633 |
blocks.py # Gradio Blocks assembly
|
| 634 |
components.py # reusable widgets (condition list, LoRA picker, constraint hints)
|
| 635 |
+
handlers.py # thin adapters: UI event -> one call -> UI update. No logic.
|
| 636 |
```
|
| 637 |
|
| 638 |
### 7.2 Layering rules
|
| 639 |
|
| 640 |
- **Dependencies point inward.** `ui/` may import anything; `generate.py` must not import `ui/`.
|
| 641 |
`model_files.py`, `errors.py` and `constraints.py` import nothing from the package.
|
| 642 |
+
- **`modes.py` is data, not branching.** Adding a mode is a registry entry, not an `if` in
|
| 643 |
+
`generate.py`.
|
| 644 |
+
- **`ui/handlers.py` contains no business logic.** Marshal inputs → call one function → marshal
|
| 645 |
+
outputs. This is what makes the app testable without a GPU and without Gradio.
|
| 646 |
+
- **Errors cross layers as typed exceptions.** The UI is the only place that renders one as a
|
| 647 |
+
string. `generate.py` never returns an error-shaped success (§5.3).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 648 |
|
| 649 |
+
### 7.3 Why this shape
|
| 650 |
|
| 651 |
+
The predecessor put backend, models, modes and workflow in four flat modules with `ui.py`
|
| 652 |
+
importing all of them. That works until the failure modes get interesting: the empty-output bug
|
| 653 |
+
(§5.3) lived precisely in the seam between "workflow returned something" and "UI rendered it",
|
| 654 |
+
and there was no layer boundary at which to assert. Here `generate.py` owns that invariant and
|
| 655 |
+
can be tested for it directly.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 656 |
|
| 657 |
+
### 7.4 The generation stage graph — **classic pipelines PLUS an explicit diffusion decode**
|
| 658 |
|
| 659 |
+
This is the most consequential architectural decision in the document, because the obvious
|
| 660 |
+
implementation silently discards 2.5's headline quality feature.
|
| 661 |
|
| 662 |
+
**`LTX25AutoBlocks` docstring, verbatim:**
|
| 663 |
+
|
| 664 |
+
> *"Auto blocks for LTX-2.5 supporting text-to-video, image-to-video, condition-to-video and
|
| 665 |
+
> in-context (IC-LoRA) generation (joint video + audio). Identical to `LTX2AutoBlocks` except
|
| 666 |
+
> that the video decoder is `LTX2DiffusionVaeDecoderStep`, **since the diffusion decoder is the
|
| 667 |
+
> native default from LTX-2.5 on.** To decode with the convolutional VAE instead, swap the
|
| 668 |
+
> decode block."*
|
| 669 |
+
|
| 670 |
+
**But the classic `LTX2ConditionPipeline` / `LTX2ImageToVideoPipeline` decode with
|
| 671 |
+
`self.vae.decode(...)` — the convolutional VAE, NOT the diffusion decoder.** So a naive classic
|
| 672 |
+
call silently gives up what the model card calls *"a new diffusion video decoder [that] replaces
|
| 673 |
+
the VAE reconstruction stage"*. It would not error. It would just be worse, forever, and nobody
|
| 674 |
+
would know why.
|
| 675 |
+
|
| 676 |
+
**Decision: classic pipelines, with the diffusion decode added as an explicit stage.**
|
| 677 |
|
| 678 |
```
|
| 679 |
+
stage 1 LTX2ConditionPipeline(..., sigmas=DISTILLED_SIGMA_VALUES,
|
| 680 |
+
output_type="latent")
|
| 681 |
+
→ latents, audio_latents
|
| 682 |
+
|
| 683 |
+
optional LTX2LatentUpsamplePipeline(latents)
|
| 684 |
+
→ upsampled latents
|
| 685 |
+
|
| 686 |
+
stage 2 same pipeline, latents=upsampled,
|
| 687 |
+
sigmas=STAGE_2_DISTILLED_SIGMA_VALUES,
|
| 688 |
+
noise_scale=STAGE_2_DISTILLED_SIGMA_VALUES[0],
|
| 689 |
+
output_type="latent"
|
| 690 |
+
→ refined latents
|
| 691 |
+
|
| 692 |
+
decode V LTX2VideoDiffusionDecodePipeline(latents=..., denormalize=False) ← see below
|
| 693 |
+
decode A audio_vae.decode(audio_latents) → mel → vocoder(...) → waveform
|
| 694 |
```
|
| 695 |
|
| 696 |
+
**`denormalize=False` is MANDATORY.** `output_type="latent"` returns *already-denormalized*
|
| 697 |
+
latents (§5.7 #6). Passing the default `True` **double-denormalizes and silently corrupts the
|
| 698 |
+
output**. This is a test case.
|
| 699 |
|
| 700 |
+
**Why not the modular path.** The model card's own Option C example uses the classic pipelines,
|
| 701 |
+
so that is the documented, lower-risk API. The modular path (`LTX25ModularPipeline` /
|
| 702 |
+
`LTX25AutoBlocks`, loaded from `modular_model_index.json`) is newer, and its interaction with
|
| 703 |
+
**ZeroGPU's tensor packing** (§8.3) and with **`load_lora_weights`** (§4) is unverified. Given
|
| 704 |
+
LoRA is the headline feature and ZeroGPU is the deploy target, betting both on an unverified
|
| 705 |
+
integration is the wrong trade.
|
| 706 |
|
| 707 |
+
The modular path is recorded here as **the canonical alternative**: if we move to it, the decode
|
| 708 |
+
block is swappable exactly as the docstring describes, and `vae/` (1.452 GB) stays in the subset
|
| 709 |
+
as the conv-VAE fallback either way.
|
| 710 |
|
| 711 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 712 |
|
| 713 |
+
## 8. ZeroGPU execution model
|
| 714 |
|
| 715 |
+
**This section replaces the conventional wisdom about diffusion-model offloading. On ZeroGPU
|
| 716 |
+
that wisdom is not merely suboptimal — it is counterproductive.**
|
|
|
|
| 717 |
|
| 718 |
+
### 8.1 The hardware is not what the API says
|
| 719 |
+
|
| 720 |
+
ZeroGPU is **NVIDIA RTX PRO 6000 Blackwell Server Edition**, partitioned via MIG (sm_120).
|
| 721 |
|
| 722 |
+
| tier | VRAM | quota |
|
| 723 |
+
|---|---:|---|
|
| 724 |
+
| `large` (default) | **48 GB** | 1× |
|
| 725 |
+
| `xlarge` | **96 GB** | 2× |
|
| 726 |
|
| 727 |
+
The Spaces API still reports the legacy id string **`zero-a10g`**. **Do not read that as an
|
| 728 |
+
A10G** — it is a stale identifier, and sizing decisions made from it would be wrong by a factor
|
| 729 |
+
of several.
|
|
|
|
|
|
|
| 730 |
|
| 731 |
+
**VRAM is not our constraint.** 67.2 GiB of weights against `xlarge`'s ~95 GiB leaves ~28 GiB
|
| 732 |
+
for activations. The constraints are CPU RAM and transient disk (§2).
|
| 733 |
+
|
| 734 |
+
### 8.2 Import order is load-bearing
|
| 735 |
+
|
| 736 |
+
```python
|
| 737 |
+
import spaces # MUST be first
|
| 738 |
+
import torch # only after
|
| 739 |
+
import diffusers # only after
|
| 740 |
+
```
|
| 741 |
+
|
| 742 |
+
`import spaces` **must precede** torch / diffusers / transformers, or startup fails with
|
| 743 |
+
`"CUDA has been initialized before importing the spaces package"`. This is why `app.py` is thin
|
| 744 |
+
and why the `spaces` shim lives in `zerogpu.py` with a guarded import for local dev.
|
| 745 |
+
|
| 746 |
+
### 8.3 The packing model: build on CPU, `.to("cuda")` at module scope
|
| 747 |
+
|
| 748 |
+
ZeroGPU **replaces** CPU offload with its own disk-packing engine. The correct pattern inverts
|
| 749 |
+
normal GPU-box practice:
|
| 750 |
+
|
| 751 |
+
1. Build the pipeline **on CPU**, at module scope.
|
| 752 |
+
2. Call `pipe.to("cuda")` **at module scope**. That call is **intercepted**, rewritten to
|
| 753 |
+
`.to("cpu")`, and every tensor is packed to disk (`~/.zerogpu/tensors`, O_DIRECT). After
|
| 754 |
+
packing, the main process holds no weights.
|
| 755 |
+
3. Apply `@spaces.GPU(...)` **statically at module import**. Dynamic or inner decoration fails
|
| 756 |
+
ZeroGPU's startup analyzer with *"No @spaces.GPU function detected during startup"*.
|
| 757 |
+
4. Inside the decorated function real CUDA is live, and a forked worker streams
|
| 758 |
+
**disk → pinned host memory → VRAM**.
|
| 759 |
+
|
| 760 |
+
HF's guidance, quoted: **models must be placed on cuda at the root module level.**
|
| 761 |
+
|
| 762 |
+
**Corollary: do NOT lazy-load models inside the decorated function.** HF documents that
|
| 763 |
+
placements are optimised for startup; lazy loading defeats the packer entirely.
|
| 764 |
+
|
| 765 |
+
### 8.4 `enable_model_cpu_offload()` must NOT be used
|
| 766 |
|
| 767 |
+
This supersedes an earlier draft of this plan, which called it mandatory. It is the opposite.
|
| 768 |
|
| 769 |
+
`enable_model_cpu_offload()` and `enable_sequential_cpu_offload()` are **counterproductive on
|
| 770 |
+
ZeroGPU**, for a specific mechanical reason: any submodule accelerate parks on CPU is **never
|
| 771 |
+
registered for packing**. Consequently it —
|
| 772 |
|
| 773 |
+
- stays resident in **main-process RAM** (which is our tightest budget, §2),
|
| 774 |
+
- is **fork-inherited by every worker**, multiplying that cost, and
|
| 775 |
+
- is **re-copied host→device on every hook fire**, on top of the packer's own streaming.
|
| 776 |
|
| 777 |
+
So we pay RAM we do not have, to duplicate work the platform already does better. The
|
| 778 |
+
`model_cpu_offload_seq` string recorded in §5.7 is upstream metadata we do not act on.
|
| 779 |
|
| 780 |
+
### 8.5 Never return CUDA tensors across the `@spaces.GPU` boundary
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 781 |
|
| 782 |
+
Returning a CUDA tensor from a `@spaces.GPU` function makes unpickling in the main process
|
| 783 |
+
trigger a **blocked lazy CUDA init**, and **the call hangs** — no error, no timeout message, a
|
| 784 |
+
dead request.
|
|
|
|
|
|
|
| 785 |
|
| 786 |
+
**Rule: return file paths, or `.cpu()` tensors. Always.** `zerogpu.py` owns this boundary and it
|
| 787 |
+
is a test case.
|
| 788 |
+
|
| 789 |
+
### 8.6 Tier and quota
|
| 790 |
+
|
| 791 |
+
We declare `size="xlarge"` **explicitly** rather than being silently promoted.
|
| 792 |
+
|
| 793 |
+
Packed bf16 weights (67.2 GiB) exceed `xlarge_threshold = 42.00 GiB`, so **ZeroGPU auto-promotes
|
| 794 |
+
us to `xlarge` and doubles quota cost with no code change** (§2.6). Declaring it explicitly means
|
| 795 |
+
the cost is visible in the source rather than being a surprise in the quota dashboard.
|
| 796 |
+
|
| 797 |
+
**Quota arithmetic:** PRO gives 2400 s/day. At `duration=75` on `xlarge` (2×), each run is
|
| 798 |
+
charged ~150 s → **~16 runs/day**. Dropping under 42.00 GiB via quantization would return us to
|
| 799 |
+
`large` at 1× — that is the entire quota argument for §2.6's fallback, and it is worth
|
| 800 |
+
approximately double the runs.
|
| 801 |
+
|
| 802 |
+
Two further traps from §5.8: the hidden **1.5× `duration_factor` on `large`** (so `duration=60`
|
| 803 |
+
schedules as 90 s; `xlarge` skips the factor but doubles the request instead), and the fact that
|
| 804 |
+
the **maximum duration is a server-side, tier-dependent, unpublished cap** — discover it, never
|
| 805 |
+
hardcode it. ~120 s is the empirical safe ceiling; the official Lightricks LTX Space ships
|
| 806 |
+
`@spaces.GPU(duration=75, size="xlarge")`.
|
| 807 |
+
|
| 808 |
+
### 8.7 Local (Apple Silicon / MPS) is a different regime
|
| 809 |
+
|
| 810 |
+
MPS has no ZeroGPU packer, no `spaces` decorator semantics, and unified memory changes the
|
| 811 |
+
arithmetic entirely. Device strategy is selected in `config.py`, the `spaces` import is guarded,
|
| 812 |
+
and the local path is expected to be **capability-limited rather than identical**. Do not assume
|
| 813 |
+
a code path that works on ZeroGPU works on MPS, or vice versa.
|
| 814 |
+
|
| 815 |
+
---
|
| 816 |
+
|
| 817 |
+
## 9. Testing strategy
|
| 818 |
+
|
| 819 |
+
We run an **H3 TDD harness**. The gate is:
|
| 820 |
|
| 821 |
```
|
| 822 |
+
.venv/bin/python -m pytest -q
|
| 823 |
```
|
| 824 |
|
| 825 |
+
**Ownership:** a separate teammate owns `tests/**`. The implementer does not create or edit
|
| 826 |
+
anything under `tests/`. This plan's job is to make the units *testable*.
|
|
|
|
|
|
|
| 827 |
|
| 828 |
+
### 9.1 Testable without a GPU, without weights, without network
|
|
|
|
| 829 |
|
| 830 |
+
| unit | how |
|
| 831 |
+
|---|---|
|
| 832 |
+
| `constraints.py` | pure functions over ints — dimension snapping, frame-count validation (with an injected compression ratio), CRF selection from a stubbed `model_type`, sigma schedule shape |
|
| 833 |
+
| `model_files.py` | data assertions: every entry has repo/revision/subfolder/filename; no duplicates; the §2.3 subset sums to 72.185; the dtype switch enumerates the three §2.6 configs |
|
| 834 |
+
| `preflight.py` | inject fake free-RAM/free-disk numbers → assert a specific error naming both numbers; assert the diffusers-version check fires on a wrong version |
|
| 835 |
+
| `lora/inspect.py` | **synthesised safetensors headers** — small fixtures, no real LoRA (this is why §4.3 stage 2 is header-only) |
|
| 836 |
+
| `lora/validate.py` | stages 1–6 against crafted good/bad headers, including the wrong-architecture case |
|
| 837 |
+
| `modes.py` | registry completeness (11 modes over 4 workflows); each mode's kwarg builder produces expected kwargs |
|
| 838 |
+
| `errors.py` | every error type's message contains the actionable detail (filename, URL, found-prefixes, measured numbers) |
|
| 839 |
+
| `conditioning.py` | pixel→latent index conversion (§5.7 #1); out-of-range raises (§5.7 #2) |
|
| 840 |
+
| `outputs.py` | 20 GB oldest-first eviction over a temp dir |
|
| 841 |
+
| `generate.py` | stub pipeline returning nothing → assert a specific exception, **not** an empty path |
|
| 842 |
+
| `zerogpu.py` | the return-boundary rejects/converts CUDA-tensor returns; `duration=` callable signature matches the decorated function |
|
| 843 |
|
| 844 |
+
### 9.2 GPU-gated tests
|
|
|
|
|
|
|
|
|
|
| 845 |
|
| 846 |
+
Anything needing real weights is marked `gpu` and deselected by default. CI and the default local
|
| 847 |
+
run must be green with zero weights on disk.
|
| 848 |
|
| 849 |
+
### 9.3 The invariant tests that matter most
|
|
|
|
| 850 |
|
| 851 |
+
Ranked by how much silent wrongness they prevent:
|
| 852 |
+
|
| 853 |
+
1. **A LoRA for the wrong architecture is rejected** — specifically at stage 7 if it survives
|
| 854 |
+
stages 1–6 (§4.3).
|
| 855 |
+
2. **A generation that produces no file raises** — never returns an empty path, and there is no
|
| 856 |
+
newest-file fallback (§5.3).
|
| 857 |
+
3. **`denormalize=False` on the diffusion decode** — the double-denormalization corruption
|
| 858 |
+
(§7.4) is silent and total.
|
| 859 |
+
4. **Pixel→latent index conversion** — off-by-8 is guaranteed without it (§5.7 #1).
|
| 860 |
+
5. **Audio/video length desync** with a supplied `audio_latents` (§5.7 #11), concrete case.
|
| 861 |
+
6. **The diffusers-version assertion fires** on 0.39.0 (§1.3) — otherwise the failure is late
|
| 862 |
+
and expensive.
|
| 863 |
+
7. **`vae.temporal_compression_ratio` tripwire** raises if upstream changes it (§5.5).
|
| 864 |
+
8. **`preload_from_hub` names only live shards** — assertable against the index JSON with no
|
| 865 |
+
download (§2.1).
|
| 866 |
+
9. **`fuse_lora` passes both components** (§4.4).
|
| 867 |
|
| 868 |
+
---
|
| 869 |
|
| 870 |
+
## 10. Resolved questions and technical risks
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 871 |
|
| 872 |
+
All six open questions from the first draft are **resolved**. What remains is one ranked risk
|
| 873 |
+
register.
|
| 874 |
|
| 875 |
+
| # | Question | Resolution |
|
| 876 |
+
|---|---|---|
|
| 877 |
+
| **OQ-1** | Pin a diffusers release, or `main`? | **RESOLVED: pin the exact SHA** `ac56fa2f…` (§1.3). A SHA is more pinned than a range, not less. Plus a startup version assertion, because 0.39.0 fails *late*. |
|
| 878 |
+
| **OQ-2** | Does `diffusion_decoder` have a consumer? | **Moot** — the SHA provides it. Both `diffusion_decoder` (0.834) and `duration_head` (0.004) stay in the subset; duration prediction ships. |
|
| 879 |
+
| **OQ-3** | Is `num_frames % 8 == 1` enforced? | **RESOLVED: no.** Upstream silently floors. We raise; the modulus is a **parameter with an `8` fallback** for pre-load validation, plus a load-time tripwire (§5.5). |
|
| 880 |
+
| **OQ-4** | Gemma-3 or Gemma-4? | **RESOLVED: both, via a union annotation** — `Gemma3ForConditionalGeneration \| Gemma4UnifiedForConditionalGeneration`. LTX-2.5 genuinely is Gemma-4; the proof is that CRF 18 is selected by `model_type in ("gemma4_unified","gemma4")`. **Derive family and CRF at runtime; hardcode neither** (§5.7 #8). |
|
| 881 |
+
| **OQ-5** | Audio length from duration or latent shape? | **RESOLVED: latent shape wins.** Upstream computes it from duration, then overrides from `audio_latents.shape` when supplied. The desync is real and directional (§5.7 #11). |
|
| 882 |
+
| **OQ-6** | Output retention bound? | **RESOLVED: 20 GB**, oldest-first, swept on startup and after each run (§2.7). |
|
| 883 |
+
|
| 884 |
+
### Technical risk register
|
| 885 |
+
|
| 886 |
+
| # | Risk | Why it is uncertain | How it gets settled |
|
| 887 |
+
|---|---|---|---|
|
| 888 |
+
| **1** | **bf16 fits ZeroGPU's CPU RAM and transient disk** | The ~68 GB RAM ceiling is a **community cgroup observation**, not official — HF publishes ZeroGPU RAM as `"-"` / "Dynamic resources". Transient disk peaks at ~144 GB against a 150 GB cap before the packer auto-prunes. | **Measurement on the Space, not argument.** §5.6's preflight turns a failure into a diagnosable message; §2.6's one-line dtype switch makes the fallback cheap. |
|
| 889 |
+
| 2 | ZeroGPU's packer handles the pipeline's tensor graph correctly | Verified for ordinary tensors; **unverified for quantised custom tensor subclasses** (bitsandbytes `Int8Params`/`Params4bit`, torchao) against the packer's `WeakTensorKeyDictionary` | only becomes live if risk #1 forces quantization — measure then |
|
| 890 |
+
| 3 | `preload_from_hub` can express our shard list within 10 entries | entry format is `repo_id file1,file2,…`, so one entry names many files; **a per-entry limit is unconfirmed** | verify before first deploy (§6.2) |
|
| 891 |
+
| 4 | Max `@spaces.GPU` duration on our tier | server-side, tier-dependent, **unpublished** | discover empirically; never hardcode (§5.8 #4) |
|
| 892 |
+
| 5 | Multishot via prompt structure produces usable results | no API support, and the enhancer system prompts actively discourage scene cuts (§3.4) | presented to users as unverified technique, not a feature |
|
| 893 |
|
| 894 |
+
### 🚨 Live blocker
|
| 895 |
|
| 896 |
+
**`Lightricks/LTX-2.5-Diffusers` gate not accepted** — §6.3. One click at
|
| 897 |
+
https://huggingface.co/Lightricks/LTX-2.5-Diffusers. Blocks everything that touches real weights.
|
|
|
|
| 898 |
|
| 899 |
---
|
| 900 |
|
| 901 |
## 11. Build order
|
| 902 |
|
| 903 |
1. ✅ **This document**, committed.
|
| 904 |
+
2. ✅ **Scaffold** — venv, `pyproject.toml`, package stubs, `.gitignore`.
|
| 905 |
+
3. ✅ **`requirements.txt`** — the §5.1 pin set, with **no `spaces` line**.
|
| 906 |
+
4. **TDD, innermost-out**, against the test teammate's failing specs:
|
| 907 |
+
`errors` → `constraints` → `model_files` → `preflight` → `lora/` → `zerogpu` → `conditioning`
|
| 908 |
+
→ `modes` → `audio` → `generate` → `outputs` → `ui`.
|
| 909 |
+
5. **Blocked on §6.3** — anything that touches real weights, and every `gpu`-marked test.
|
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Startup assertions that fail loudly before anything expensive happens.
|
| 2 |
+
|
| 3 |
+
PLAN §5.6 and §1.3. Two jobs:
|
| 4 |
+
|
| 5 |
+
1. **Version fail-fast.** ``diffusers.__version__`` must be ``"0.40.0.dev0"``
|
| 6 |
+
(the pinned SHA). Release 0.39.0 imports ``LTX2Pipeline`` fine and only fails
|
| 7 |
+
LATE, on a missing ``temporal_latent_upsampler``, after ~70 GB of download.
|
| 8 |
+
|
| 9 |
+
2. **Resource preflight.** Measure free RAM and free disk, compare against the
|
| 10 |
+
requirement computed from the ``model_files`` table, and raise an error that
|
| 11 |
+
NAMES THE NUMBERS if short -- e.g. "need ~67.2 GiB RAM for the bf16 load, saw
|
| 12 |
+
41.3 GiB". This turns an anonymous OOM, or a "Workload evicted, storage limit
|
| 13 |
+
exceeded (150G)", into a diagnosable message that names the fix (PLAN §2.6's
|
| 14 |
+
quantised fallbacks).
|
| 15 |
+
|
| 16 |
+
Both are test cases: inject fake numbers, assert the specific error.
|
| 17 |
+
"""
|
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""The ZeroGPU boundary: the ``spaces`` shim, GPU decoration, and return safety.
|
| 2 |
+
|
| 3 |
+
PLAN §8. This module exists because ZeroGPU inverts normal GPU-box practice and
|
| 4 |
+
several of its rules are silent-failure rules.
|
| 5 |
+
|
| 6 |
+
* ``import spaces`` must precede torch/diffusers/transformers, or startup dies
|
| 7 |
+
with "CUDA has been initialized before importing the spaces package". The
|
| 8 |
+
import here is guarded (``try/except ImportError``) so local dev works --
|
| 9 |
+
``requirements.txt`` deliberately does NOT pin ``spaces`` (PLAN §5.1).
|
| 10 |
+
|
| 11 |
+
* ``@spaces.GPU(...)`` must be applied STATICALLY at module import. Dynamic or
|
| 12 |
+
inner decoration fails the startup analyzer with "No @spaces.GPU function
|
| 13 |
+
detected during startup".
|
| 14 |
+
|
| 15 |
+
* **Never return CUDA tensors** across the boundary. Unpickling one in the main
|
| 16 |
+
process triggers a blocked lazy CUDA init and THE CALL HANGS -- no error, no
|
| 17 |
+
timeout, a dead request. Return file paths, or ``.cpu()``. Always. This
|
| 18 |
+
module owns that boundary and it is a test case.
|
| 19 |
+
|
| 20 |
+
* ``enable_model_cpu_offload()`` is NOT used -- see PLAN §8.4. It is
|
| 21 |
+
counterproductive here, not merely suboptimal.
|
| 22 |
+
|
| 23 |
+
* The ``duration=`` callable must accept the EXACT same positional signature as
|
| 24 |
+
the decorated function, trailing kwargs included, or it TypeErrors at
|
| 25 |
+
schedule time.
|
| 26 |
+
|
| 27 |
+
* Quota attribution rides a contextvar (``gradio.context.LocalContext.request``)
|
| 28 |
+
that does NOT cross a plain ``threading.Thread``. Without
|
| 29 |
+
``contextvars.copy_context()`` every user buckets as Space-anonymous and you
|
| 30 |
+
get "Space app has reached its GPU limit" at zero minutes used.
|
| 31 |
+
|
| 32 |
+
* Do NOT auto-retry a ZeroGPU timeout with a captured Gradio request: the
|
| 33 |
+
``X-IP-Token`` ages past TTL during attempt 1, so attempt 2 always 401s with
|
| 34 |
+
"Expired ZeroGPU proxy token" -- masquerading as the root cause.
|
| 35 |
+
"""
|
|
@@ -7,10 +7,17 @@ requires-python = ">=3.12"
|
|
| 7 |
license = { text = "MIT" }
|
| 8 |
authors = [{ name = "Mayank Gupta", email = "techfreakworm@gmail.com" }]
|
| 9 |
|
| 10 |
-
# Runtime dependencies are declared in requirements.txt
|
| 11 |
-
# Hugging Face Space build reads
|
| 12 |
-
# docs/PLAN.md §5.1
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
[project.urls]
|
| 16 |
Repository = "https://github.com/techfreakworm/ltx2.5-studio"
|
|
|
|
| 7 |
license = { text = "MIT" }
|
| 8 |
authors = [{ name = "Mayank Gupta", email = "techfreakworm@gmail.com" }]
|
| 9 |
|
| 10 |
+
# Runtime dependencies are declared in requirements.txt — that is the file the
|
| 11 |
+
# Hugging Face Space build reads, and it is the single source of truth for pins.
|
| 12 |
+
# See docs/PLAN.md §5.1, including why `spaces` is deliberately NOT pinned there.
|
| 13 |
+
|
| 14 |
+
[project.optional-dependencies]
|
| 15 |
+
# Dev tooling lives here rather than in requirements.txt so the Space image stays
|
| 16 |
+
# lean. Install with: .venv/bin/python -m pip install -e ".[dev]"
|
| 17 |
+
dev = [
|
| 18 |
+
"pytest>=9.0,<10",
|
| 19 |
+
"ruff>=0.16,<0.17",
|
| 20 |
+
]
|
| 21 |
|
| 22 |
[project.urls]
|
| 23 |
Repository = "https://github.com/techfreakworm/ltx2.5-studio"
|
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# LTX-2.5 Studio — runtime dependencies.
|
| 2 |
+
#
|
| 3 |
+
# Every direct dependency has a floor AND a ceiling. See docs/PLAN.md §5.1 for
|
| 4 |
+
# the reasoning; the short version is that the predecessor project shipped ~50
|
| 5 |
+
# unbounded requirements and got burned by a silent torch/torchaudio ABI break.
|
| 6 |
+
#
|
| 7 |
+
# Dev/test tooling (pytest, ruff) is NOT here — it lives in pyproject.toml's
|
| 8 |
+
# [project.optional-dependencies] dev extra, so the Space image stays lean.
|
| 9 |
+
|
| 10 |
+
# --- NOT PINNED, DELIBERATELY -------------------------------------------------
|
| 11 |
+
# `spaces` is intentionally absent. The Spaces platform pins its own version
|
| 12 |
+
# (live Spaces report 0.49.3 / 0.50.0 / 0.50.4 while PyPI is at 0.51.1), and a
|
| 13 |
+
# conflicting pin here breaks pip resolution at build time. The 2.3 repo's own
|
| 14 |
+
# AGENTS.md:19 states this rule and its requirements.txt:2 then violated it.
|
| 15 |
+
# The import is guarded in ltx25_studio/zerogpu.py so local dev works without it.
|
| 16 |
+
# ------------------------------------------------------------------------------
|
| 17 |
+
|
| 18 |
+
# --- torch trio: pinned exactly, and pinned TOGETHER --------------------------
|
| 19 |
+
# ZeroGPU supports exactly torch 2.8.0 / 2.9.1 / 2.10.0 / 2.11.0 (CUDA 12.8;
|
| 20 |
+
# sm_120 Blackwell needs cu128+). 2.11.0 is the ceiling.
|
| 21 |
+
# Trap: torchvision 0.28.0 hard-declares torch==2.13.0, so letting torchvision
|
| 22 |
+
# float silently drags torch past the ceiling. torchaudio must match torch
|
| 23 |
+
# exactly or the C++ ABI mismatch crashes the whole audio path on import.
|
| 24 |
+
torch==2.11.0
|
| 25 |
+
torchvision==0.26.0
|
| 26 |
+
torchaudio==2.11.0
|
| 27 |
+
|
| 28 |
+
# --- diffusers: exact SHA, PEP 508 direct reference (not #egg=) ---------------
|
| 29 |
+
# LTX-2.5 is not in any release. 0.39.0 imports LTX2Pipeline fine and then fails
|
| 30 |
+
# LATE on a missing temporal_latent_upsampler — after ~70 GB of download. This
|
| 31 |
+
# SHA is HEAD at 2026-08-19T12:11:04Z and contains PR #14447 (LTX-2.5 merge,
|
| 32 |
+
# 2026-08-11) plus #14453 and #14462. ltx25_studio/preflight.py asserts
|
| 33 |
+
# diffusers.__version__ == "0.40.0.dev0" at startup so a wrong revision fails
|
| 34 |
+
# immediately instead of expensively. See PLAN §1.3.
|
| 35 |
+
diffusers @ git+https://github.com/huggingface/diffusers@ac56fa2f25c1c15b4cbaa1e0b3255e9bc38c06dc
|
| 36 |
+
|
| 37 |
+
# --- model stack --------------------------------------------------------------
|
| 38 |
+
transformers>=5.15.1,<6
|
| 39 |
+
accelerate>=1.14.0,<2
|
| 40 |
+
peft>=0.20.0,<0.21 # LoRA backend; USE_PEFT_BACKEND gates load_lora_weights
|
| 41 |
+
safetensors>=0.8.0,<0.9
|
| 42 |
+
huggingface_hub>=1.28.0,<2
|
| 43 |
+
hf_xet>=1.6.0,<2
|
| 44 |
+
|
| 45 |
+
# --- UI -----------------------------------------------------------------------
|
| 46 |
+
# gradio 5.x is END OF LIFE (last stable 5.50.0, 2025-11-21). The Space's
|
| 47 |
+
# frontmatter `sdk_version` is authoritative for the installed gradio; this range
|
| 48 |
+
# must stay compatible with it (currently 6.24.0). Breaking changes from 5.x that
|
| 49 |
+
# affect us: gr.Video subtitle-tuple return removed; show_download_button /
|
| 50 |
+
# show_share_button -> buttons=[...]; gr.Audio show_controls ->
|
| 51 |
+
# show_recording_waveform; min_length / max_length -> event-listener validators.
|
| 52 |
+
gradio>=6.24.0,<7
|
| 53 |
+
|
| 54 |
+
# --- media I/O ----------------------------------------------------------------
|
| 55 |
+
# av (PyAV) is not optional: image-conditioning CRF re-compression is applied by
|
| 56 |
+
# default and requires it. See PLAN §5.7 #7.
|
| 57 |
+
imageio>=2.37.4,<3
|
| 58 |
+
imageio-ffmpeg>=0.6.0,<0.7
|
| 59 |
+
av>=18.1.0,<19
|
| 60 |
+
|
| 61 |
+
# --- numerics -----------------------------------------------------------------
|
| 62 |
+
# numpy 2, deliberately. Do NOT pin <2: 1.26.4 is two years EOL and nothing in
|
| 63 |
+
# this stack caps numpy. numpy 2.5.2 requires Python >=3.12, which is why the
|
| 64 |
+
# Space frontmatter must set python_version: "3.12" — the default 3.10.13 image
|
| 65 |
+
# silently resolves numpy and av DOWNWARD rather than failing.
|
| 66 |
+
numpy>=2.5.2,<3
|
| 67 |
+
pillow>=12.3.0,<13
|