Spaces:
Running on Zero
Download docs/PLAN.md from techfreakworm/LTX2.5-Studio: direct link, hf CLI and curl.
- Browser
- Download file 160 kB
-
https://huggingface.co/spaces/techfreakworm/LTX2.5-Studio/resolve/main/docs/PLAN.md
- Command line
-
hf download hf://spaces/techfreakworm/LTX2.5-Studio/docs/PLAN.md
-
curl -L -o PLAN.md https://huggingface.co/spaces/techfreakworm/LTX2.5-Studio/resolve/main/docs/PLAN.md
A newer version of the Gradio SDK is available: 6.32.0
LTX-2.5 Studio β Design Plan
Status (2026-11-16): implemented. After the 2026-11-16 fixes, all eleven modes are
verified on real weights on a Mac (Apple silicon); none has been re-run on the Space, which still
runs the 2026-08-26 build. What has been verified, and how, is in
SPACE_MODE_MATRIX.md Β§10. (Originally: "implemented; not yet run on a
GPU. The matrix in docs/TEST_PLAN.md is what proves it works.")
Branch: soma/ltx25-studio Β· Remote: git@github.com:techfreakworm/ltx2.5-studio.git
Target: private Hugging Face Space techfreakworm/LTX2.5-Studio (gradio SDK, ZeroGPU).
β Status superseded by events (annotated 2026-08-25). "Not yet run on a GPU" was true when written. Since then: all eleven modes ran on hinode (704Γ480Γ121 baseline, stage 2 at 1408Γ960, duration head f=97, LoRA both directions, mel round-trip β
OPERATOR_HANDOFF.mdΒ§2) and all eleven execute on the live ZeroGPU Space at sha287ba9c, 256Γ256Γ9 (SPACE_MODE_MATRIX.md). (Corrected 2026-11-16: this said "generate". Every Space output up toff142d9was colour static, because the int8 build quantised the conditioning path; fixed ateb1aa9f.) The line above is kept because this document records how the build was decided; where the project stands is the handoff's job. Where a decision below was overtaken β the bf16 default (Β§2.6), the ~68 GB ceiling (Β§2), the explicitsize=(Β§8.6), the disk routes (Β§7.4, Β§10 risk 1) β a dated annotation sits on it.
β οΈ Read this before you trust the test suite
This suite was green at 1102 passing tests while five blockers were live in the tree. At that moment the app could not reach a GPU, could not apply a LoRA, and reported every successful run in its three flagship modes as an internal error. Nothing was failing.
The ~700 tests added since are almost entirely seam and reachability tests, not more unit coverage β because more tests of the original shape would have found none of it. Β§9.1.1 explains the pattern and the one check that catches it.
If you are about to trust a green gate here the way we did: that is the mistake this document exists to warn you out of.
This document is the contract for the build. Every section is either a decision (settled, build to it) or a risk (Β§10, needs measurement not argument). Where a claim was verified against real source or real repo metadata, the evidence is cited inline so a future reader can re-check it instead of trusting it.
Prime directive: this app must not repeat the failure modes of its predecessor
(ltx2.3-AIO-generator). Those were not model failures. They were packaging, pinning, and
silent-failure failures. Β§5 and Β§8 are the sections the build will be judged on.
A note on how to read this document. Several statements here contradict things that are
true of other diffusion apps, or true of LTX-2.5 on a normal GPU box but false on ZeroGPU. The
biggest is Β§8: enable_model_cpu_offload() must not be used. If a future reader's instinct
says otherwise, Β§8.4 explains why the instinct is wrong here.
1. Decision 1 β Integration path: Diffusers, pinned to an exact SHA
We build on diffusers, loading Lightricks/LTX-2.5-Diffusers.
1.1 Options considered
| Path | What it is | Verdict | |
|---|---|---|---|
| A | ltx-pipelines |
Lightricks' own repo, installed via uv sync of a git workspace |
Rejected |
| B | ComfyUI runtime | Vendor ComfyUI + custom nodes, drive it via API graphs | Rejected |
| C | diffusers |
diffusers.pipelines.ltx2, weights from Lightricks/LTX-2.5-Diffusers |
CHOSEN |
1.2 Why C
Path A is a heavy, fragile install for a Space. It needs uv sync of a git workspace,
CUDA >= 12.7, torch ~= 2.7. A Space build that resolves a git workspace at image-build time is
a build we cannot reproduce or pin.
Path B is the documented cause of the fragility we are told to avoid. The 2.3 project's
requirements.txt vendors an entire ComfyUI runtime β ~50 dependencies, nearly all unpinned:
spandrel, torchsde, kornia, PyOpenGL, glfw, SQLAlchemy, alembic, Mako, blake3,
comfyui-frontend-package, comfyui-workflow-templates, comfyui-embedded-docs, plus a
hand-maintained block of custom-node deps we discovered are required but aren't auto-installed
by their respective nodes (gguf, imageio_ffmpeg, opencv-python, matplotlib,
scikit-image, onnxruntime, yt-dlp). That comment block is the evidence: the dependency
surface was discovered by breakage, not by design. Diffusers replaces all of it with 19 direct
deps, each pinned to one version (Β§5.1), and zero custom-node surface.
Decisive coupling to Β§2. The *-comfy-int8-convrot quantised weights are ComfyUI-only
β the model card says so. Choosing A or C means those 36.9 GB of int8 weights are unusable to
us. Path choice and disk budget are one decision, not two.
1.3 The pinned revision β and why a release tag will not do
diffusers @ git+https://github.com/huggingface/diffusers@ac56fa2f25c1c15b4cbaa1e0b3255e9bc38c06dc
HEAD at 2026-08-19T12:11:04Z. Contains the LTX-2.5 merge (PR #14447, 2026-08-11) plus #14453
and #14462. PEP 508 direct reference β not the legacy #egg= form.
A SHA is more pinned than a version range, not less. It is byte-for-byte reproducible and cannot drift. It costs a git clone at build time and manual upgrades; both are acceptable.
The latest release (0.39.0) does not have 2.5, and it fails LATE. LTX2Pipeline imports
fine on 0.39.0 β so the app starts, the UI renders, weights begin downloading β and only then
does it fail on a missing temporal_latent_upsampler. A failure that happens after 70 GB of
download is the worst kind.
Confirmed by the repo itself. model_index.json on Lightricks/LTX-2.5-Diffusers
declares _diffusers_version: 0.40.0.dev0. The weights and the code agree on the dev version;
our SHA pin is not a guess.
Therefore: a startup fail-fast assertion.
assert diffusers.__version__ == "0.40.0.dev0", (
f"LTX-2.5 needs the pinned diffusers SHA (reports 0.40.0.dev0); got {diffusers.__version__}. "
"See docs/PLAN.md Β§1.3."
)
This converts a late, confusing, expensive failure into an immediate, obvious one. It lives in
preflight.py (Β§7.1) and it is a test case (Β§9.3).
1.4 The single-file escape hatch: investigated and REJECTED
diffusers/loaders/single_file_utils.py does contain an LTX-2 route:
CHECKPOINT_KEY_NAMES["ltx2"] β model_type = "ltx2-dev" β config repo
{"pretrained_model_name_or_path": "Lightricks/LTX-2"}. Note the repo: LTX-2, i.e. 2.0,
not 2.5. The converters it ships are only convert_ltx2_transformer_to_diffusers,
convert_ltx2_vae_to_diffusers and convert_ltx2_audio_vae_to_diffusers β all keyed on
LTX_2_0_* rename dicts.
There is no single-file converter for 2.5's connectors, diffusion_decoder, vocoder,
duration_head or latent_upsampler. Even a successful load would give us a pipeline missing
the components that make 2.5 what it is. Settled; do not revisit without new upstream
converters.
β CORRECTED 2026-08-25 β the observation is right, the conclusion is wrong
from_single_fileIS implemented for LTX2 at the pinned SHA. π Verified by readingac56fa2f:
LTX2VideoTransformer3DModelinheritsFromOriginalModelMixinβmodels/transformers/transformer_ltx2.py:1081.- It is registered in
SINGLE_FILE_LOADABLE_CLASSESatloaders/single_file_model.py:207, withAutoencoderKLLTX2Video(211) andAutoencoderKLLTX2Audio(215).The
LTX_2_0_*naming above is accurate and is not a blocker: those are local rename dicts insideconvert_ltx2_transformer_to_diffusers(single_file_utils.py:4060), not a version gate. What survives from this section is the narrower half β there is no single-file route for 2.5'sconnectors,diffusion_decoder,vocoder,duration_headorlatent_upsampler, so a whole-pipeline single-file load is unavailable. We would never want one: the Diffusers pack supplies those, and the only thing worth swapping is the transformer, which is loadable this way.The real reason the vendor quantised weights are rejected is FORMAT, and it is a worse reason than the one written here β π verified by range-reading the real safetensors headers over the Hub:
ltx-2.5-22b-distilled-transformer-comfy-int8-convrot.safetensorscarries.comfy_quantU8[72]per-Linear ComfyUI metadata.convert_ltx2_transformer_to_diffusersknows nothing about it, so the file would load silently as garbage rather than raise β the single worst failure mode available, and precisely the one this project has just spent three weeks failing to detect (see the invalidation blocks inSPACE_MODE_MATRIX.md/SPACE_SESSION_2026-08-25.md).ltx-2.5-22b-distilled-transformer-nvfp4.safetensorsis FP4-packed with halved inner dims and needs NVIDIA ModelOpt to unpack. It is not a safetensors file diffusers can interpret.Still rejected β for these reasons, not the stated one. Do not re-derive the rejection from the
LTX_2_0_*naming; that argument is void.
2. Decision 2 β Disk, RAM and dtype budget
Three separate resource ceilings, and the tightest one is not the obvious one.
| resource | ceiling | our load | margin |
|---|---|---|---|
VRAM (xlarge) |
~95 GiB | 67.2 GiB weights | ~28 GiB for activations β comfortable |
| CPU RAM | ~68 GB (community cgroup observation, not an official figure β HF publishes ZeroGPU RAM as "-" / "Dynamic resources") |
~72 GB | marginal β risk #1 |
| Ephemeral disk | 150 GB, wiped on every restart/rebuild | 152β156 GB peak (Β§2.5) | EXCEEDS THE CAP β settled, not a risk |
VRAM was never the problem. The problems are CPU RAM and the startup disk peak β and the second is now settled rather than uncertain: it does not fit (Β§2.5).
β The CPU RAM row is FALSIFIED (annotated 2026-08-25). The ~68 GB figure was inherited β a community cgroup observation, never ours β and the platform measured past it: a load peaked at 87.13 GiB = 93.56 GB RSS on ZeroGPU and was not killed, so the ceiling is above 93.56 GB and "marginal β risk #1" died with the number (
SPACE_GENERATION_FINDING.mdΒ§6). The disk row's verdict stood for bf16 β and was then routed around: the deployed Space ships int8 (ZEROGPU_DISK.mdOption B) and fits at 82.2 GB of 150 [OK].β Corrected 2026-11-16, two rows. (1) "82.2 GB of 150 [OK]" is a reading taken before ZeroGPU packs (packed term 0.0). On the 2026-11-13 boot the platform's progress bar read
40.6Gpacked; the disk peak has never been measured. (2) The VRAM row assumedxlarge. Until 2026-08-26 the Space ran on the 47.375 GiBlargepartition (autonever promotes, Β§2.6), where stage 2 at 704Γ480Γ49 β 55.906 GiB reserved β could not fit: onlarge, VRAM was the problem. Onxlarge(94.971 GiB, now the code's default) that is the largest peak measured, with ~39 GiB to spare.
2.1 Trap A β duplicate shard sets
Two folders ship the same weights twice, in two different shardings:
| folder | set 1 | set 2 | note |
|---|---|---|---|
transformer/ |
4 shards: 9.932 + 9.919 + 9.986 + 8.138 = 37.975 GB | 8 shards: 4.991 + 4.944 + 4.994 + 4.928 + 4.977 + 4.995 + 4.994 + 3.163 = 37.986 GB | only one diffusion_pytorch_model.safetensors.index.json |
connectors/ |
single file: 6.344 GB | 2 shards: 5.000 + 1.344 = 6.344 GB | same |
Resolved against the actual index files β which set is live is now measured, not inferred:
| folder | LIVE set | DEAD weight |
|---|---|---|
transformer/ |
the 4-shard set β 37.976 GB | 37.986 GB |
connectors/ |
the 2-shard set β 6.345 GB | 6.344 GB |
text_encoder/ |
all 5 shards β 23.920 GB | none |
transformer_full/ |
4 shards (excluded entirely, Β§2.2) | none |
| total avoidable waste: 44.33 GB |
Note the direction on connectors/: the 2-shard set is live and the single 6.344 GB file is
the dead one β the opposite of the naive guess that a single file must be canonical.
A naive snapshot_download of those folders pays the full 44.33 GB. from_pretrained reads the
index JSON and fetches only the referenced shards.
Binding consequences: (1) we rely on the index, never hand-rolling a snapshot_download of
transformer/; (2) any preload_from_hub: entry must name only the live shards. A wrong
list here does not fail fast β it fails after tens of GB, and on a 150 GB cap it evicts us.
2.2 Trap B β transformer_full/ is the DEV transformer
A further 37.975 GB, the dev (non-distilled) transformer. We ship the distilled 8-step path, so we do not ship it. Documented as opt-in (Β§2.5), not deleted from our mental model.
2.3 The bf16 subset we ship β 37 files, 72.185 GB (67.23 GiB)
Measured from the Hub's own byte counts, not estimated. The GB column is decimal
(bytes / 1e9), matching how the Hub reports sizes.
| component | GB | loaded via | why |
|---|---|---|---|
transformer/ |
37.9762 | index | the model (4 live shards + config + index) |
text_encoder/ |
23.9195 | index | Gemma4UnifiedForConditionalGeneration (Β§10 OQ-4) |
connectors/ |
6.3445 | index | text β transformer conditioning; also a LoRA target |
vae/ |
1.4522 | index | video VAE: encodes conditions, and decodes wherever the diffusion decoder cannot run (Β§7.4) |
latent_upsampler/ |
0.9957 | subfolder | stage-2 spatial upscale |
diffusion_decoder/ |
0.8343 | index | the native 2.5 decoder β Β§7.4 |
temporal_latent_upsampler/ |
0.2619 | subfolder | stage-2 temporal upscale |
vocoder/ |
0.2583 | index | LTX2VocoderWithBWE β not plain LTX2Vocoder |
audio_vae/ |
0.1065 | index | audio VAE (mel β latent) |
tokenizer/ |
0.0322 | index | GemmaTokenizer |
duration_head/ |
0.0038 | index | duration prediction |
scheduler/ |
~0 | index | FlowMatchEulerDiscreteScheduler -- config-only, but every pipeline constructor needs it, and it uses scheduler_config.json, not config.json |
model_index.json (repo root) |
~0 | β | the component manifest |
| TOTAL | 72.185 | = 67.23 GiB, 44% of the repo β we skip 92.005 GB |
β Corrected 2026-11-17: the Space loads 35 of these files, 71.92 GB.
temporal_latent_upsampler/(0.2619 GB) is no longer fetched: nothing loads it, since the stage graph wires only the spatiallatent_upsampler. Every file's size is recorded inmodel_files.py, and the image brings the set (preload_from_hub, Β§6.2).
model_index.json, verbatim β this is the authority for what the pipeline expects:
_class_name: LTX2Pipeline
_diffusers_version: 0.40.0.dev0
audio_vae: ['diffusers', 'AutoencoderKLLTX2Audio']
connectors: ['ltx2', 'LTX2TextConnectors']
diffusion_decoder: ['diffusers', 'LTX2VideoDiffusionDecoderModel']
duration_head: ['ltx2', 'LTX2DurationHead']
processor: [None, None]
prompt_enhancer: [None, None]
scheduler: ['diffusers', 'FlowMatchEulerDiscreteScheduler']
text_encoder: ['transformers', 'Gemma4UnifiedForConditionalGeneration']
tokenizer: ['transformers', 'GemmaTokenizer']
transformer: ['diffusers', 'LTX2VideoTransformer3DModel']
vae: ['diffusers', 'AutoencoderKLLTX2Video']
vocoder: ['ltx2', 'LTX2VocoderWithBWE']
Two loading mechanisms, and model_files.py must model the difference.
latent_upsampler and temporal_latent_upsampler are not in model_index.json. They are
loaded separately by subfolder, exactly as the model card does:
LTX2LatentUpsamplerModel.from_pretrained(MODEL_ID, subfolder="latent_upsampler")
Index-components and subfolder-components are fetched by different calls. A resolver that assumes one mechanism silently misses two components β and they are precisely the two the stage-2 quality path needs.
2.4 What we exclude, and why
| excluded | GB | reason |
|---|---|---|
transformer_full/ |
37.977 | the DEV transformer; not needed for the distilled 8-step path |
| duplicate shard sets (transformer 8-shard + connectors single-file) | 44.330 | Trap A β same weights, different sharding |
ltx-2.5-22b-distilled-lora-450-bf16.safetensors (repo root) |
9.699 | the distilled LoRA for DEV workflows; useless when we ship the distilled transformer |
Arithmetic check: 72.185 + 37.977 + 44.330 + 9.699 = 164.191 GB, closing against the repo's stated 164.19 GB across 56 files (37 shipped, 19 skipped).
Precision note, because it has already bitten once. Summing figures that were each
truncated to three decimals gives 37.975 for the transformer; the exact byte count
(37,976,221,088 B) gives 37.9762. The difference is 220 KB and irrelevant to the disk budget,
but it is larger than a Β±0.001 assertion tolerance. model_files.py carries the byte-exact
values, so any test asserting on sizes should use a tolerance of Β±0.01, not Β±0.001. The accounting is complete; there is no
unexplained remainder.
2.5 The startup disk peak β negative margin, not tight margin
Authoritative section for this arithmetic. The mechanism is Β§8.3; the consequence for the deploy is Β§7.4. If those and this ever disagree, this one is wrong β do not reconcile by editing the other two.
Provenance: verified against
spaces0.51.1 source (zero/torch/patching.py,zero/configs.json), not inferred.
Startup holds two copies: the HF cache (72.19 GB) and the packed copy ZeroGPU writes to
`/.zerogpu/tensors(~72.19 GB). The 2.3 project was evicted with exactly this:"Workload evicted, storage limit exceeded (150G)"`.
The packer does auto-prune the HF blobs β ZEROGPU_MMAP_AUTOPRUNE_PATTERN =
~/.cache/huggingface/hub/models--*/blobs/*, logging "Cleaned NGB of tensor files ... after packing". That is verified, and it does not save us, because the unlink runs strictly
after _pack() returns:
| GB | |
|---|---|
| HF cache snapshot | 72.19 |
ZeroGPU packed copy (one pipeline, post-from_pipe) |
72.19 |
| peak, before the prune can run | 144.37 |
| the image itself β torch + CUDA | +8β12 |
| total against a 150 GB cap | 152β156 |
The margin is negative, not tight. An earlier draft of this section read it as "6 GB of margin is not much" β that figure never counted the image, which is not optional and is present for the whole window. The prune is real; it simply happens after the peak it would have to prevent.
Steady state afterwards is fine at ~72 GB. It is the peak that evicts us, not the residency, and the two have different fixes β which is why they are stated separately here.
Β§5.6's preflight measures free disk and reports a number, so an eviction arrives as a diagnosable message rather than anonymously.
Remaining headroom after packing: 150 β 72.19 β 77.8 GB for the pip environment, user LoRA
uploads and outputs. transformer_full/ (37.98 GB) fits in that β which is why it is opt-in
rather than impossible β but enabling it leaves ~40 GB and makes output eviction urgent.
2.6 dtype and quantization β default bf16, quantization is a documented fallback
Packed weight size decides which ZeroGPU tier we land on, and the tier decides quota cost. The
threshold is xlarge_threshold = 42.00 GiB: exceed it and ZeroGPU silently auto-promotes to
xlarge and doubles quota cost, with no code change.
β The premise in bold is FALSE, and finding that out cost this project half its GPU (2026-08-27). Crossing the threshold sets
_default_gpu_size, which is read by exactly one function βget_duration_secondsβ and feeds exactly one branch: theduration_factormultiplier. It never reaches thegpuSizethe scheduler allocates on, which is sent only when a size is passed explicitly. β Measured from inside the worker:LTX25_ZEROGPU_SIZE=autoran the Space onlargeat 47.375 GiB; settingxlargegives 94.971 GiB. The tier argument below is still the right argument β it is just an argument for setting the variable, never for trusting the packer to set it.MIG_ALLOCATOR_SIGHTINGS.mdΒ§1.
| config | approx total | packed vs 42.00 GiB | tier | quota | verdict |
|---|---|---|---|---|---|
| bf16 everything | 72.19 GB (67.2 GiB) | over | xlarge |
2Γ | DEFAULT β documented by the model card, matches the official Lightricks Space pattern; RAM/disk marginal |
| fp8 transformer ( |
over | xlarge |
2Γ | comfortable RAM/disk, no quota win | |
| fp8 transformer + int8 TE | under | large |
1Γ | best quota economics, highest accuracy risk |
β οΈ The quantized checkpoints are not available to us at all β the blocker is FORMAT, not
size. The int8 / nvfp4 files exist only in Lightricks/LTX-2.5, the native repo; there are
none in the Diffusers pack. And our single-file path cannot load them:
CHECKPOINT_KEY_NAMES["ltx2"] β ltx2-dev β config repo Lightricks/LTX-2 (that is 2.0),
with every converter named LTX_2_0_* and nothing for 2.5's connectors, diffusion_decoder,
vocoder, duration_head or latent_upsampler (Β§1.4). So no plan should be written around the
quantized files being reachable β they are a different repo in a format we have no converter for.
β CORRECTED 2026-08-25: the verdict holds; the reason given here does not.
from_single_fileis wired forLTX2VideoTransformer3DModelat the pinned SHA (loaders/single_file_model.py:207,FromOriginalModelMixinattransformer_ltx2.py:1081) π, so "our single-file path cannot load them" is false. The real blockers, verified by range-reading the Hub headers: thecomfy-int8-convrotfile carries.comfy_quantU8[72]per-Linear ComfyUI metadata and would load silently as garbage rather than raise, andnvfp4is FP4-packed with halved inner dims and needs ModelOpt. Β§1.4's correction carries the detail.
Why bf16 is the default and not the quantised config. Shipping quantised by default would trade a known uncertainty for an unverified one:
- The RAM ceiling (~68 GB) is a community cgroup observation, not an official number. It is uncertain, but it is uncertainty about a configuration that upstream documents and that a production Space already runs.
Quantization carries its own unquantified risk: bitsandbytes and torchao use custom tensor subclasses and the packer's handling of them is unverified.This was wrong.spaces/zero/torch/bitsandbytes.pyis a 43-line version check that no-ops for bnb β₯ 0.46.0, wired into the packing path atpatching.py:328/355/445β so modern bitsandbytes needs no special handling at all. There is no equivalent module for torchao, and that asymmetry is what actually selects the library. Verified againstspaces0.51.1. Seedocs/ZEROGPU_DISK.md.
So bf16 ships and size="xlarge" is declared explicitly rather than being silently promoted
(Β§8.5).
β Both halves of that sentence were overtaken (annotated 2026-08-25), and the section's default inverted with them. The deployed Space ships int8, not bf16 β
components_to_quantize=["transformer", "text_encoder"]viaPipelineQuantizationConfig+ bitsandbytes (text encoder widened at02fe943) β landed as the fix for the eviction this section's bf16 default ran into, and it is what makes the Space fit disk (82.2/150 [OK]). Andsize=was never declared: every deploy has runLTX25_ZEROGPU_SIZE=autoβsize=None, so ZeroGPU promoted toxlargeby inference (packed ~51.1 GB int8 > 42.00 GiB) β see Β§8.6's annotation andOPERATOR_HANDOFF.mdΒ§4 gap 0. Quantization is therefore no longer "a documented fallback" β it is the configured, deployed route; the reasoning above stands as the record of why bf16 was chosen first, and its load-bearing premise (the ~68 GB ceiling, "uncertain but documented") is falsified besides (SPACE_GENERATION_FINDING.mdΒ§6). The Β§6.2.1-class route list at the end of this section omitted the route actually taken.β Corrected 2026-11-16. "ZeroGPU promoted to
xlargeby inference" is wrong:autonever promotes (the block above this section's table), so the Space ran onlargeuntilLTX25_ZEROGPU_SIZE=xlargewas set as a Space variable on 2026-08-26. The code now defaults toxlarge(config.DEFAULT_ZEROGPU_SIZE), sosize="xlarge"is declared explicitly after all β by default, not by a variable. "Packed ~51.1 GB": the platform's progress bar read40.6Gon the 2026-11-13 boot. "82.2/150 [OK]" is a reading taken before packing.
β οΈ There is no precision switch, and the reason is a correction rather than a decision.
LTX25_TRANSFORMER_PRECISION and LTX25_TEXT_ENCODER_PRECISION existed, resolved cleanly, and
were wired to nothing β pipelines.get_pipeline never read the config, so the load was
always bf16. Worse, expected_weights_gb applied the precision factor anyway, so the startup
report stated 53.20 GB for a load that was still 72.19 GB. It did not fail silently: it
spoke, and the number was wrong in the direction that misdirects the next decision. It was one
approval away from being reached for to fix a disk eviction, twice.
An advertised lever attached to nothing is worse than no lever at all, so it was removed
rather than half-wired. Wiring it properly needs bitsandbytes or torchao β neither pinned,
neither verifiable without a GPU. The reason originally given here β that the packer's handling
of quantised tensors was unknown β was false; see above. The lever is gone because it was
wired to nothing, which is reason enough on its own. Quantization remains a documented
option; it
is not a configured one, and the difference is now visible in the code rather than implied.
β Corrected 2026-11-16: there is a precision switch now, and it is wired.
LTX25_QUANTIZATION=auto(the default: int8 on ZeroGPU, bf16 everywhere else) |bf16|int8, resolved inconfig.pyanddevice.py; int8 is bitsandbytes (pinned, Β§5.1) and is refused at startup off CUDA. The two removed variables stay removed.
The routes actually available under disk pressure are persistent storage, a public Space, or smaller checkpoints β see Β§6.2.1.
β οΈ Flipping precision is a REBUILD, not a runtime switch. Preloading is all-or-nothing
(Β§6.2): preloaded files are owned by the build user and read-only to the runtime, so a quantised
variant needing different files cannot be fetched on demand into that cache. If a future
precision setting changes the file list, the Space must be rebuilt with a new preload_from_hub
block. Setting the variable alone changes how the existing weights are loaded -- nothing more. This is technical risk #1 (Β§10), and it is to be settled by
measurement on the Space, not by argument. (2026-11-16: moot as built. int8 is applied at
load time to the same files, so changing LTX25_QUANTIZATION needs no new download and no
rebuild.)
2.7 Output retention β 20 GB, evicted oldest-first
outputs/ is capped at 20 GB, bounded-bytes, oldest-first eviction, as a single constant in
config.py overridable by env. Swept on startup (app.py) and after every successful
generation (generate.py), logging each eviction and surfacing it on the result's warnings.
Both call sites are deliberately non-fatal: the user's video exists and is fine, so a retention failure must not turn a good run into an error, and a bad cap must not stop the app from starting. A policy that exists but is never called prevents nothing -- which is precisely what was true here until the call sites were added.
Rationale: ~78 GB headroom minus the pip environment and a user LoRA or two. 20 GB is many generations and leaves the margin the 2.3 Space did not have when it filled its disk and went unhealthy.
(As built, 2026-11-16: the folder is LTX25_OUTPUT_DIR, outputs by default, and the writer
and the startup sweep read the same setting. Gradio serves a file a step returns only from the
working directory, the system temp dir, its own cache or launch(allowed_paths=β¦), so a folder
that resolves outside the first two is passed by app.main as the only allowed path (e7cde1a);
in Gradio 6.24 a non-empty allowed_paths replaces GRADIO_ALLOWED_PATHS for that launch. A
launcher that calls app.demo.launch() itself must pass the same. A run whose result Gradio
still cannot show ends with Generate back and a card saying so, never on "Generatingβ¦"
(55692f5).)
3. Decision 3 β Mode coverage: 11 UI modes over 4 real workflows
All 11 requested modes are reachable. They are not 11 code paths, and the plan says so plainly β describing them as independent implementations would be dishonest and would produce a much worse registry.
3.1 The four real workflows
From _workflow_map in diffusers/modular_pipelines/ltx2/modular_blocks_ltx25.py:
text2video {prompt}
image2video {image, prompt}
condition {conditions, prompt}
in_context {reference_conditions, num_frames, prompt}
Our 11 UI modes are a presentation layer over those four, plus the audio-latent input route
and the audio-only export. That is the honest description, and it is also the design that keeps
the registry small: modes.py is data over four builders, not eleven.
3.2 Three facts that shape everything
(a) One conditioning mechanism covers both image and video.
LTX2VideoCondition.frames is typed
PIL.Image.Image | list[PIL.Image.Image] | np.ndarray | torch.Tensor β "the image (or
video) to condition the video on" β and carries index and strength. Image- and
video-conditioning are the same code path with a different payload. See Β§5.7 for the
index trap, which is severe.
(b) The model is jointly audio-visual. There is no video-off switch.
The return type is LTX2PipelineOutput(frames=..., audio=...), both non-optional tensors. Every
call spends video compute and returns a video. This is the most important cost fact in the app.
(c) Audio input is not a first-class kwarg.
There is no audio= argument. Audio enters as audio_latents β upstream documents it as
"Optional pre-encoded audio latents; random noise is used when not provided" β combined with
noise_scale. prepare_audio_latents routes through _create_noised_state, whose body is
noise_scale * noise + (1 - noise_scale) * latents: img2img-style strength, where 1.0
discards the input entirely.
3.3 The mode registry
| # | Mode | Workflow | How it is reached | Cost note |
|---|---|---|---|---|
| 1 | text β video | text2video | prompt |
first-class |
| 2 | image β video | condition | one LTX2VideoCondition(frames=PIL, index=0), empty prompt |
first-class |
| 3 | image + text β video | condition | as #2 with a prompt | first-class |
| 4 | video β video | condition | LTX2VideoCondition(frames=<video>, strength<1.0) |
first-class |
| 5 | text β audio+video | text2video | #1 β the audio track is always produced | the default shape |
| 6 | image β audio+video | condition | #2 | first-class |
| 7 | image + text β audio+video | condition | #3 | first-class |
| 8 | text β audio | text2video | #1, export only the audio track | β οΈ still pays full video compute |
| 9 | video β audio | condition | #4, export only the audio track | β οΈ same |
| 10 | audio β audio | condition | audio_latents + noise_scale, export only audio |
β οΈ same |
| 11 | audio β video | condition | audio_latents + noise_scale, export only video |
β οΈ pays audio encode on top |
β Corrected 2026-11-16, rows 4, 10 and 11. One
noise_scalealso set the video's starting noise in the condition pipeline, which washed out audio β video (measured on real weights). Now (audio_conditioning.py): 11 holds the supplied audio fixed while the picture is generated to it, and exports video with that audio (exports=("audio", "video")); 10 changes the supplied audio by one of the model's own levels (0.42, 0.73, 0.91, or 1.0, which ignores it). Row 4's mechanism was replaced on 2026-11-16 as well (modes.py,video_conditioning="variation"): the clip is encoded, noised to one of the distilled schedule's own levels β 0.42, 0.73 or 0.91 ("a little", "moderately", "a lot", the default) β and denoised along the rest of the schedule, so the result is a variation of the clip, at the clip's length. Verified on real weights on a Mac: SSIM to the clip 0.78 / 0.65 / 0.36β0.40 at the three levels, and the prompt acts only at 0.91 (SPACE_MODE_MATRIX.mdΒ§10.2). The clip as a condition with a strength, as in the row above, was a switch: the clip came back nearly unchanged (SSIM 0.72β0.83) at three strengths, and the prompt replaced it at the two lowest.
Mandatory mitigation for modes 8β10: run at the minimum legal resolution, skip the
stage-2 spatial upscale, export only the requested track. The UI must say in words that
audio-only modes still render video internally β a user who does not know that will read the
runtime as a bug. (Corrected 2026-11-16: this said 8β11. Mode 11 returns video, so it gets
none of these mitigations, modes.py.)
3.4 Multishot β downgraded to honest
There is no multishot or scene API. Multishot is prompt-structure driven only.
Worse: the 2.5 enhancer system prompts explicitly instruct the model not to describe scene cuts, so prompt-level multishot actively fights the enhancer. We present it as prompt-structure driven, unverified, no API support β a documented technique in the UI copy, not a feature with a button that implies support we cannot back.
(Superseding an earlier draft of this plan that called it a first-class affordance. It is not.)
β Corrected 2026-11-16: the UI no longer offers a shot list. Its rows fed the prompt-segment text to the pipeline as frames, which fails in the worker; it never worked on real weights. The API keeps its
conditionsslot, and rows that carry text instead of media are refused before a GPU is reserved (generate.scope_request_to_mode).
3.5 Prompt enhancement: ships, as a real toggle, default off
β οΈ This section is a correction. It read "unavailable by default; we ship no dead toggle", and the policy was right β an inert control is a lie with a checkbox. Both reasons given for it were wrong, and the second one was wrong in the expensive direction, because it priced a feature out on a number nobody had measured.
β Wrong reason #1: "the pack ships it as absent, so enable_prompt_enhancement=True
fails on our subset." It is true that model_index.json carries
prompt_enhancer: [None, None] and that there is no prompt_enhancer/ folder in the 56
files. It does not follow that the enhancer is unavailable. Upstream types the component
prompt_enhancer: Gemma4ForConditionalGeneration | None (pipeline_ltx2.py:249) and lists
it in _optional_components (:235) β a plain transformers class, nothing
diffusers-specific. The [None, None] entry is there precisely because the component is
expected to be constructed, not resolved from the snapshot. So it is an ordinary
from_pretrained into the HF cache, and none of model_files.py / preflight.py /
preload_from_hub needs to know it exists β all of which are single-repo by construction.
β Wrong reason #2: "ZeroGPU charges it twice at the packing peak, ~20.6 GB." A
placement artefact, not a property of the model. torch.pack() runs exactly once per
container, at demo.launch() (spaces/zero/__init__.py:32), over tensors registered in
cuda_aliases during the module-scope import phase. Inside a @spaces.GPU call the forked
worker has already run torch.unpatch() (wrappers.py:147). A model loaded lazily inside a
request handler is in neither window. Load it lazily and it is charged once, as a
download β and if it never goes to CUDA, it is never packed at all.
β
And the download is 2.49 GB. google/gemma-4-E2B-it-qat-mobile-transformers is the
quantisation-aware-trained build: public, ungated, one model.safetensors, 2.490 GB
against 10.279 GB for the bf16 google/gemma-4-E2B-it. Its config.json carries
quantization_config.quant_method == "gemma", which transformers registers as
GemmaQuantizer from 5.15.1 on; the unpacking layers it installs are plain torch with no CUDA
or kernel dependency, which is what lets it run on the CPU. (Corrected 2026-11-16: this said
5.15.1 was "already the floor in requirements.txt". The file now pins transformers==5.23.0
exactly (Β§5.1), whose loader refuses one per-layer scale table of this checkpoint, so
enhance.py reads that one weight back from the cached checkpoint and refuses any other
mismatch.)
β οΈ It has no top-level architectures key, so AutoModel* cannot dispatch on it and
Gemma4ForConditionalGeneration must be named explicitly. LTX25_PROMPT_ENHANCER_REPO
selects the repo; the bf16 build is one environment variable away.
What ships. ltx25_studio/enhance.py, called from generate() below the dispatch
branch β i.e. inside the @spaces.GPU worker, on the GPU, within the metered window. It is
loaded on first use and kept for the life of the worker β freeing it per call cost a β
measured +30.78 s on every enhanced generation, and the VRAM headroom that justified the
freeing turned out to be an artefact of running on half a GPU (MIG_ALLOCATOR_SIGHTINGS.md).
It is never loaded at module scope, which is what keeps it out of the packing window. Failure
raises PromptEnhancementError naming the repository and the reason; it never falls through
to the original prompt while the UI implies enhancement happened, and the prompt that was
actually sent is quoted in the run report.
β οΈ That placement is itself a correction, and the reason is the transferable part. Enhancement first shipped in the web process, on the CPU, and the argument for it was the disk argument above: run it outside the module-scope import phase and never move it to CUDA, and ZeroGPU can never pack it. That argument was researched, checked against the packer's source, and is correct β it is restated intact below, because it survives.
What it was not was complete. Measured on the live Space,
text_to_videoat 256x256x9 β the cheapest configuration that exists here:
run wall enhance OFF 15.38 s enhance ON, cold 654.15 s (+638.8) enhance ON, warm 553.45 s (+538.1) The cold/warm split isolates it. The 2.46 GB fetch takes 88 s and the weights load in under a second, so the download is not the cost: ~538 s is the CPU decode of 600 greedy tokens, about 0.9 s/token. The number that justified the CPU placement was "1-2 s" β a GPU figure applied to a CPU placement. 600 tokens in 2 s is 300 tok/s, which no CPU does on a 5B-parameter model.
"Where does it run" has two axes β disk and latency β and only one of them was optimised. The disk axis was reasoned about carefully and got the right answer. The latency axis was never measured at all, so a nine-minute wall shipped priced at two seconds. A placement decision that answers one axis and is silent on the other is half a decision, and the missing half is the one the user experiences. This is the lesson that should outlive the feature: when a component moves for a resource reason, state what it costs on every axis, and measure the ones you did not choose it for.
β
The disk argument survives the move, and that is why the move is affordable. Packing
fires once per container at demo.launch(), over cuda_aliases registered during the
module-scope import phase. Inside a @spaces.GPU call the forked worker has already run
torch.unpatch() (wrappers.py:147), so a .to("cuda") there is a real H2D copy that is
never packed β the in-repo precedent is pipelines.py, which moves the stage-2 upsampler
and diffusion decoder to CUDA inside the worker for exactly this reason. The enhancer stays
~2.5 GB in the HF cache, out of the snapshot and out of preload_from_hub, and gets CUDA
speed: ~1-2 s of billed wall against ~85 s for a real generation, under 5% overhead where
the CPU placement was +3500%.
β Corrected 2026-11-16: "~1-2 s" is the decode alone. Loading the enhancer in a fresh worker takes about 29 s, and the reservation grows by
ENHANCE_DURATION_SECONDS= 180 s for every enhanced run, because the estimator cannot know whether the worker is warm (zerogpu.py). The pre-check needs that reservation Γ 2 of quota free onxlarge.
What that costs, and what now pays for it. Enhancement is inside the reservation, so
zerogpu.estimate_duration adds ENHANCE_DURATION_SECONDS when the toggle is set β an
enhanced run killed at the reservation boundary would be a self-inflicted wound β and the UI
caption gives up "costs no quota" to be honest about "no longer costs nine minutes". (It then
said "a second or two of quota", which was also wrong. Since 2026-11-16 it says the run
reserves 180 s more and usually uses a few seconds.)
And the download is prefetched, because it would otherwise be billed. Without it the
first enhanced call in each container spends the 88 s Hub fetch inside the metered window:
~176 GPU-seconds to move a file. enhance.prefetch pulls the weights into the HF cache at
startup via huggingface_hub.snapshot_download β a download only, constructing no torch
tensors, so there is nothing registered in cuda_aliases for torch.pack() to capture and
the disk argument is untouched. snapshot_download rather than a per-file allow-list because
LTX25_PROMPT_ENHANCER_REPO is a real lever: any filename list we hardcoded would be wrong
the moment an operator pointed it at the bf16 build. It runs on a daemon thread and is
best-effort by contract β it may never block demo.launch() and may never fail startup.
A Space whose prefetch failed still starts, still serves un-enhanced generations, and merely
pays the download on its first enhanced run: degraded, not broken. Which is also why the
reservation increment is sized for the download rather than for the decode.
LTX25_PREFETCH_ENHANCER=0 declines the ~2.5 GB for a deploy that will never enable
enhancement β it lands while demo.launch() is packing, which is the window
ZEROGPU_DISK_CAP_GB is measured against.
β Corrected 2026-11-16: on the deployed build the prefetch fails at startup (
AttributeError: 'tqdm' object has no attribute '_progress', in the 2026-11-13 log; the cause applies to every boot): the UI'sgr.Progress(track_tqdm=True)patched tqdm process-wide and broke the prefetch's progress bars. The degraded path is the normal one, so the first enhanced run in each container pays the download inside its reservation. The UI no longer installs that patch (ui/blocks.py); a prefetch that works on the Space is not verified yet.β Corrected 2026-11-17: on deploy 1 the prefetch ran (the Space's log of 2026-11-16 shows the 2.49 GB download under way once the page was up), and the image now preloads the enhancer's 8 files with the model's (
preload_from_hub, Β§6.2), so the prefetch is the fallback and finds them cached.
The settings, unchanged from what this section banked in advance and now imported from
upstream rather than restated: prompt_max_new_tokens=600, greedy decoding
(do_sample=False, no_repeat_ngram_size=5), prompt_enhancement_seed inert under greedy
decoding β so no seed control is offered, and the system prompt auto-selected between
LTX2_5_T2V_DEFAULT_SYSTEM_PROMPT and LTX2_5_I2V_DEFAULT_SYSTEM_PROMPT on whether a
reference frame is present. enhance.py deliberately omits upstream's torch.manual_seed
call for the same reason: under greedy decoding it changes no output, and it would reseed the
global RNG that this app's own seeded generator is drawn from.
3.6 Duration prediction: ships
duration_head is in the pack (0.004 GB) and in _optional_components. Omit num_frames, pass
min_seconds / max_seconds. See Β§5.7 for the num_frames default trap and the >1-prompt
restriction.
4. Decision 4 β LoRA: the headline feature
User-uploadable LoRA at inference time is why this app exists. It must be the most robust thing in it.
4.1 Verified mechanics
- All generation pipelines are declared
class LTX2β¦Pipeline(DiffusionPipeline, FromSingleFileMixin, LTX2LoraLoaderMixin). LTX2LoraLoaderMixin._lora_loadable_modules = ["transformer", "connectors"]. Noteconnectorsβ a LoRA may legitimately touch the text-conditioning path.- Both key conventions load.
lora_state_dict()detectsis_non_diffusers_format = any(k.startswith("diffusion_model.") for k in state_dict)and runs_convert_non_diffusers_ltx2_lora_to_diffusers(state_dict); separately it detectshas_connector = any(k.startswith("text_embedding_projection.") for k in state_dict)and converts those into the connectors sub-dict. lora_state_dict(..., return_lora_metadata=True)returns the file's__metadata__.
4.2 The danger: upstream validation is almost non-existent
The only built-in check is a substring test:
ValueError("Invalid LoRA checkpoint. Make sure all LoRA param names contain 'lora' substring.")
A LoRA for a completely different architecture passes that check β it has lora in its key
names β then attaches to zero modules. Generation proceeds. Output looks like the base
model. No error is raised. That is silent garbage, and it is precisely the class of failure
the operator has told us to eliminate.
We do have a real compatibility signal. Reading the official LoRA's safetensors header directly:
3320 tensors, all under prefix diffusion_model., peft-style lora_A.weight/lora_B.weight,
targeting to_q/to_k/to_v/to_gate_logits, adaln_single, audio_adaln_single; and
__metadata__ = {'lora_rank': '450', 'lora_alpha': '450', 'model_version': '2.5.0', 'license': β¦}.
model_version is machine-readable.
4.3 The validator (9 stages β each stage is a test unit)
Accepts either a Hub repo-id or a direct file upload. Both routes converge on one parser.
| # | Stage | Rule | On failure |
|---|---|---|---|
| 1 | Size bound | reject above a configured max | refuse before any download or parse. On the Hub route this comes from the HTTP Content-Length β before a single weight byte moves |
| 2 | Header-only parse | read the safetensors header, never materialise the payload | β |
| 3 | Format | safetensors with a parseable header | distinct messages: not-safetensors vs unparseable-header |
| 4 | Version | read a set of signals in priority order (Β§4.5): model_version, ss_ltx_version, modelspec.architecture, ss_base_model_version |
warn if all absent, reject only on explicit mismatch. Absent is the common case, not an edge case |
| 5 | Convention | classify: diffusion_model. / transformer. / text_embedding_projection. / unknown |
on unknown, the error names the prefixes actually found |
| 5.5 | IC-LoRA detection | an in-context LoRA is marked by __metadata__.reference_downscale_factor |
refuse with a specific explanation. An IC-LoRA applied to the plain pipeline neither errors nor works β see below |
| 6 | Pairing | lora_A/lora_B (or lora_down/lora_up) must pair |
name the unpaired keys |
| 6.5 | Per-module dimension compatibility | resolve each pair to its target module in the real transformer and compare against that module's in_features/out_features |
reject naming module, expected, actual. A key whose target module does not exist at all is a distinct failure β that is what a genuinely foreign adapter looks like |
| 7 | Attach assertion | after load_lora_weights, assert the adapter attached to > 0 modules |
specific error if zero β the anti-silent-garbage check |
Stage 2 is a hard rule: a multi-GB upload must never be fully materialised just to discover it is the wrong architecture.
Stage 7 is necessary but NOT sufficient β do not delete 6.5 as redundant
The case that proves it: charlied98/ltxv_2b_lora_soft_draw_v1, an LTX-Video-2B adapter, i.e.
flatly the wrong architecture. Header measured:
- prefix
diffusion_model.β passes stage 5 - peft
lora_A/lora_Bpairs β passes stage 6 - every key contains
loraβ passes diffusers' one and only built-in check __metadata__empty β passes stage 4 under our own (correct) absent-is-not-fatal rule- module names are
transformer_blocks.N.attn1.to_kβ the same names LTX-2.5 uses, so it may attach to > 0 modules and pass stage 7 as well
The only thing wrong with it is that to_k.lora_A is [32, 2048] where this model is 4096.
Stage 6.5 is the only gate that catches it.
Stage 6.5 compares PER MODULE β there is no single "expected hidden dim"
The transformer carries two streams, and a naive "everything must be 4096" rule would reject
the most-downloaded LTX LoRA on the Hub. Measured from joyfox/LTX-2.3-Transition-LORA:
| module family | in-dim |
|---|---|
attn1.to_q/k/v/out.0, attn2.β¦ |
4096 (video stream) |
ff.net.N.proj |
4096 |
ff.net.2 |
16384 (4Γ feed-forward expansion) |
audio_ff.net.N.proj |
2048 (audio stream) |
audio_ff.net.2 |
8192 |
Corroborated by the official 2.5 adapter, which pairs adaln_singleβ¦linear_1.lora_B [4096, 256]
with audio_adaln_singleβ¦linear_1.lora_B [2048, 256].
So "contains 2048" is not a signal. "2048 at a module that expects 4096" is. Build the
expectation map by walking the loaded transformer's nn.Linear modules β never from a hardcoded
table β so it stays correct across checkpoints. The derived
num_attention_heads Γ attention_head_dim = 32 Γ 128 = 4096 is one entry in that map, not the
rule.
5.7.1 The anchor is a frame, and the conversion is lossy
Ruling: the user types a pixel frame. conditioning.py converts frame β latent. Users think
in frames β "put my second shot at frame 24". A latent index is an artefact of the VAE's 8:1
temporal compression and means nothing to someone generating a video. This keeps the conversion
in exactly one place, which is what this row assigns to conditioning.py, and it handles the
trap rather than explaining it: labelling the field LATENT would make the off-by-8 the user's
problem.
The lossy direction must not be silent. Frame β latent is 8:1, so a run of frames collapses onto one anchor. The readout says so rather than snapping quietly:
| shot | anchor (frame) | lands on latent | frames sharing it |
| 2 | `25` | **4** | 25β32 |
| 3 | `30` | **4** | 25β32 β΅ |
β΅ Temporal compression is 8:1, so these anchors snap. Frame `30` anchors at the
same place as frame `25` β frames 25β32 all share latent **4**.
conditioning.latent_group_for() owns that arithmetic; the UI words the answer and computes none
of it. A second copy in the render layer would eventually disagree with the real one, and a
readout that disagrees with what runs is worse than no readout.
Historical note: the two halves of this once disagreed β the column was labelled
"Anchor (LATENT index)" while normalise_condition treated it as a pixel frame β because an
instruction asked for both at once. Wiring them naively would have double-converted every anchor.
Stage 5.5 β why IC-LoRAs must be refused rather than loaded
DoctorDiffusion/LTX-2.3-IC-LoRA-Colorizer carries
__metadata__ = {'reference_downscale_factor': '1'}. That maps to the reference_downscale_factor
kwarg on LTX2InContextPipeline, which upstream documents as needing to match the factor the
IC-LoRA was trained with. Applied to the plain pipeline an IC-LoRA will not error and will not
work β the exact silent-garbage shape stage 7 exists to prevent, arriving through a door stage 7
does not watch.
4.4 Fusing β the connectors trap
fuse_lora / unfuse_lora default to components=["transformer"] only.
Connector LoRA layers are silently not fused unless we pass
components=["transformer", "connectors"]. A LoRA that trains the text-conditioning path would
partially apply β worse than not applying, because it looks like it worked. We always pass both
components explicitly, and this is a test case.
4.5 What real LoRAs actually look like β measured, and it changes stage 4
Technique note, since this document cites measured headers in several places: every such
measurement was taken by HTTP range-reading the safetensors header β Range: bytes=0-7 for
the 8-byte little-endian header length, then Range: bytes=8-(8+N-1) for the JSON β with no full
download. It reads a 9 GB adapter's complete key list in well under a second. This is the same
technique lora/inspect.py uses for the Hub route (Β§4.6).
I range-read the headers of three real public LTX LoRAs:
| LoRA | size | tensors | prefix | __metadata__ |
model_version? |
connectors keys |
|---|---|---|---|---|---|---|
joyfox/LTX-2.3-Transition-LORA (19.4k downloads) |
0.390 GB | 1152 | diffusion_model. |
ABSENT | β | 0 |
vrgamedevgirl84/LTX_2.3_Crisp_Enhance_Style_LoRa |
0.705 GB | 1152 | diffusion_model. |
present, sd-scripts style | no such key | 0 |
elix3r/LTX-2.3-22b-AV-LoRA-talking-head |
0.428 GB | 2304 | diffusion_model. |
ABSENT | β | 0 |
Four findings, each of which changes the design:
Not one of them carries
model_version. Stage 4 as originally written β "read__metadata__.model_version" β would be warn-only for 100% of real community uploads, i.e. effectively dead code. The official Lightricks LoRA does carry it ('2.5.0'), so the field is real -- but the official adapter is the exception, not the template. A validator written from it as the reference would have been written from the one file in the population that is unrepresentative of the rest.The absent case has two distinct shapes and the code must handle both: no
__metadata__key at all, and__metadata__: {}present but empty. Both mean "no version signal". Since this is the common path rather than an edge case, stage 4's warn-not-reject branch must not be "tightened" later by someone who assumes metadata is normally present.There is a second, richer convention we must read. The sd-scripts / musubi-tuner trainer writes a large metadata block with entirely different keys β from the measured header:
ss_ltx_version: "2.3",ss_base_model_version: "ltx2_v1",modelspec.architecture: "LTX2/lora",ss_network_module: "networks.lora_ltx2",ss_sd_model_name: "ltx-2.3-22b-dev.safetensors",ss_network_dim: "64",ss_network_alpha: "32",ss_lora_target_preset: "full".modelspec.architecture == "LTX2/lora"is a better architecture signal thanmodel_versionis β a Flux or Wan LoRA carries a different value there, which is exactly the wrong-architecture case stage 7 exists to catch. Reading it lets us catch that at stage 4 instead, cheaply, before any weights load.Revised stage 4 reads, in priority order:
model_versionβss_ltx_versionβmodelspec.architectureβss_base_model_version. All absent β warn and proceed; rejecting on absence would reject the most popular LTX LoRA on the Hub.diffusion_model.is the dominant real-world convention, not an edge case. All three use it, so_convert_non_diffusers_ltx2_lora_to_diffusersis on the main path. The diffusers-nativetransformer.convention is the rarer one.None of them touch the connectors (
text_embedding_projection.count is 0 across all three). So in practice stage 7 will see transformer-only attachment. Β§4.4's both-components fusing rule stays β it must be right for the LoRA that does touch them β but we should not expect connector attachment as the normal case, and a stage-7 assertion demanding it would reject every real LoRA.
These three headers become offline test fixtures for stages 1β6.5 (Β§9.1): real-world data, zero GPU, zero weights, no network at test time.
The convention detector is now backed by real inputs in both conventions.
Until the H13 work it had only ever seen fixtures we wrote, shaped by our own
reading of the format. charlied98/ltxv_2b_lora_soft_draw_v1 ships the same
adapter twice β lora_weights_step_03000.safetensors under transformer.
(diffusers) and comfy_lora_weights_step_03000.safetensors under
diffusion_model. (native) β and running the real validator against both gives:
transformer. (diffusers) -> rejected at stage 6.5, LoRADimensionError
diffusion_model. (native) -> rejected at stage 6.5, LoRADimensionError
Both converge on the correct stage. The detector accepted two conventions neither of us invented and routed each to the dimension check β which retires the worry that the two paths might diverge, or that one might not reject at all.
It also fixes what "rejected" means as a result: a run reporting "rejected"
without naming LoRADimensionError has not distinguished success from the
convention detector breaking. Two mechanisms could produce the same verdict, and
only one of them is the validator working.
4.6 lora/inspect.py has ZERO heavy dependencies β and that buys a real feature
The safetensors container is trivial:
[8 bytes little-endian uint64 = header length N][N bytes of JSON header][raw tensor bytes]
struct + json from the stdlib read __metadata__, every key name, every dtype and every
shape. No torch, no diffusers, no safetensors package. Three reasons, in order:
- A bad upload must be rejectable without loading torch. A validator that needs a 2 GB import to say "this is not a safetensors file" is the wrong shape.
- It keeps the gate fast and portable. Stages 1β6.5 become offline, torch-free unit tests
that run in milliseconds, with fixtures hand-constructed from
struct+json. Only stage 7 needs real weights, and that isgpu-marked. - It lets us validate a Hub LoRA before downloading it. The header sits at the very start of the file, so one range request validates the whole thing.
Hub route: resolve the file β range-read the header β run stages 1β6.5 β only then
download. A user who pastes a repo-id for a 2B-architecture adapter gets a specific rejection in
about a second instead of after a 300 MB download. Stage 1 comes free from Content-Length.
β Corrected 2026-11-16: the range-read is not built.
lora/fetch.pychecks the size from the listing, then downloads, then validates: a wrongly-built adapter costs one bounded, cached download (its docstring says so).
Upload route: the bytes are already local, so the same parser runs against the file. Both routes converge on one code path, which is what Β§4.3 asks for.
validate.py stays dependency-light too. apply.py is the only module in lora/ that may
import diffusers or peft, because it is the only one that touches a real model.
5. Decision 5 β Resilience doctrine
Binding on all code in the repo.
5.1 Pin every dependency to one exact version β and know what NOT to pin
Evidence from the predecessor: spaces>=0.30.0 and torch>=2.4.0 β floors with no ceiling
β alongside exactly one hard pin, torchaudio==2.10.0, carrying this comment:
"HF Spaces ships torch==2.10.0; bare 'torchaudio' lets pip pick an older wheel whose C++ ABI doesn't match β import crashes the whole audio code pathβ¦ Pin the matching version."
That comment is a post-mortem, applied to one line while ~50 stayed unbounded.
But the inverse mistake is also real, and the 2.3 repo made it. Its own AGENTS.md:19 says
do not pin spaces β the platform installs its own version, and a conflicting pin breaks pip
resolution at build time (in August, live Spaces reported 0.49.3 / 0.50.0 / 0.50.4 while PyPI had
0.51.1; this Space reports 0.51.3 on 2026-11-16, and PyPI has had 0.52.0 since 2026-11-13). Its
requirements.txt:2 then pins spaces>=0.30.0 anyway. We follow the rule the repo wrote and
did not keep: no spaces line in requirements.txt. app.py guards the import
(try: import spaces, and on ImportError the local stub zerogpu.local_spaces_stub(), whose
GPU returns the function unchanged), so local dev works without it.
Corrected 2026-11-16: every line is now ONE exact version, not a range. This section used to give each dependency a floor and a ceiling. A range is resolved again on every Space rebuild, so the image gets whatever is newest that day instead of what was tested; on 2026-11-16 a fresh resolution gave transformers 5.23.0, on which the pinned prompt enhancer did not load.
requirements.txtnow pins the versions the real-weights runs of 2026-11-16 used, andtests/test_requirements_exact.pyfails on any range (its only exemption is a line marked# platform-provided). Those runs were on Apple silicon with PyTorch 2.15.0, so the CUDA-only lines below β the torch trio as the Space installs it,bitsandbytes,kernelsβ have no real-weights evidence under these pins until the Space runs. In particular the bitsandbytes int8 load of the transformer and the text encoder under transformers 5.23.0 has not run yet.β Corrected 2026-11-17: they ran on the Space with deploy 1 (2026-11-16): the pins installed exactly, and the int8 load of the transformer and the text encoder under transformers 5.23.0 ran (
SPACE_MODE_MATRIX.mdΒ§10.1).
The full pin set, as requirements.txt has it:
| package | pin | why |
|---|---|---|
torch |
==2.11.0 |
ZeroGPU supports exactly 2.8.0 / 2.9.1 / 2.10.0 / 2.11.0; ceiling is 2.11.0. CUDA 12.8, sm_120 needs cu128+. Not for local Apple-silicon runs: 2.11 computes attention wrongly on MPS (README, "Run it locally") |
torchvision |
==0.26.0 |
trap: torchvision 0.28.0 hard-declares torch==2.13.0, so a floating torchvision silently upgrades torch past the ceiling |
torchaudio |
==2.11.0 |
must match torch exactly (the ABI break above) |
diffusers |
@ git+β¦@ac56fa2fβ¦ |
Β§1.3 |
transformers |
==5.23.0 |
the version the 12B text encoder and the pipelines ran with on real weights. Its loader refuses one per-layer scale table of the prompt enhancer's checkpoint, so enhance.py reads that one weight back from the cached checkpoint and refuses any other mismatch |
tokenizers |
==0.23.3 |
pinned beside transformers because it decides whether transformers imports at all |
accelerate |
==1.15.0 |
|
peft |
==0.20.0 |
LoRA backend |
safetensors |
==0.8.0 |
|
huggingface_hub |
==1.33.0 |
|
hf_xet |
==1.8.0 |
|
kernels |
==0.16.2 |
fetches the NATTEN kernel the diffusion decoder needs; CUDA only |
bitsandbytes |
==0.50.2 |
the int8 route on ZeroGPU. Must stay above 0.46.0, below which the packer tries to move quantised tensors itself |
gradio |
==6.24.0 |
must equal the README's sdk_version (a drift test asserts it). 5.x is END OF LIFE (last stable 5.50.0, 2025-11-21) |
imageio |
==2.38.1 |
|
imageio-ffmpeg |
==0.6.0 |
|
av |
==18.1.0 |
PyAV β required by CRF re-compression (Β§5.7) |
numpy |
==2.5.5 |
numpy 2. Do NOT go back to 1.x: 1.26.4 is 2 years EOL and nothing in the stack caps it |
pillow |
==12.3.0 |
|
spaces |
absent, deliberately | the platform installs it; see above |
The named fallback. If the int8 load misbehaves on the Space, the fallback is
transformers==5.22.2 with exactly the same companion pins (the header of requirements.txt
says so). On 5.22.2 the enhancer's checkpoint loads as stored and nothing is read back.
gradio 6 breaking changes that hit us: gr.Video subtitle-tuple return removed;
show_download_button / show_share_button β buttons=[...]; gr.Audio show_controls β
show_recording_waveform; min_length / max_length β event-listener validators.
Python 3.12 is required, not preferred: numpy 2.5.5 needs β₯3.12 and av 18.1.0 needs β₯3.11
(their Requires-Python). The default 3.10.13 image resolved both downward when the pins were
ranges; with exact pins it fails the build instead.
5.2 One central model-file indirection
Every Hub repo-id, revision, subfolder, filename and dtype/quantization choice (Β§2.6) lives
in one module β ltx25_studio/model_files.py β as data, never inlined at a call site.
A missing file must raise an error naming both the file and its URL. Not hypothetical: Lightricks deleted a 2.3 checkpoint from the Hub with no warning and broke a live Space. When it happens again, the message should tell the operator which URL to go look at.
Pinning a revision per repo is part of this module's job β a repo whose content can change under us is not pinned just because its name is stable.
Corrected 2026-11-16: two things live elsewhere. The quantization choice is
LTX25_QUANTIZATION, resolved inconfig.py/device.pyand applied inpipelines.py, not here. The prompt enhancer is outside this module by design (Β§3.5) and is pinned on its own (enhance.ENHANCER_REVISION). The diffusion decoder has its own revision here (aba1b702, the vendor's corrected upload of 2026-10-08); every other component stays at871165de. One revision is still not ours to pin: the NATTEN kernel the diffusion decoder fetches (version=1, hard-coded upstream).
New requirement: model_files.py emits the download allow-list as data. It must be able to
produce the exact list of 37 paths (Β§2.3), so that:
- the runtime download and the Space README
preload_from_hub:list are generated from the same source, never hand-maintained in two places; and - a drift test can assert the committed README list equals what
model_files.pydeclares.
And the live shard names must come from reading the index files, not from a hardcoded list. That is what makes an upstream re-shard get picked up automatically instead of silently downloading the dead set.
But the index alone is not enough, and this bug was made for real. A first attempt at
generating the list produced 33 files: for the three sharded components it took only the
shards named in the weight map plus the index, and so silently dropped transformer/config.json,
text_encoder/config.json, connectors/config.json and text_encoder/generation_config.json.
from_pretrained cannot construct a component without its config, so a Space preloaded from that
list would have failed at startup after a 72 GB download.
The weight-map index tells you which weights are live. It says nothing about sibling config files. So the rule is per component:
- sharded (
transformer,text_encoder,connectors): the shards named in the index, plus the index itself, plus every non-.safetensorsfile in the folder - whole-file (everything else): every file in the folder
- plus root
model_index.json
And the emitter asserts, as a self-check, that every component contributes a config file --
config.json for most, scheduler_config.json for scheduler, tokenizer_config.json for
tokenizer. That assertion is what would have caught the mistake, and it is cheap. Hardcoding today's shard names would reintroduce Trap A (Β§2.1) the
moment Lightricks re-shards β which, given they have already deleted a checkpoint without
warning, is not a remote possibility.
This is the single-source-of-truth principle (Β§5.2) applied to deployment, and it is the thing that stops the shard-set trap from silently reappearing.
5.3 Never return an empty output β and the four layers that let it happen
Any generation that completes without producing a file must raise a specific, actionable error that the UI renders visibly.
The full anatomy of the 2.3 silent reset, because each layer was individually "reasonable" and that is the lesson:
models.ensure_modelshit a missing file and didlogger.warning(...) + continue.- ComfyUI's executor caught every node exception and returned a FAILURE tuple, without raising.
- An
OutputEvent(video_path="")was pushed unconditionally. - The UI did
event.video_path if event.video_path else gr.update()β rendering an empty path as a silent form reset.
The user saw "nothing happened". No error, no log, no signal. It hid for a week.
The exports tuple is a contract, checked in BOTH directions (generate._artifacts_for):
- declared but missing β raises. A mode promising a track and producing nothing is the never-empty bug one level up.
- written but undeclared β dropped and noted, not raised. The registry decides a mode's
exports, so an over-generous encoder is over-generous rather than wrong; failing a run that
produced a perfectly good video is the harsher answer to the milder problem. The extra track
never reaches the result, and the drop is surfaced on
GenerationResult.warnings.
Enforcing only the first direction is what let a mode declare exports=("audio","video") and
return audio_path=None, so the UI reported every successful joint-mode run as an internal
error. One-directional contracts have now failed here twice; the direction that is not checked
is the one that bites.
Our rules, both halves mandatory:
- An empty path becomes an error AT THE SOURCE, and every layer propagates it. A success path that can return a falsy artifact is a bug regardless of current reachability.
- No "newest file in the output directory" fallback. The 2.3 one had no run correlation and would show the previous run's video for a failed run β a worse failure than showing nothing. Each run is stamped into its own output subdirectory, and a run's artifact is only ever looked up inside that run's directory.
5.4 Read parameters from config, never hardcode
Mel/audio parameters come from audio_vae.config at runtime: audio_sampling_rate,
audio_hop_length, audio_mel_bins, audio_vae_mel_compression_ratio,
audio_vae_temporal_compression_ratio, audio_latent_channels.
β Corrected 2026-11-16: the analysis window is in no config, and taking it from the wrong one cost 9 dB. No config describes the audio VAE's mel analysis window. The app took its FFT size and window (512) from
vocoder.config, whose values set up the vocoder's own internal mel stage, so every uploaded sound reached the model about 9 dB too quiet and duller: the app's encode β decode round trip, with no generation, returned speech at β8.9 dB. Since7cf1547the window is a declared convention inaudio.py(MEL_CONVENTIONS: FFT and window 1024, slaney mel scale and norm, magnitude), chosen by measured round trip (speech +0.2 dB, ocean β0.3 dB) and pinned by a test against the real audio VAE and vocoder. The sample rate, hop and mel bins still come fromaudio_vae.config.
This mirrors the pipeline itself, whose constructor reads self.vae.spatial_compression_ratio,
self.vae.temporal_compression_ratio, self.audio_vae.mel_compression_ratio,
self.audio_vae.temporal_compression_ratio and self.transformer.config.patch_size from the
loaded components, using literals only as fallbacks. Same discipline. Β§5.7 extends this to CRF.
5.5 Enforce and surface the hard constraints
| constraint | source | handling |
|---|---|---|
height % 32 == 0, width % 32 == 0 |
enforced upstream (raises) in all generation pipelines and the latent-upsample pipeline | validate before the call, snap in the UI, show the snapped value |
num_frames % 8 == 1 |
not hard-raised upstream. Falls out of latent_num_frames = (num_frames - 1) // vae.temporal_compression_ratio + 1, which silently floors a bad value |
we raise. The 8 is vae.temporal_compression_ratio β see below |
| audio/video length agreement | audio and video lengths are computed independently, and a supplied audio_latents overrides the duration-derived audio length (Β§5.7) |
validate they agree; raise rather than emit a desynced file. (As built, 2026-11-16: in a mode that takes audio, a set length longer than the audio is refused before a GPU is reserved, and audio longer than the video is trimmed with a note, never padded; automatic length there means "match my audio", taken from the file, because the duration head reads only the text. generate.py) |
The num_frames modulus is a parameter, not a literal. Hardcoding 8 would violate Β§5.4.
Resolution:
constraints.pytakes the ratio as a parameter with a documented fallback of8, because the UI must validate input before any VAE exists.pipelines.pyasserts the loadedvae.temporal_compression_ratioequals that fallback at load time, raising loudly if upstream ever changes it.
That gives a working pre-load validator and a tripwire. Both are test cases.
Verified constants (diffusers/pipelines/ltx2/utils.py):
DISTILLED_SIGMA_VALUES = [1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875] # 8 steps, CFG=1
STAGE_2_DISTILLED_SIGMA_VALUES = [0.909375, 0.725, 0.421875] # 3 steps
Re-exported and asserted in constraints.py against the installed diffusers, so a schedule
change upstream fails a test instead of silently changing output quality.
5.6 Preflight β fail loudly on resources, before the platform kills us
At startup, preflight.py measures free RAM and free disk and compares them against the
requirement computed from the model_files.py table. If short, it raises a specific error
naming the numbers:
need ~67.2 GiB RAM for the bf16 load, saw 41.3 GiB(see Β§2.6 for the quantised fallbacks)
This is the fail-loudly principle applied to the resource budget. It turns an anonymous OOM or a
"Workload evicted, storage limit exceeded (150G)" into a diagnosable message that names the
fix. It also runs the Β§1.3 diffusers-version assertion. Test case.
β Corrected 2026-11-16: only half of this exists.
preflight.pychecks the diffusers version and that the weights are complete, and the startup report logs the disk floor; it measures no free RAM, on the Space or anywhere (its docstring still promises it). A memory preflight exists for local runs only:memory.pychecks committed memory plus what the next load or the denoise will add against a limit (LTX25_MEMORY_LIMIT_GB, 90 GB on macOS by default) and refuses with the numbers.On the live gauge it waits for memory just freed (
ba2d582). Freed MPS memory is wired, and the kernel un-wires it over a moment: the text encoder's 24 GB left the gauge over 0.1β0.35 s after its release, so a check that read the gauge at once read it 3.5β23 GB too high and refused runs that fitted. So after the local runner frees device memory, the next check reads the gauge every 50 ms until two consecutive readings are within 0.25 GB and judges that reading; a check that fails is read again every 0.1 s and passes as soon as a reading fits, else it refuses on the last reading. Both waits share one bound,LTX25_MEMORY_SETTLE_SECONDS(2 s by default, 0 to 60; 0 is a single reading, as before). A projected judgement (the dry run), no gauge (off macOS) or no limit never waits. A refusal is its own failure category,memory, notbug(errors.py).
The startup report (pipelines.startup_report) prints, before anything expensive:
LTX-2.5 Studio starting
config : transformer=bf16 text_encoder=bf16 zerogpu_size=auto duration=60s
expected_weights=72.19GB outputs<=20GB
weights cached : 36/37 files MISSING 1
disk : 412.5 GB free of 1995.2 GB
zerogpu size : not yet known (reported after packing)
model revision : 871165de037793df7d75c23a02dd7f45fd364c97
first missing : vae/config.json
Stale example (annotated 2026-08-25). The
transformer=bf16 text_encoder=bf16config fields shown above were removed with the precision variables (Β§2.6). The live Space prints, instead:dtype : bf16 (fixed; no precision switch)andquantised transformer, text_encoder as int8 | peak RSS 0.91 -> 87.13 GiBβ captured inSPACE_GENERATION_FINDING.mdΒ§5 andOPERATOR_HANDOFF.mdΒ§5. The load-bearing line survives unchanged: the live Space reads37/37 files.β Stale again (corrected 2026-11-16). The report was rewritten: it no longer prints
dtype : bf16 (fixed; β¦), which was false under int8, butdevice,precisionandrunnerlines resolved from the configuration;packed copy : not measured yet, ZeroGPU writes it at launch after this report; andzerogpu size : requested xlarge; reservation 180 s per run (x1.5 with stage 2, +60 s with LoRAs, +180 s with prompt enhancement). The oldzerogpu size : not yet known (reported after packing)was never followed by a report. The new report has not run on the Space yet. (Corrected 2026-11-17: it ran on deploy 1,SPACE_MODE_MATRIX.mdΒ§10.1, "What ran". Since deploy 2's code its reservation line names the size-and-length rule.)
The cache count is the load-bearing line: 36/37, with the first missing file named, makes
"preload worked" and "preload silently did nothing" two different sentences. When a GPU run fails
on a metered Space behind a 72 GB build, these lines are the difference between diagnosing it in
one round trip and burning several -- and there is no opportunity to add a print statement
afterwards.
5.7 API gotchas that cause silent wrongness
Every row here is a test case, not a note. These are the ways this API is wrong-by-default.
| # | Gotcha | Consequence | Our handling |
|---|---|---|---|
| 1 | LTX2VideoCondition.index is a LATENT index, not a pixel frame index. start_idx = max((latent_start_idx - 1) * vae_temporal_compression_ratio + 1, 0) |
off-by-8 guaranteed if a pixel frame number is passed | convert at the boundary in conditioning.py. The user types a FRAME (Β§5.7.1); the pipeline gets a latent index |
| 2 | Out-of-range condition indices are silently SKIPPED with only a logger.warning |
a whole conditioning image does nothing, silently | validate range ourselves and raise; this is a silent-failure path we convert into a loud one |
| 3 | noise_scale default differs by class: 0.0 on LTX2Pipeline / LTX2ImageToVideoPipeline, None on LTX2ConditionPipeline (resolving to sigmas[0] or 1.0) |
copying a call between classes changes behaviour silently | always pass noise_scale explicitly; never rely on the default |
| 4 | num_frames default differs: None (auto-duration) on LTX2Pipeline/ImageToVideo/Condition; hard 121 on LTX2InContextPipeline and LTX2HDRPipeline, which have no duration_head at all |
auto-duration silently unavailable on two pipelines | registry records per-workflow duration support; auto-duration also raises with >1 prompt and num_frames=None β validate before the call |
| 5 | latents must be UNPACKED 5D [B,C,F,H,W]; audio_latents UNPACKED 4D [B,C,L,M] |
shape errors, or worse, silent misinterpretation | assert shapes at our boundary |
| 6 | output_type="latent" returns ALREADY-DENORMALIZED latents |
re-feeding them re-normalizes β corrupted output | drives the mandatory denormalize=False in Β§7.4 |
| 7 | Image-conditioning CRF re-compression is applied BY DEFAULT, needs PyAV + uint8/PIL input; a float tensor raises unless crf=0 |
crashes on float input | normalise to uint8/PIL in conditioning.py; av is a pinned dep for this reason |
| 8 | CRF is 18 when text_encoder.config.model_type in ("gemma4_unified", "gemma4"), else 33 |
hardcoding either number is wrong for the other | derive from text_encoder.config.model_type at runtime (Β§5.4) |
| 9 | from diffusers.pipelines.ltx2.export_utils import encode_video is DEPRECATED, removal in 0.40.0 β and our pin is 0.40.0.dev0 |
breaks on the very revision we pin | use from diffusers.utils import encode_video |
| 10 | spatio_temporal_guidance_blocks defaults to [28] for 2.3/2.5 ([29] was 2.0) and is a mutable shared default |
in-place mutation corrupts every later call in the process | never mutate in place; copy before modifying |
| 11 | A supplied audio_latents overrides the duration-derived audio_num_frames: upstream logs "Got audio_latents of shape [β¦], audio_num_frames will be inferred" then does _, _, audio_num_frames, _ = audio_latents.shape |
supplying audio silently redefines audio length while num_frames still governs video β desync |
Β§5.5 validation, with a concrete desync test case |
| 12 | fuse_lora/unfuse_lora default components=["transformer"] |
connector LoRA layers silently unfused | Β§4.4 β always pass both |
Reference: _optional_components = ["processor", "prompt_enhancer", "duration_head"] (plus
audio_scheduler on Condition), and
model_cpu_offload_seq = "prompt_enhancer->text_encoder->connectors->duration_head->transformer->vae->audio_vae->vocoder".
Recorded for completeness β but see Β§8.4: we do not use accelerate's offload at all.
5.8 Operational lessons from the 2.3 post-mortem
| # | Lesson | Rule |
|---|---|---|
| 1 | ZeroGPU quota attribution rides a contextvar (gradio.context.LocalContext.request). It does not cross a plain threading.Thread β without it every user is bucketed as Space-anonymous and you get "Space app has reached its GPU limit" at 0 minutes used |
any thread we spawn must carry contextvars.copy_context(); better, do not spawn plain threads across the GPU boundary |
| 2 | The duration= callable must accept the exact same positional signature as the decorated function, trailing kwargs included |
otherwise TypeError at schedule time; test the signature match |
| 3 | Hidden 1.5Γ duration multiplier on large (duration_factor): duration=60 is scheduled as 90 s. On xlarge the factor is skipped but the request is doubled instead |
never reason about wall-clock quota from the duration= number alone |
| 4 | Default duration is 60 s. The max is a server-side, tier-dependent, unpublished cap | discover it, never hardcode it. Empirically ~120 s is the safe ceiling; the official Lightricks LTX Space ships @spaces.GPU(duration=75, size="xlarge"). (Corrected 2026-11-16: this app's base reservation is now 180 s, LTX25_GPU_DURATION_SECONDS, and zerogpu.estimate_duration can ask for up to 510 s: Γ1.5 with stage 2, +60 s with LoRAs, +180 s with the enhancer. The server's maximum is still not known.) (Corrected 2026-11-17: the reservation now follows each run's size and length, fitted to the Space's measurements with a 1.3 margin per term (zerogpu.estimate_duration; SPACE_MODE_MATRIX.md Β§10.3): 60 s for the default 704Γ480, 5 s run, +45 s with stage 2 at that size and more for larger or longer, +40 s with adapters, +80 s with the enhancer; LTX25_GPU_DURATION_SECONDS is ignored, and LTX25_GPU_DURATION_FLOOR_SECONDS sets a minimum. The largest run measured to finish, 1920Γ1088 at 10 s through stage 2, reserves 240 s. The server's maximum is still not known.) |
| 5 | Do NOT auto-retry a ZeroGPU timeout using the captured Gradio request: the X-IP-Token ages past TTL during attempt 1, so attempt 2 always 401s with "Expired ZeroGPU proxy token" β masquerading as the root cause |
surface the original timeout; never retry behind the user's back |
| 6 | preload_from_hub is capped at 10 entries, and preloaded files are owned by the build user and read-only to runtime uid 1000 β a later lazy hf_hub_download fails with Permission denied (os error 13) unless the cache is mirrored |
Β§6.2 |
| 7 | Accumulated generations filled the 2.3 Space's disk and it went unhealthy | Β§2.7 β sweep on startup and after each run |
6. Decision 6 β Deploy shape
| Space | techfreakworm/LTX2.5-Studio, public (private until 2026-11-16) |
| SDK | sdk: gradio, sdk_version: 6.24.0 β a custom Dockerfile forfeits ZeroGPU entirely |
| Python | python_version: "3.12" β required, not optional (Β§5.1) |
| Hardware | ZeroGPU. suggested_hardware: zero-a10g is the legacy id string the Spaces API still reports β it is NOT an A10G (Β§8.1) |
| Size | size="xlarge" declared explicitly (Β§8.6): LTX25_ZEROGPU_SIZE defaults to xlarge |
| Weights | from the image: preload_from_hub: fetches the 35 model files and the prompt enhancer's 8 before the app starts; preflight.ensure_weights() downloads at start-up only what is missing, if the disk can hold it (Β§6.2) |
| Secret | HF_TOKEN β required; Lightricks/LTX-2.5-Diffusers is gated |
| Git | GitHub git@github.com:techfreakworm/ltx2.5-studio.git; Space and GitHub stay in sync |
On the Size row (annotated 2026-08-25): never declared in practice β every deploy has run
auto, and the live Space was promoted toxlargeby inference. Β§8.6's annotation carries the detail.β Corrected 2026-11-16: not "by inference" β
autonever promotes, so the Space ran on the 47.375 GiBlargepartition untilLTX25_ZEROGPU_SIZE=xlargewas set as a Space variable on 2026-08-26 (Β§8.6). The code now defaults toxlarge, so the row holds as written even without the variable. The Weights row is corrected in place: the download moved to startup whenpreload_from_hubfailed on the private Space.β Corrected 2026-11-17: the Space is public, and the Space and Weights rows are corrected in place again: with the Space public, the preload is live (Β§6.2).
Naming precedent: techfreakworm/LTX2.3-Studio, wan-studio (private), SAM3-cutout-studio
(private).
6.2 Download strategy β REVISED: runtime download, not preload_from_hub
β Corrected 2026-11-17: the image brings the weights again. The Space is public, so the 401 below no longer applies, and
README.md's front matter carries a livepreload_from_hubblock, generated bymodel_files.preload_from_hub_entries()and held to it, revisions included, bytests/test_deploy_drift.py. Before the app starts it fetches the 35 model files the int8 Space loads (71.92 GB: the 37 below lesstemporal_latent_upsampler/, which nothing loads) at their pinned commits, and the prompt enhancer's 8 files (2.49 GB) at its own.preflight.ensure_weights()stays as the fallback: at start-up it checks every file in the cache, logs one line when all are there, and otherwise downloads only the missing ones after one warning that says how many files and GB. It first checks the room: the missing bytes, ZeroGPU's packed copy still to be written and a 10 GB margin must fit under the 150 GB cap, or start-up stops before writing a byte, naming the remedy (remove the preload block, so start-up downloads everything into an empty disk, as the first deploy did). How long a cold start takes with the preload: deploy 2's check on the Space, on issue #32. The disk arithmetic:ZEROGPU_DISK.md, its deploy-2 block. The body of this section is the record of the private Space.
preload_from_hub cannot work for a PRIVATE Space, and this was found the
expensive way β by a RUNTIME_ERROR after a successful build.
Initialization step 'preload' failed with exit code: 1
GatedRepoError: 401 Client Error
Cannot access gated repo .../LTX-2.5-Diffusers/.../audio_vae/config.json
401 is unauthenticated, not unauthorised. The preload runs as a separate, platform-managed init step that does not receive Space secrets, so the request arrives with no credentials at all. Verified both directions:
| Space | privacy | repo gate | preload |
|---|---|---|---|
techfreakworm/LTX2.3-Studio |
public | gated=manual (stricter) |
works β RUNNING |
techfreakworm/LTX2.5-Studio |
private | gated=auto (looser) |
401 |
Privacy is the differentiating variable β not the gate, not the token. The same URL at the same revision fetches 200 by hand.
So the 37 files are downloaded at runtime by preflight.ensure_weights(),
inside the app process, where the token exists. Same
model_files.download_allow_list(), so the single-source property is unchanged.
This is the better design even if the diagnosis above is wrong, which is why
it was chosen over simply making the Space public: it removes the ownership bug
class entirely. Preloaded files belong to the build user and are read-only to
runtime uid 1000, so any later lazy fetch fails Permission denied (os error 13)
β the 2.3 failure. When the runtime user downloads its own cache, it owns it, and
that class cannot occur. A fix that holds even if the root-cause analysis is
wrong is worth more than one that depends on it.
6.2.1 The cost: a 72 GB refetch on every cold start
(Corrected 2026-11-17: no longer; the image brings the weights, see the head of Β§6.2.)
Measured on the live Space:
storage : None
gcTimeout : 172800 s = 48 hours
There is no persistent storage, so the full 72.186 GB is refetched on
every cold start β restart, rebuild, or 48 hours idle. startup_duration_timeout: 1h covers it. This is accepted for the development phase, not endorsed.
(Still so on 2026-11-16: storage None, gcTimeout 48 h. The cold start of 2026-11-13 took
13 min 59 s to serving, about 11.5 min of it this download.)
Two routes back to preloading, either of which restores it:
- Make the Space public β the plan, once every mode is verified.
- Attach persistent storage β the cache survives, so the refetch stops mattering.
The preload_from_hub block is therefore kept in README.md, commented,
with its re-enable conditions beside it and a drift test asserting it still
matches model_files.preload_from_hub_entry(). A commented block drifts more
readily than a live one β nothing exercises it, so nothing ever points at it, and
it is trusted on exactly the day it is re-enabled.
6.2.2 Original strategy, superseded
preload_from_hub: with the file list generated from model_files.py (Β§5.2), plus a runtime
fallback that fetches anything missing.
Preloading bakes the weights into the image so a restart does not re-fetch 72 GB β which matters because ZeroGPU's ephemeral disk is wiped on every restart/rebuild (Β§2). The runtime fallback exists because a preload that silently drops a file must not become a mysterious load failure.
Three hazards to code around:
Only live shards (Β§2.1 Trap A). A wrong list fails after tens of GB, not fast. Mitigated structurally: the list is generated, and a drift test asserts README ==
model_files.py.Capped at 10 entries. Not a problem β we are one repo, so one entry, with comma-separated filenames. It fits in ~1.5 KB. (A naive one-file-per-entry list would not: 37 files.)
Ownership. Preloaded files belong to the build user and are read-only to runtime uid 1000, so a later lazy
hf_hub_downloadinto the same cache fails withPermission denied (os error 13).β οΈ DO NOT move
HF_HOMEto dodge this. An earlier draft of this plan said to pointHF_HOME/HF_HUB_CACHEat a writable location. That advice was wrong and would have caused the exact failure it was meant to prevent. The Spaces config reference warns, verbatim, onpreload_from_hub:"Files are saved in the default
huggingface_hubdisk cache~/.cache/huggingface/hub. If your application expects them elsewhere or you changed yourHF_HOMEvariable, this preloading does not follow that at this time."Moving
HF_HOMEmeans the 72 GB preload lands in the default cache, the app looks elsewhere, finds nothing, and re-downloads 72 GB at runtime β on top of the preloaded copy, which on a 150 GB cap is a near-certain eviction.Correct approach: leave
HF_HOMEalone, read from the default cache, and avoid ever needing a write. See Β§6.2.1.Related and reassuring: ZeroGPU's packer deletes the HF cache blobs after packing anyway (Β§2.5). Treat the HF cache as transient by design β it is not a durable store, and any code that assumes a file stays there after startup is wrong.
Revision. The entry format is
repo_id file1,file2,β¦ commit_sha. The SHA is not optional.model_files.pypinsREVISION; if the preload fetchesmaininstead, those are different cache keys, the preload is wasted, and the app re-downloads everything. The drift test therefore compares the whole entry string including the revision, not just the file set β anddownload_allow_list()returns sorted paths so that comparison is stable.Startup timeout.
startup_duration_timeoutdefaults to 30 minutes. Downloading and packing 72 GB will plausibly exceed that; the Space frontmatter sets1h. If the first deploy still times out, this is the first thing to raise β not the second.
6.2.1 The completeness invariant β the real fix for the permission problem
If every file we ever load is in the preload set, nothing needs to be fetched lazily, and the read-only-cache problem cannot arise. That is a stronger position than mirroring, so we make it an explicit, checked invariant rather than a hope:
At startup, verify every path in
download_allow_list()is present in the cache. If any is missing, fail loudly with the specific filename AND its URL rather than silently triggering a runtime download.
This converts the whole 2.3 permission-bug class into a diagnosable startup error. It lives in
preflight.py (Β§5.6) and it is a test case.
Mirroring stays documented as the fallback if a lazy write ever does prove necessary: hardlink
blobs so it is zero-copy, byte-copy anything under refs/ because the HF library rewrites those,
and preserve snapshot symlinks as relative symlinks. But needing it at all should be treated as a
bug in the allow-list, not as normal operation.
6.3 β Main gate RESOLVED Β· β οΈ one secondary gate still shut
Lightricks/LTX-2.5-Diffusers β OPEN. The gate was accepted and access is verified:
model_index.json returns HTTP 200 with the operator's token. Weights work is unblocked.
Still shut: the Lightricks LoRA repos. Verified separately β LTX-2.5-22b-IC-LoRA-* and
LTX-2.3-22b-IC-LoRA-* are each gated on their own acceptance and the token does not have
them. This is not a blocker: Β§4.5 shows the ungated community LoRAs are a better R3
fixture anyway, because they are what users actually upload. Recorded so nobody is surprised
when an official IC-LoRA 403s. (Checked in August; not re-checked since.)
Local reality check: the gate being open does not mean we can generate locally. The 22B
model does not run on macOS. Real generations happen on the Space, under the quota discipline
in docs/TEST_PLAN.md.
β Corrected 2026-11-16: it runs on Apple silicon now. With a separate PyTorch 2.15 environment, the Q8_0 GGUF transformer and staged residency (the text encoder and the transformer are never resident together), all eleven modes are verified on real weights on a 128 GB Mac (
SPACE_MODE_MATRIX.mdΒ§10.2; how to run it: the README). Upscale Γ2, the diffusion decoder and int8 still need CUDA, so they are checked on the Space only.
7. Architecture
7.1 Module layout
app.py # entrypoint. `import spaces` FIRST (Β§8.2), then build + launch.
ltx25_studio/
config.py # every LTX25_* setting, resolved and validated: GPU size, duration,
# quantization, device, retention cap (Β§2.7), paths
device.py # LTX25_DEVICE (cuda/mps/cpu), dtype and int8 per device, the MPS
# numerics self-test, RoPE on the CPU for MPS
model_files.py # Β§5.2 β THE single source of truth: repo ids, revisions (one per
# component where they differ), subfolders, filenames, sizes
errors.py # typed exception hierarchy; every error carries an actionable detail
preflight.py # weights complete (or the local runner's files present);
# Β§1.3 β the diffusers-version fail-fast assertion. No RAM check
memory.py # local runs: committed memory + the next step vs a limit (Β§5.6)
local_runtime.py # the local StageRunner on mps/cpu: GGUF transformer, staged
# residency, supplied through one seam (Β§8.7)
zerogpu.py # Β§8 β the `spaces` shim, the static @spaces.GPU policy, the
# duration callable, contextvar propagation, and the
# "never return CUDA tensors" boundary
constraints.py # Β§5.5 β dims/frames/CRF/sigmas. Pure functions over numbers.
pipelines.py # construction on CPU, module-scope .to("cuda"), component wiring,
# and the vae.temporal_compression_ratio tripwire (Β§5.5)
conditioning.py # UI inputs -> list[LTX2VideoCondition]; pixel->latent index
# conversion (Β§5.7 #1); uint8/PIL normalisation (Β§5.7 #7)
audio.py # waveform <-> mel <-> audio_vae latents; params from config (Β§5.4);
# vocoder decode
audio_conditioning.py # supplied audio held fixed or varied by a level, on its own schedule
modes.py # Β§3.3 registry: 11 UI modes as data over the 4 real workflows
generate.py # Β§7.4 stage graph. OWNS the never-empty invariant (Β§5.3) and the
# per-run output directory
enhance.py # Β§3.5 β the Gemma prompt rewrite. Loaded on first use and kept
# for the worker's life, INSIDE the @spaces.GPU worker (never at
# module scope β that is the packing window); weight prefetch
outputs.py # mux video+audio, single-track export, 20 GB oldest-first eviction
lora/
inspect.py # header-only safetensors parse; metadata + convention detection
validate.py # validator stages 1-6.5 (Β§4.3)
apply.py # load_lora_weights, stage-7 attach assertion, both-component fusing
fetch.py, keys.py, manifest.py, staging.py
# Hub fetch + bounded cache, where each key lands (diffusers'
# renames), the shape manifest stage 6.5 checks, staged adapters
ui/
blocks.py # Gradio Blocks assembly
components.py # reusable widgets (condition list, LoRA picker, constraint hints)
handlers.py # thin adapters: UI event -> one call -> UI update. No logic.
fields.py, render.py, chrome.py, showcase.py, uploads.py
# field specs, user-facing strings, styling, examples and samples,
# app-owned copies of uploaded LoRA files
(Corrected 2026-11-16 in place: device.py, memory.py, local_runtime.py,
audio_conditioning.py and the extra lora/ and ui/ modules were missing; device selection
moved out of config.py, the quantization switch is not in model_files.py, and preflight.py
has no RAM check.)
7.2 Layering rules
- Dependencies point inward.
ui/may import anything;generate.pymust not importui/.errors.pyimports nothing from the package at all;model_files.pyandconstraints.pymay importerrorsand nothing else. An architecture test asserts it, because the value of a bottom layer is entirely in what it refuses to depend on. modes.pyis data, not branching. Adding a mode is a registry entry, not anifingenerate.py.ui/handlers.pycontains no business logic. Marshal inputs β call one function β marshal outputs. This is what makes the app testable without a GPU and without Gradio.- Errors cross layers as typed exceptions. The UI is the only place that renders one as a
string.
generate.pynever returns an error-shaped success (Β§5.3).
7.3 Why this shape
The predecessor put backend, models, modes and workflow in four flat modules with ui.py
importing all of them. That works until the failure modes get interesting: the empty-output bug
(Β§5.3) lived precisely in the seam between "workflow returned something" and "UI rendered it",
and there was no layer boundary at which to assert. Here generate.py owns that invariant and
can be tested for it directly.
7.4 The generation stage graph β classic pipelines PLUS an explicit diffusion decode
This is the most consequential architectural decision in the document, because the obvious implementation silently discards 2.5's headline quality feature.
LTX25AutoBlocks docstring, verbatim:
"Auto blocks for LTX-2.5 supporting text-to-video, image-to-video, condition-to-video and in-context (IC-LoRA) generation (joint video + audio). Identical to
LTX2AutoBlocksexcept that the video decoder isLTX2DiffusionVaeDecoderStep, since the diffusion decoder is the native default from LTX-2.5 on. To decode with the convolutional VAE instead, swap the decode block."
But the classic LTX2ConditionPipeline / LTX2ImageToVideoPipeline decode with
self.vae.decode(...) β the convolutional VAE, NOT the diffusion decoder. So a naive classic
call silently gives up what the model card calls "a new diffusion video decoder [that] replaces
the VAE reconstruction stage". It would not error. It would just be worse, forever, and nobody
would know why.
Decision: classic pipelines, with the diffusion decode added as an explicit stage.
β Corrected 2026-11-16: as built until today, only half of this happened. The diffusion decode was wired into the stage-2 branch alone, and stage 2 is off by default, so every default run decoded with the convolutional VAE β exactly the silent loss described above. Now the decoder is a per-run choice,
LTX25_DECODER(autoby default):autodecodes with the diffusion decoder where it can run β CUDA, its kernel available, a video large enough for it β and with the conv VAE otherwise, and the run says which. Modes that discard their video always use the conv VAE. The decoder's weights are pinned to the vendor's corrected upload (aba1b702, Β§5.2). None of this has run on the Space yet; off CUDA (a Mac) every run decodes with the conv VAE.
stage 1 LTX2ConditionPipeline(..., sigmas=DISTILLED_SIGMA_VALUES,
output_type="latent")
β latents, audio_latents
optional LTX2LatentUpsamplePipeline(latents)
β upsampled latents
stage 2 same pipeline, latents=upsampled,
sigmas=STAGE_2_DISTILLED_SIGMA_VALUES,
noise_scale=STAGE_2_DISTILLED_SIGMA_VALUES[0],
output_type="latent"
β refined latents
decode V LTX2VideoDiffusionDecodePipeline(latents=..., denormalize=False) β see below
decode A audio_vae.decode(audio_latents) β mel β vocoder(...) β waveform
denormalize=False is MANDATORY. output_type="latent" returns already-denormalized
latents (Β§5.7 #6). Passing the default True double-denormalizes and silently corrupts the
output. This is a test case.
The model is loaded ONCE and every workflow shares it
LTX2Pipeline and LTX2ConditionPipeline take identical weight-bearing
components at the pinned revision β scheduler, vae, audio_vae, text_encoder, tokenizer, connectors, transformer, vocoder, processor, prompt_enhancer, duration_head. The only difference is that the condition pipeline additionally
accepts audio_scheduler: a scheduler, zero weights, defaulting to None.
Calling from_pretrained per workflow therefore materialises two complete
independent copies of the model, ~67 GiB each:
| host | consequence |
|---|---|
| ZeroGPU | |
| a 96 GiB card | outright OOM on the second build |
So pipelines._base_pipeline() loads once and derive_pipeline() builds the
other workflow from those same modules. There is no configuration in which two
copies are wanted, which is why this was fixed before the measurement confirming
its magnitude landed β the fix is correct in every branch.
Settled by inspection, not assumption. diffusers/pipelines/pipeline_utils.py
at the pinned revision has no module-level cache of any kind β no lru_cache,
no loaded-module map β and load_sub_model calls load_method(...)
unconditionally, so every call materialises fresh. Our own _CACHE was keyed on
the workflow name, so it never deduplicated across the two.
β οΈ Necessary, and NOT sufficient for ZeroGPU
This fix removes one of three causes of the 150 GB eviction. It does not make
the Space work, and recording it as if it did would be the same confidently-wrong
statement as the expected_weights=53.20GB line deleted in Β§2.6.
The packed copy is separate from the HF cache snapshot, and the prune that would reclaim it runs only after packing completes. The authoritative arithmetic is Β§2.5; the short version is that even the fixed, single-pipeline case peaks at 144.37 GB before the image is counted, and 152β156 GB with it, against a 150 GB cap.
So one pipeline does not fit either. Double-loading took it to 216.56 GB; the Xet chunk cache would have taken it to 288.74 GB. The eviction was over-determined β three independent causes, and the smallest of them absorbed two cycles of tuning.
Settled, not open. ZeroGPU does free the snapshot β verified in spaces 0.51.1 β and it
frees it too late, because the unlink follows _pack(). So bf16 on storage: None is
structurally infeasible for this checkpoint set, and the routes are persistent storage, a
public Space with preload_from_hub (Β§6.2), or smaller checkpoints. All three are operator
decisions; none is a knob we can turn.
The route list was incomplete, and the missing entry is the one that shipped (annotated 2026-08-25). Int8 runtime quantization (
ZEROGPU_DISK.mdOption B) was a knob we could turn: it shrinks the packed copy while the snapshot stays bf16, it was turned, and the deployed Space fits β disk 82.2 GB of 150 [OK], prune freed 65.25 GB, all eleven modes generating at sha287ba9c. The finding itself stands: bf16 onstorage: Noneremains structurally infeasible β which is exactly why the live Space is not bf16.β Corrected 2026-11-16: "82.2 GB of 150 [OK]" is a reading taken before packing (the platform's progress bar read
40.6Gpacked on the 2026-11-13 boot; the disk peak was never measured), and at287ba9cthe eleven modes executed: every output up toff142d9was colour static.
The base is always the superset class, LTX2ConditionPipeline. Deriving
superset β subset can only drop audio_scheduler, which the subset does not
take. The reverse would silently drop it from the pipeline that does use it,
and a missing audio scheduler degrades the joint audio track in a way the video
output never reveals β this build's recurring failure shape.
The tests assert module identity (is), not equality. Two copies work
perfectly, right up until the disk or the card runs out, so any check that only
asks whether both pipelines function is blind to it.
Stage 2 is opt-in, and the registry overrides the request. GenerationRequest.enable_stage_2
defaults to off: it is a real quota cost on a metered Space and the user should own that
trade. But a mode whose skip_stage_2 is set generates video nobody sees (Β§3.2), so upscaling it
is pure quota β generate.stage_2_enabled() forces it off there, because honouring the toggle
would be honouring a mistake. StageRunner owns the graph so generate.py carries no branch on
it, and requesting stage 2 without the upsampler loaded raises rather than quietly rendering
single-stage, which would look like it worked and be visibly worse. (2026-11-16, c712e7e: the
local runner is a StageRunner too, so a Mac runs this same graph and supplies only the
residency, Β§8.7.)
Why not the modular path. The model card's own Option C example uses the classic pipelines,
so that is the documented, lower-risk API. The modular path (LTX25ModularPipeline /
LTX25AutoBlocks, loaded from modular_model_index.json) is newer, and its interaction with
ZeroGPU's tensor packing (Β§8.3) and with load_lora_weights (Β§4) is unverified. Given
LoRA is the headline feature and ZeroGPU is the deploy target, betting both on an unverified
integration is the wrong trade.
The modular path is recorded here as the canonical alternative: if we move to it, the decode
block is swappable exactly as the docstring describes, and vae/ (1.452 GB) stays in the subset
as the conv-VAE fallback either way. (Corrected 2026-11-16: until today the conv VAE was not a
fallback but the decoder of every default run; see the correction under "Decision" above.)
8. ZeroGPU execution model
This section replaces the conventional wisdom about diffusion-model offloading. On ZeroGPU that wisdom is not merely suboptimal β it is counterproductive.
8.1 The hardware is not what the API says
ZeroGPU is NVIDIA RTX PRO 6000 Blackwell Server Edition, partitioned via MIG (sm_120).
| tier | VRAM | quota |
|---|---|---|
large (default) |
48 GB | 1Γ |
xlarge |
96 GB | 2Γ |
The Spaces API still reports the legacy id string zero-a10g. Do not read that as an
A10G β it is a stale identifier, and sizing decisions made from it would be wrong by a factor
of several.
VRAM is not our constraint. 67.2 GiB of weights against xlarge's ~95 GiB leaves ~28 GiB
for activations. The constraints are CPU RAM and transient disk (Β§2).
β Corrected 2026-11-16: only on
xlarge, and the Space did not getxlargeuntil 2026-08-26 (autonever promotes, Β§8.6). On the measured 47.375 GiBlargepartition, stage 2 at 704Γ480Γ49 (55.906 GiB reserved) could not fit. Onxlarge(94.971 GiB, now the code's default) the largest peak measured leaves ~39 GiB.
8.2 Import order is load-bearing
import spaces # MUST be first
import torch # only after
import diffusers # only after
import spaces must precede torch / diffusers / transformers, or startup fails with
"CUDA has been initialized before importing the spaces package". This is why app.py is thin
and why the spaces shim lives in zerogpu.py with a guarded import for local dev.
8.3 The packing model: build on CPU, .to("cuda") at module scope
ZeroGPU replaces CPU offload with its own disk-packing engine. The correct pattern inverts normal GPU-box practice:
- Build the pipeline on CPU, at module scope.
- Call
pipe.to("cuda")at module scope. That call is intercepted, rewritten to.to("cpu"), and every tensor is packed to disk (~/.zerogpu/tensors, O_DIRECT). After packing, the main process holds no weights. - Apply
@spaces.GPU(...)statically at module import. Dynamic or inner decoration fails ZeroGPU's startup analyzer with "No @spaces.GPU function detected during startup". - Inside the decorated function real CUDA is live, and a forked worker streams disk β pinned host memory β VRAM.
HF's guidance, quoted: models must be placed on cuda at the root module level.
β Corrected 2026-11-16: step 2's timing is wrong, and Β§3.5 has it right. The module-scope
.to("cuda")registers the tensors; the packing itself runs once, atdemo.launch()(torch.pack()inspaces/zero/__init__.py). The 2026-11-13 boot log shows it: a disk reading labelled "after packing" withpacked 0.0, then the platform's own progress bar packing40.6G, then "Cleaned 65.25GB of tensor files β¦ after packing" 29 s later. The startup report was relabelled accordingly (Β§5.6).
β οΈ The snapshot is freed after packing β but only if the mmap survives
Read from spaces 0.51.1 (zero/torch/patching.py:395) rather than inferred:
with unmap_capture(mmap_addrs) as unmapped_paths:
total_size = _pack(Config.zerogpu_offload_dir)
for path in unmapped_paths:
path.unlink(missing_ok=True)
unmap_capture reads /proc/self/maps, maps each tensor's data_ptr() back to
the blob file it is mmap'd from, and unlinks those matching
~/.cache/huggingface/hub/models--*/blobs/*.
So the 72 GB snapshot IS reclaimed β conditionally. Two preconditions, both easy to break and both silent when broken:
- Tensors must still be mmap'd from the blobs when packing runs. A dtype
cast, a
.clone(), an eagerlow_cpu_mem_usage=False, or a non-safetensors path materialises them into anonymous memory, leaving nothing in the map to capture. Nothing is freed, 72 GB stays forever, no error and no log line. This is whytorch_dtypemust equal the checkpoint's on-disk dtype rather than being a free parameter β recorded at the risk site inpipelines.get_pipeline. - The cache must live under
~/.cache/huggingface/hub, or the autoprune pattern stops matching. This is the sharper reason not to moveHF_HOMEβ Β§6.2 gives the preload reason, and this one is worse because it fails silently rather than loudly.
And the unlink runs strictly AFTER _pack() completes, so both copies
coexist for the whole packing window β which is the peak in Β§2.5, the authoritative arithmetic.
Steady state afterwards is fine at ~72 GB. It is the peak that evicts us, not
the residency.
Observed 2026-11-13, recorded 2026-11-16: under int8, 1344 transformer and 331 text-encoder parameters are materialised into anonymous memory (the app logs a warning for them), and the boot still cleaned 65.25 GB. So precondition 1 does not hold per tensor; most likely a blob is unlinked while any of its tensors is still mapped. Not verified beyond that one log.
Corollary: do NOT lazy-load models inside the decorated function. HF documents that
placements are optimised for startup; lazy loading defeats the packer entirely. (As built,
two exceptions are deliberate: the stage-2 parts and the prompt enhancer load lazily inside the
worker, to keep them out of the packing window, and pay a per-worker load instead (Β§3.5).)
(Corrected 2026-11-17: the diffusion decoder is no longer one of them. app.py builds it once at
start-up on CUDA, beside the pipeline (pipelines.build_decoder_at_startup), so ZeroGPU packs it
and every run's worker uses that instance; a start-up build that fails logs one line, and each
run builds its own as before. The latent upsampler and the prompt enhancer still load inside
the worker.)
Considered and rejected: deferring the second workflow
Building only one pipeline at startup and the other on demand would halve the packed footprint, and the disk arithmetic in Β§7.4 makes that look attractive.
It is a worse fix, and worse in a way visible only on ZeroGPU. A pipeline built lazily inside the decorated function is never packed, so it is re-transferred from RAM on every cold worker β trading a one-off disk cost for a per-request cost, on the platform where cold starts are the normal case. It would show up as a win in the disk numbers and a loss in every generation.
from_pipe gets both properties at once: both pipelines exist at module scope so
both are packed, and they share module objects so there is only one set of tensors
to pack. Strictly better on both axes.
Recorded rather than dropped because the arithmetic makes it an obvious idea, and the next person to have it deserves the reason it does not work rather than silence.
8.4 enable_model_cpu_offload() must NOT be used
This supersedes an earlier draft of this plan, which called it mandatory. It is the opposite.
enable_model_cpu_offload() and enable_sequential_cpu_offload() are counterproductive on
ZeroGPU, for a specific mechanical reason: any submodule accelerate parks on CPU is never
registered for packing. Consequently it β
- stays resident in main-process RAM (which is our tightest budget, Β§2),
- is fork-inherited by every worker, multiplying that cost, and
- is re-copied hostβdevice on every hook fire, on top of the packer's own streaming.
So we pay RAM we do not have, to duplicate work the platform already does better. The
model_cpu_offload_seq string recorded in Β§5.7 is upstream metadata we do not act on.
8.5 Never return CUDA tensors across the @spaces.GPU boundary
Returning a CUDA tensor from a @spaces.GPU function makes unpickling in the main process
trigger a blocked lazy CUDA init, and the call hangs β no error, no timeout message, a
dead request.
Rule: return file paths, or .cpu() tensors. Always. zerogpu.py owns this boundary and it
is a test case.
8.6 Tier and quota
We declare size="xlarge" explicitly rather than being silently promoted.
Packed bf16 weights (67.2 GiB) exceed xlarge_threshold = 42.00 GiB, so ZeroGPU auto-promotes
us to xlarge and doubles quota cost with no code change (Β§2.6). Declaring it explicitly means
the cost is visible in the source rather than being a surprise in the quota dashboard.
β Neither premise survived deployment (annotated 2026-08-25). We have never declared a size: every deploy has run
LTX25_ZEROGPU_SIZE=auto, soapp.pysendssize=Noneand the explicit-xlargebranch has executed zero times on the platform (OPERATOR_HANDOFF.mdΒ§4 gap 0). And the live Space no longer packs bf16 β it packs ~51.1 GB of int8 weights. The conclusion holds on the new number (51.1 GB > 42.00 GiB βxlarge), β οΈ by inference: the promotion is deduced from the threshold, not read back, because the endpoint that would confirm it is the same one that 404s for quota.β The inference was wrong, and this annotation came within one step of catching it (2026-08-27)
It correctly flagged the promotion as β οΈ deduced, not read back, and then accepted the deduction because the only way it could imagine reading it back was an endpoint that 404s. There was another way, and it was cheap: print
torch.cuda.get_device_properties().total_memoryfrom inside the worker.0a4ddbbeventually did exactly that for unrelated reasons and the answer was β 47.375 GiB βlarge. Notxlarge.autosends nogpuSizeat all, and the promoted value never leavesget_duration_seconds.β
LTX25_ZEROGPU_SIZE=xlargeis now set, and the same probe reads 94.971 GiB. Β§8.6's original instruction β declare the size explicitly rather than being silently promoted β was right on the first try and was talked out of by a threshold reading. Trace:MIG_ALLOCATOR_SIGHTINGS.mdΒ§1.β And since 2026-11-16 it is the code's default:
LTX25_ZEROGPU_SIZEdefaults toxlarge(config.DEFAULT_ZEROGPU_SIZE), so a Space that loses the variable still asks for the full GPU. The int8 packed size the 2026-08-25 annotation estimated at ~51.1 GB was40.6Gon the platform's own progress bar (2026-11-13 boot).
Quota arithmetic: PRO gives 2400 s/day. At duration=75 on xlarge (2Γ), each run is
charged 150 s β **16 runs/day**. Dropping under 42.00 GiB via quantization would return us to
large at 1Γ β that is the entire quota argument for Β§2.6's fallback, and it is worth
approximately double the runs.
β Corrected 2026-11-16: both halves of that paragraph are wrong. (1) The tier follows the
sizepassed, not the packed size: the Space is quantised and explicitlyxlarge, so dropping under 42.00 GiB would buy nothing. (2) The reservation is not 75 s. It iszerogpu.estimate_duration: the baseLTX25_GPU_DURATION_SECONDS(180 s by default and on the Space), Γ 1.5 with stage 2, + 60 s with LoRAs, + 180 s with prompt enhancement, so 180 to 510 s. Billing is the GPU time actually used Γ 2 onxlarge(Hugging Face's documented rate; not measurable here, no quota endpoint answers), so a runs-per-day figure depends on what each run uses and is not stated here. To start, a run needs its reservation Γ 2 of quota free.β Corrected 2026-11-17: the reservation follows each run's size and length now.
zerogpu.estimate_durationis a fit of the Space's measured runs (SPACE_MODE_MATRIX.mdΒ§10.3) with a 1.3 margin per term, rounded up to 5 s: the default 704Γ480, 5 s run reserves 60 s and needs 120 s of quota free to start (it was 180 s and 360 s); stage 2 adds 45 s at that size and more for larger or longer runs, adapters 40 s, the enhancer 80 s. On the Space a run past the measured envelope (64,000 latent tokens in a pass, 520 million returned pixel-frames) is refused before any GPU time.LTX25_GPU_DURATION_SECONDSis ignored (start-up says so once);LTX25_GPU_DURATION_FLOOR_SECONDS, unset by default, raises every reservation to at least its value.
Two further traps from Β§5.8: the hidden 1.5Γ duration_factor on large (so duration=60
schedules as 90 s; xlarge skips the factor but doubles the request instead), and the fact that
the maximum duration is a server-side, tier-dependent, unpublished cap β discover it, never
hardcode it. ~120 s is the empirical safe ceiling; the official Lightricks LTX Space ships
@spaces.GPU(duration=75, size="xlarge"). (Corrected 2026-11-16: this app now asks for
180β510 s, see above; the server's maximum is still not known.)
8.6.1 Where Β§8 actually executes β and why the decorator is unconditional
Everything above is a rule about app.py, and for a while every one of them existed only as
prose: no executable @spaces.GPU, no executable .to("cuda"), and zerogpu.py imported by
nothing. The app would have deployed, started, and never touched a GPU.
What it looks like now, and each line is load-bearing:
import spaces # FIRST, before anything that initialises CUDA
CONFIG = load_config() # unguarded: a bad LTX25_* reaches the operator
preflight.run(CONFIG, strict=ON_ZEROGPU)
pipelines.get_pipeline(..., device="cuda") # module scope -> the packer runs
_GPU_KWARGS = zerogpu.gpu_decorator_kwargs(spaces.GPU, duration=..., size=...)
@spaces.GPU(**_GPU_KWARGS) # literal, static, UNCONDITIONAL
def run_generation(**params):
return zerogpu.assert_returnable(_GENERATE(**params))
zerogpu.check_duration_signature(run_generation, zerogpu.estimate_duration)
generate_module.set_gpu_dispatch(run_generation)
The decorator is unconditional on purpose. Off-Space, spaces is a stub whose GPU() returns
the function unchanged, so the wiring is identical everywhere and only its effect differs. A
decorator behind an if is precisely how a documented rule quietly stops being applied: the Space
path stops being the path anyone runs, nothing local exercises it, and the docstring keeps
reassuring. Making the stub carry the difference means the line you run locally is the line that
runs on the Space.
_GENERATE is bound at import rather than resolved by attribute, because a wrapper installed
after startup would otherwise send the worker back through itself.
8.7 Local (Apple Silicon / MPS) is a different regime
MPS has no ZeroGPU packer, no spaces decorator semantics, and unified memory changes the
arithmetic entirely. Device strategy is selected in config.py, the spaces import is guarded,
and the local path is expected to be capability-limited rather than identical. Do not assume
a code path that works on ZeroGPU works on MPS, or vice versa.
As built, 2026-11-16. The device is chosen in
device.py:LTX25_DEVICE=auto(CUDA where it exists, otherwise no generation device: the UI starts and Generate says how to opt in) |cuda|mps|cpu. Onmps/cpureal weights go tolocal_runtime.py, whoseLocalStageRunneris apipelines.StageRunner(sincec712e7e): the stage graph β the decode policy, stage 2 and its refusals, the audio conditioning, the clip variation, the run facts β is one piece of code for the Space and the Mac, andgenerate.pydrives both runners the same way. The local runner supplies only residency, through one seam around every pipeline call: the GGUF transformer through diffusers' own loader, staged residency (text encoder and transformer never resident together; the prompt is encoded first and passed as embeddings), RoPE computed on the CPU in float64, and a memory preflight before every heavy step (memory.py). An MPS numerics self-test refuses a PyTorch which computes wrongly on this GPU (2.11 does; 2.15.0 is what was verified). Capability-limited as predicted, by the same rules on both runners: off CUDA the default decode is the conv VAE and the run says why, and Upscale Γ2 or an explicit diffusion decode runs only at very small sizes and is refused before dispatch at real ones; bitsandbytes int8 needs CUDA. Verified on real weights for all eleven modes (SPACE_MODE_MATRIX.mdΒ§10.2); most of those runs predatec712e7e, and the lip-sync runs (aa7ff84,482800d) ran on it, as did the final pass, which repeated every mode one59944e(three of its runs onba2d582): every output compared with a run made after the RoPE fix was identical to it.
9. Testing strategy
We run an H3 TDD harness. The gate is:
.venv/bin/python -m pytest -q
Ownership: a separate teammate owns tests/**. The implementer does not create or edit
anything under tests/. This plan's job is to make the units testable.
9.1 Testable without a GPU, without weights, without network
| unit | how |
|---|---|
constraints.py |
pure functions over ints β dimension snapping, frame-count validation (with an injected compression ratio), CRF selection from a stubbed model_type, sigma schedule shape |
model_files.py |
data assertions: every entry has repo/revision/subfolder/filename; no duplicates; the Β§2.3 subset sums to 72.185; the dtype switch enumerates the three Β§2.6 configs |
preflight.py |
inject fake free-RAM/free-disk numbers β assert a specific error naming both numbers; assert the diffusers-version check fires on a wrong version |
lora/inspect.py |
synthesised safetensors headers β small fixtures, no real LoRA (this is why Β§4.3 stage 2 is header-only) |
lora/validate.py |
stages 1β6 against crafted good/bad headers, including the wrong-architecture case |
modes.py |
registry completeness (11 modes over 4 workflows); each mode's kwarg builder produces expected kwargs |
errors.py |
every error type's message contains the actionable detail (filename, URL, found-prefixes, measured numbers) |
conditioning.py |
pixelβlatent index conversion (Β§5.7 #1); out-of-range raises (Β§5.7 #2) |
outputs.py |
20 GB oldest-first eviction over a temp dir |
generate.py |
stub pipeline returning nothing β assert a specific exception, not an empty path |
zerogpu.py |
the return-boundary rejects/converts CUDA-tensor returns; duration= callable signature matches the decorated function |
9.1.1 The lesson this build actually taught
A module can be complete, documented, specced and green, and still be connected to nothing.
And it can be imported, and still be connected to nothing that matters.
The corollary is the part that took five separate blockers to learn, every one of them found after the gate first went green:
| # | What was disconnected | How it looked |
|---|---|---|
| 1 | generate β handlers disagreed about what success is |
every successful run in the three flagship modes reported as an internal error |
| 2 | zerogpu wired to nothing |
the app would have deployed, started, and never touched a GPU |
| 3 | generate β its own GPU entrypoint |
the worker resolved generate by attribute and re-entered itself |
| 4 | conditioning reached only from a UI readout |
raw dataframe rows went to the pipeline; the pixelβlatent conversion never ran |
| 5 | lora/apply β importer count zero, 19 passing specs |
the entire LoRA validator was itself never applied |
#5 is the one to remember: the machine built to prevent silent wrongness was itself silently doing nothing, and 19 green specs said otherwise. Nine validation stages, a header-only parser, a per-module dimension check β a user would upload an adapter, watch it validate, generate, receive base-model output, and be told it worked.
They were not five unrelated mistakes. They were one mistake with five faces, and the reason
it recurred matters more than the instances: everyone verified the thing they had just built,
using a check they designed, against the mental model they already held. Unit specs passed.
Hand-written probes passed. AST scans and sys.modules checks passed. Every one of them tested a
belief rather than the system.
The only checks that found anything were the ones whose shape did not depend on the checker being right β seam tests driving real layer A into real layer B, and the evict-and-diff sweep that asks the program what it actually reached. Stated as the person who kept missing them:
I probed the path I had in mind rather than the paths other code takes.
The numbers make it concrete, and this is the load-bearing fact of the whole build: the gate was green at 1102 passing tests with all five blockers still in the tree. Nothing was failing. The app could not reach a GPU, could not apply a LoRA, and reported successful runs as internal errors β at 1102 passing, zero failing. It stands at ~1800 now, and the difference is almost entirely seam and reachability tests β not more unit coverage. More tests of the same shape would have found none of them.
Why the usual checks miss this class. Unit tests prove a module correct in isolation, which
is exactly what a disconnected module is. A probe you design tests the mental model you already
have β and a wrong mental model is precisely what a seam bug is made of. conditioning was
import app: yes and would have passed any "is it imported" check, because the UI's live readout
loaded it; it was the generation path that never touched it.
The check that does catch it, and the only one here that does not depend on the checker already suspecting the answer:
import app # 1. load the entrypoint
evict every ltx25_studio.* that import loaded # 2. clear the slate
run one real generation # 3. image + shot row + staged LoRA
diff sys.modules again # 4. what loaded is what the path reaches
Not "is it imported" but "is it reached by the path that does the work". It cannot miss a
module nobody remembered to think about, which is the whole point. tests/test_module_reachability.py
pins it.
9.1.2 Two rules for prose, since no test can fail on it
All four confidently-wrong artefacts on this build lived in the documentation
layer β expected_weights=53.20GB, weights cached : 37/37, "blows the
ceiling", and a Β§2.5/risk-#1 contradiction where the body stated a fact as known
and the risk table stated the same fact as unverified. Both sentences were
well-formed markdown and the gate has no opinion about prose.
Two rules, both cheap, both applied rather than merely recorded:
1. A claim stated in two places is a claim that will diverge. Where a number or a mechanism appears more than once, make one place authoritative and have the others point at it. Β§2.5 now carries that marker explicitly; Β§7.4, Β§8.3, the resource table and the risk register reference it instead of restating it.
2. Mark provenance on load-bearing claims β verified against spaces 0.51.1
source, measured on hinode, inferred. 53.20GB and 37/37 were both numbers
with nothing visible behind them, so no reader could tell there was nothing to
check.
And the rule that generalises furthest, from the contradiction above:
A claim moving from known β unknown is nearly always a regression.
Documents accrete facts and rarely lose them. When an edit is about to record something as uncertain, that is the moment to search the document for it being known β because the likeliest explanation is that someone already established it and the current author has not read it.
That is what happened here: the answer was already in PLAN.md, and it was
overwritten with "unverified" by an author who had the file open and searchable.
9.1.3 Process finding: history that misdescribes itself
git log -- ltx25_studio/outputs.py returns a commit whose message describes conditioning.py
and lora/apply.py. An uncommitted change by another agent was swept into 3d53dd3 by a
git add ltx25_studio/ that was not path-scoped.
Recorded because the next reader deserves the warning: history that misdescribes its own contents is a trap for whoever reads it next, and it is the same failure as documentation that reassures β just in the log instead of the source. The root cause was structural β three agents sharing one worktree with files reassigned mid-flight β so the mitigation is structural too: stage by explicit path, never by directory, whenever more than one agent can write.
9.1.4 Two more rules, both of the Β§9.1.1 shape
Neither is here because it sounds wise. Both are here because their shape does not depend on the author already suspecting the answer β the property Β§9.1.1 identifies as the only one that found anything.
3. Ask of every input: "if this were silently ignored, would the run still pass?"
Wherever the answer is yes, a green run is not evidence of anything. The mode still
renders, the file still carries the right streams, every check passes β and the field
under test was never read. Auditing the request fields on this build turned up five
such inputs (enable_stage_2, loras, auto_duration, noise_scale, strength), and
in every case the remedy is not a sharper assertion but a second run differing in
exactly that field. docs/TEST_PLAN.md Β§A.4.1 is authoritative for the per-field
table and the runs it implies.
The rule earns its place by having caught itself. The six comparison runs it
generated would every one of them have passed vacuously: seed=-1 means draw a fresh
seed, so both halves of each pair would have run at different seeds and differed for
that reason alone. Established by running it rather than by reasoning about it β two
runs at the default returned 777437012 and 3499445304. A rule about inputs that
are silently ignored was nearly defeated by an input that is silently ignored, which
is the most compact argument available for checking a test can fail before trusting that
it passed.
4. Record a fact once; derive every consumer of it.
Rule 1 says a claim stated in two places will diverge. That is not a property of prose β
it is a property of copies, and code is not exempt. model_files._SPECS records the
live shard sets once, read off the repo's own index files; both consumers are then
derived from it, the runtime download and the Space preload_from_hub: list alike,
through download_allow_list(). A drift test compares the committed README against that
function's output, so the two cannot separate quietly.
The payoff was not theoretical. The 37-file, 72.185 GB list was correct on hinode β a
machine nobody wrote it for β including the duplicate-shard trap it exists to dodge:
transformer/ ships a 4-shard and an 8-shard copy of the same weights, connectors/ a
2-shard and a single-file copy, and a naive fetch of those folders pays 44.33 GB for
nothing. A hand-maintained second copy would have been accurate about the machine its
author had in front of them β the only machine that never needed it to be right.
9.1.5 Four rules the hardware taught, which no amount of reading would have
Β§9.1.1 through Β§9.1.4 came from reading the code. These came from running it, and each cost a real defect to learn.
5. Fixtures must cover the union of what the callers pass, not the intersection that is convenient to construct.
_as_array is reached by two callers and only one ever passes the awkward type.
test_encoder_contract.py drove the real encoder through real containers
to a real file β and built its audio as np.float32, because that is what is
easy to make. The model produces bfloat16, numpy has no such dtype, and six
of eleven modes were broken at the last step while 1926 tests passed. The path
was covered. The dtype was not. "Cover the code paths" does not say this;
"cover what the callers actually pass" does.
6. A fake that cannot reproduce the failure is not a test of it β and a fake derived from the mapping can never catch a wrong mapping.
Three instances in one day, each in a test that was otherwise good:
- the
np.float32audio fixture above; - its replacement, whose
.float()returned anndarraywhere torch returns a tensor β written the convenient way it would have passed a broken fix; - the audio-VAE fake, which used our parameter names. All six were wrong, every runtime path raised on real weights, and every offline test passed β because the fake agreed with the code and with nothing else.
The instinctive repair for the third β build the fake from the same mapping the
code uses β is a trap for exactly the same reason. The fix is a recorded fact
about reality: REAL_AUDIO_VAE_CONFIG_KEYS, REAL_VOCODER_CONFIG_KEYS,
MODEL_CARD_DISTILLED, each copied from the vendor's source rather than derived
from ours. The fake checks semantics; the recorded fact checks the world.
Where a fake stands in for something unavailable locally, something must check
the fake still behaves like the real thing β that check found a defect in its
own fixture within a minute of existing.
7. Prefer a check to a comment, because a comment has no purchase on its own author five minutes later.
Twice a constraint did not survive the next function written by the person who
wrote the constraint: a dtype requirement violated two commits after the β οΈ
comment demanding it, and a 44.33 GB refetch reintroduced while reading the
docstring that predicts it. A comment is a message to a future stranger.
Make the wrong thing unsayable instead β derive the value once and share it,
as snapshot_dir() is shared by the loader and the completeness check, so the
two cannot drift.
8. A refutation from source is only a refutation of the mechanism it examined.
video_to_audio came back 50 dB down. Two candidates were proposed; both were
killed from the pinned source with line numbers, correctly β the conditioning
mask is applied to video only, and the strength ceiling never binds. The
conclusion drawn was too broad. The cause was condition strength, by a
mechanism neither of us had considered: strength sets where in the sigma
schedule the run starts, so the audio stream β which nothing anchors β was never
denoised at all. The refutation was of the denoise loop; the effect was in the
initialisation.
When a refutation kills a candidate, say which mechanism was examined and which were not. Neither party could have reached this alone: source reading eliminated the wrong path with certainty and could not find the right one; measurement found the right one and could not say why.
9. Source reading says what cannot be true. Measurement says what is. Neither alone says why.
The sharpest instance, and it cost a commit and a revert. video_to_audio came
back 50 dB down.
- Source established that a video condition cannot attenuate audio in the
denoise loop β the conditioning mask is applied to
noise_pred_videoand never touchesnoise_pred_audio. β Correct, and it stayed correct. - Measurement produced a clean single-lever sweep at fixed seed: strength 0.8 β β55.0 dBFS, 0.5 β β5.1. β Also correct.
- The synthesis was wrong. The sweep measured a real interaction between condition strength and a separate, broken guidance configuration β a genuine effect inside a pipeline that no longer exists. Fixing the actual defect made H9 healthy at the unchanged strength of 0.8.
A single-lever same-seed sweep is strong evidence that something moves, and no evidence about why. Treating a clean correlation as a mechanism, against a source citation, is the error β and the citation was doubted because the correlation was tidier.
The corollary is about sets, not single measurements. A cross-section comparing several misconfigured outputs to each other argued the guidance was innocent, because they all sorted plausibly. Comparing a set of misconfigured outputs to each other cannot reveal a defect they all share. Only the intervention could β and when it ran, all twelve moved. That is the same shape as one artifact being rejected as a control because it was pinned to the rail: a baseline has to be established, not assumed from internal consistency.
10. At a hand-off between our code and a model, every attribute that can disagree eventually will β and fixing them as they surface is not the same as enumerating them.
host-runner's words, after the fourth one. One call site, mel -> audio_vae.encode, and every attribute of that tensor was wrong in turn:
| attribute | how it failed | |
|---|---|---|
| 1 | dtype, outward | _as_array calling .numpy() on bf16 β numpy has no such dtype |
| 2 | dtype, inward | float32 mel into a bf16 conv_in |
| 3 | rank + axis order | unbatched, mel-major, where the encoder wants batched and time-major |
| 4 | device | CPU mel into CUDA weights, at the same line as (2) |
Each was found by running it, none by the offline gate, and each fix was correct and incomplete because it addressed the attribute that had crashed.
The crashing end announces itself and the quiet end does not. Every one of these raised a clear exception naming the thing that was wrong, which is exactly why the fix kept being scoped to it.
_as_arrayraised aTypeErrornaming bfloat16; nobody asked what happened on the way in.
The mitigation is a written list, not more care. The helper that owns the
hand-off now names all four attributes and which function owns each β so the
fifth is looked for rather than waited for. And the helper is named for the
module it matches against, not for one attribute, because
_matched_dtype is a name that invites the device half to be deleted by
someone tidying it.
11. A number is not evidence for a hypothesis until you check which variable produced it.
A run came back at β10.4 dBFS and was read as confirming that noise_scale
must equal sigmas[0]. The manifest showed the variable that had actually
moved was MATRIX_AUDIO_FIXTURE β a different audio file, not a different
noise_scale. One run, read as evidence for a hypothesis it never tested, and
a schedule change shipped on it.
The 2Γ2 that settled it is the shape to copy:
| content | noise_scale |
mismatch present? | result |
|---|---|---|---|
| realistic | 0.5 | yes | healthy, β10.4 dBFS |
| synthetic tones | 0.5 | yes | silent, β87.9 |
| synthetic tones | 1.0 | no | healthy, β7.6 |
Row 1 is the whole answer: the mismatch is present and the output is fine, so content is sufficient to explain the silence and the mismatch is not necessary. The single-variable test that had been pre-registered β healthy at 1.0 confirms it β was consistent with both hypotheses and discriminated neither.
An experiment that holds the suspected cause constant cannot distinguish it from anything else β and one that varies two things at once cannot either. The manifest is what makes the second recoverable after the fact, which is why recording a result's inputs is worth more than recording the result.
12. An advertised lever attached to nothing is not written by careless people. It is written by careful people who do not revisit.
decode_kwargs(generator=...) was added with a docstring explaining why the
diffusion decoder draws its own noise and must be seeded. The reasoning was
right, the parameter was right, and it read a key nothing ever set β there
was no generator anywhere in the package, and _resolve_seed's result went only
to sidecar metadata. Every "same seed" comparison on this build was two
independent runs.
It was written by the person who spent the same day removing other people's
inert levers, and it surfaced only because an unrelated question about
sample vs argmax sent them past the line.
The care that writes the docstring is spent at the moment of writing. What catches the lever is a check that runs later β which is Β§9.1.5 rule 7 arriving from the other side: not "prefer a check to a comment" because comments rot, but because a comment is written by someone who has just understood the problem, and that is exactly the person least able to notice they have not finished solving it.
9.1.6 The ladder, and the rung that is still hand-built
Each rung was added because the one below it passed while something was broken:
| the check | what it still misses | |
|---|---|---|
| 1 | the module is imported | it can be imported by a UI readout and dead on the generation path |
| 2 | it is reached by the path that does the work | a live module can contain a dead function |
| 3 | this specific function ran | a function can run with the wrong arguments |
| 4 | the arguments match the vendor's specification | an argument of the right shape can be the wrong type |
| 5 | the argument is the type the callee accepts | βΈοΈ hand-built, one call site at a time |
Rung 5 was added by a failed deploy. A quantization_config was built,
threaded, and proven to reach from_pretrained by a test written for rung 4 β
and the pipeline rejected it, because a bare BitsAndBytesConfig is what a
component takes and DiffusionPipeline.from_pretrained requires
PipelineQuantizationConfig. Both arrive equally well; the test could not
tell them apart, and nothing at rung 4 could.
That is the first time on this build we were bitten one step past the rung the instrument was built for, which is the useful shape: an instrument answers the question it was designed for, and the next question is invisible to it precisely because it looked answered.
Rung 4 is where DISTILLED_SIGMA_VALUES hid: defined, documented as mandatory,
quoted in its own module docstring, and referenced β from a function applied
only when stage 2 was enabled. Every check we owned passed while eleven of
twelve runs used a schedule the model was not trained for. It is the rung with
no general instrument, and the only thing that works today is recording the
vendor's specification independently and asserting against it.
9.2 GPU-gated tests
Anything needing real weights is marked gpu and deselected by default. CI and the default local
run must be green with zero weights on disk.
9.3 The invariant tests that matter most
Ranked by how much silent wrongness they prevent:
- A LoRA for the wrong architecture is rejected β specifically at stage 7 if it survives stages 1β6 (Β§4.3). (Corrected 2026-11-16: Β§4.3 is right and this line was not. Stage 6.5 is the gate β per-module shape fit, now checked at Validate time against a checked-in shape manifest, with no weights β and stage 7's attach count is the second line, not the first.)
- A generation that produces no file raises β never returns an empty path, and there is no newest-file fallback (Β§5.3).
denormalize=Falseon the diffusion decode β the double-denormalization corruption (Β§7.4) is silent and total.- Pixelβlatent index conversion β off-by-8 is guaranteed without it (Β§5.7 #1).
- Audio/video length desync with a supplied
audio_latents(Β§5.7 #11), concrete case. - The diffusers-version assertion fires on 0.39.0 (Β§1.3) β otherwise the failure is late and expensive.
vae.temporal_compression_ratiotripwire raises if upstream changes it (Β§5.5).preload_from_hubnames only live shards β assertable against the index JSON with no download (Β§2.1).fuse_lorapasses both components (Β§4.4).
10. Resolved questions and technical risks
All six open questions from the first draft are resolved. What remains is one ranked risk register.
| # | Question | Resolution |
|---|---|---|
| OQ-1 | Pin a diffusers release, or main? |
RESOLVED: pin the exact SHA ac56fa2f⦠(§1.3). A SHA is more pinned than a range, not less. Plus a startup version assertion, because 0.39.0 fails late. |
| OQ-2 | Does diffusion_decoder have a consumer? |
Moot β the SHA provides it. Both diffusion_decoder (0.834) and duration_head (0.004) stay in the subset; duration prediction ships. (2026-11-16: until today its only consumer was stage 2; it now decodes single-stage runs too, on CUDA, Β§7.4.) |
| OQ-3 | Is num_frames % 8 == 1 enforced? |
RESOLVED: no. Upstream silently floors. We raise; the modulus is a parameter with an 8 fallback for pre-load validation, plus a load-time tripwire (Β§5.5). |
| OQ-4 | Gemma-3 or Gemma-4? | CLOSED ON EVIDENCE: Gemma-4. model_index.json says text_encoder: ['transformers', 'Gemma4UnifiedForConditionalGeneration']. The diffusers annotation is a union (Gemma3ForConditionalGeneration | Gemma4UnifiedForConditionalGeneration); the weights resolve it. So CRF provably takes the gemma4 branch β 18. Still derive family and CRF at runtime; hardcode neither (Β§5.7 #8). |
| OQ-5 | Audio length from duration or latent shape? | RESOLVED: latent shape wins. Upstream computes it from duration, then overrides from audio_latents.shape when supplied. The desync is real and directional (Β§5.7 #11). |
| OQ-6 | Output retention bound? | RESOLVED: 20 GB, oldest-first, swept on startup and after each run (Β§2.7). |
Technical risk register
| # | Risk | Why it is uncertain | How it gets settled |
|---|---|---|---|
| 1 | Not a risk any more, a finding. The snapshot IS freed after packing (verified, spaces 0.51.1) β and the unlink runs strictly after _pack(), so the 144.37 GB peak precedes it, and with an 8β12 GB image the total is 152β156 GB against a 150 GB cap. See Β§2.5 for the authoritative arithmetic. The ~68 GB RAM ceiling remains a community cgroup observation, separate and still open. |
No longer settled by measurement β it is settled. The routes are persistent storage, a public Space with preload_from_hub (Β§6.2), or smaller checkpoints. All three are operator decisions, none is a knob we can turn. ZeroGPU is parked; hinode is the critical path. |
|
| 2 | spaces/zero/torch/bitsandbytes.py no-ops for bnb β₯ 0.46.0, wired in at patching.py:328/355/445. No equivalent module exists for torchao β bnb is supported, torchao is not, and that decides the library. Verified, spaces 0.51.1. |
Not a risk. Recorded because the asymmetry is invisible from outside. See docs/ZEROGPU_DISK.md. |
|
| 3 | preload_from_hub 10-entry cap |
RETIRED. We are one repo β one entry with comma-separated filenames. Fits comfortably (Β§6.2). | β |
| 4 | Max @spaces.GPU duration on our tier |
server-side, tier-dependent, unpublished | discover empirically; never hardcode (Β§5.8 #4). (2026-11-16: still unknown, and now load-bearing: this app asks for up to 510 s.) |
| 5 | Multishot via prompt structure produces usable results | no API support, and the enhancer system prompts actively discourage scene cuts (Β§3.4) | presented to users as unverified technique, not a feature. (2026-11-16: the UI no longer offers a shot list, Β§3.4; several shots can still be written into the prompt.) |
| 7 | β Measured on hinode: two stage-2 generations in one process, exactly one kernel fetch. So a worker needs outbound network at its first stage-2 use, not on every press of Generate. | Still a real dependency β each cold worker pays it once, and get_kernel("shi-labs/natten", version=1) is hardcoded inside diffusers so we cannot pin it by revision. Lazy construction keeps it off the startup path for the eight modes that never reach stage 2. (Corrected 2026-11-16: three modes skip stage 2, not eight. And with LTX25_DECODER=auto the diffusion decoder now decodes single-stage video runs on CUDA too, so a cold worker pays the fetch at its first video run; if the decoder or its kernel cannot be built, auto falls back to the conv VAE and says so. Not yet seen on the Space.) (Corrected 2026-11-17: the decoder, NATTEN processor included, is built once at start-up now (pipelines.build_decoder_at_startup), so the kernel is fetched at start-up rather than in a worker; if that build fails, start-up logs one line and each run builds its own, as before. On deploy 1 it was still loaded again inside every run's worker (SPACE_MODE_MATRIX.md Β§10.1).) |
|
| 6 | An LTX-2.3 LoRA attaches cleanly to the 2.5 distilled transformer | the model card says most 2.3 LoRAs run on 2.5 unchanged, but our R3 fixture is a 2.3 LoRA and our transformer is 2.5-distilled; the official 2.5 LoRA targets the DEV transformer, so it is not a valid control | TEST_PLAN R3 β stage 7 asserts attachment count > 0 against real weights. (2026-11-16: the validator now warns on a 2.3 adapter instead of refusing it. Two 2.5 adapters visibly change the output on real weights, locally (SPACE_MODE_MATRIX.md Β§10.2); a 2.3 adapter has not been run on real weights.) |
| 8 | temporal_latent_upsampler has no documented role β we download 0.2619 GB and preflight carries a dedicated error path for it |
It is the same class as the spatial one (LTX2LatentUpsamplerModel), differing only by spatial_upsample/temporal_upsample and use_rational_resampler. But upstream's component table lists only latent_upsampler/ as belonging to the two-stage recipe, and neither documented recipe nor LTX25AutoBlocks mentions the temporal one. Read off the model card at revision 871165de⦠and the configs themselves, not inferred |
Deliberately not wired. Inventing a role for it in a graph upstream does not describe is worse than 0.26 GB of unused download. If it proves to be dead weight the allow-list can drop it β but on evidence, not on tidiness |
Risk 1, re-read on 2026-08-25 β three of its clauses aged, differently. (a) The disk finding stands: bf16 does not fit, and that is now demonstrated as well as derived. (b) "The ~68 GB RAM ceiling remains β¦ still open" β no longer open: falsified by measurement, the ceiling is above 93.56 GB (
SPACE_GENERATION_FINDING.mdΒ§6). (c) "ZeroGPU is parked; hinode is the critical path" β inverted: ZeroGPU is the live deployment, generating all eleven modes at sha287ba9cvia the route the "how it gets settled" column did not list β int8 runtime quantization (ZEROGPU_DISK.mdOption B), fitting at 82.2/150 [OK].β Corrected 2026-11-16: at
287ba9cthe eleven modes executed; every output up toff142d9was colour static. "82.2/150 [OK]" is a reading taken before packing; the disk peak has never been measured. hinode is retired.
β Blocker cleared
Lightricks/LTX-2.5-Diffusers gate is OPEN (Β§6.3), verified by an authenticated HTTP 200 on
model_index.json. Weights work is unblocked.
Remaining access note, not a blocker: the Lightricks LoRA repos are separately gated and still shut. Β§4.5 shows the ungated community LoRAs make a better R3 fixture anyway. (Checked in August; not re-checked since.)
11. Build order
- β This document, committed.
- β
Scaffold β venv,
pyproject.toml, package stubs,.gitignore. - β
requirements.txtβ the Β§5.1 pin set, with nospacesline. - β
docs/TEST_PLAN.mdβ the quota-bounded GPU matrix. Must exist and be reported before the first GPU call. - TDD, innermost-out, against the test teammate's failing specs:
errorsβconstraintsβmodel_filesβpreflightβlora/βzerogpuβconditioningβmodesβaudioβgenerateβoutputsβui. - GPU runs on the Space, in the TEST_PLAN order (R1 first β it is the riskiest), logging actual cost after each and stopping hard at 1920 GPU-seconds.