Spaces:
Running on Zero
Running on Zero
Commit History
2 days ago
-
test: with the diffusion decoder built at start-up, and with its per-run fallback, a run's time line still has its own "decoding the frames" phase after sampling, and its split still adds up (#17)
Unverified signatureAfter the rebase onto the time-line work (#18), the run's "took ... s" diagnostic times the progress bar's phases. The decode phase is stamped by StageRunner._diffusion_decoded right before the decode call, which the start-up build does not change: what changes is that the runner's decoder lookup before sampling no longer loads anything, so the decode phase times the decode alone. On the tiny checkpoint, through generate() in the worker context, both runs with the start-up decoder and the run after a failed start-up build now also assert their time line, using #18's own parser (test_run_report_lines._adds_up / took_lines): one line, "decoding the frames" present and after "sampling", the phases adding up to the total. Seen locally: "took 0.9 s on cpu: loading the model 0.0 s, preparing the inputs 0.0 s, sampling 0.0 s, decoding the frames 0.7 s, writing the file 0.1 s". A confirmation of existing behaviour; it passes as written.5e06a43 -
fix: the start-up decoder is moved to its device only once it is fully built: the decode-pipeline wrapper is constructed before the move, so a build that fails at any step after the load has moved nothing and the per-run build still finds its file (#17)
Unverified signatureAt start-up ZeroGPU registers what is moved to cuda, packs it at launch and then prunes the file each packed tensor came from. In _decoder_part the LTX2VideoDiffusionDecodePipeline wrapper was built after decoder_model.to(device). Had that constructor raised, the decoder would have been moved (so packed, and its file pruned) but never cached, and each run's fallback build would then have found no file to load. The wrapper is now built first and the move is the last step before caching: load, NATTEN processor, tiling, the mmap-backing line, the wrapper, then the move. Wrapping before moving changes nothing else: the wrapper holds the same module, which moves in place. This also makes true what the first #17 commit's message said ("the move to the device is the last step before caching"); it was not, by one constructor. Test: tests/test_decoder_at_startup.py::test_a_failure_after_the_load_leaves_the_decoder_unmoved_and_the_run_builds_it (fake diffusers tree): the wrapper's first constr…d229cdc -
fix: the diffusion decoder is built once at start-up, beside the pipeline, so ZeroGPU packs it and every run's worker uses that instance instead of loading it again; a start-up build that fails logs one line and leaves the per-run build as before (#17)
Unverified signatureOn the Space the default decode is the diffusion decoder. It was built lazily inside each GPU worker and lost with it: every run loaded its 0.83 GB and resolved its NATTEN kernel over the network again (its load message appeared in every run's log of deploy 1). pipelines.build_decoder_at_startup(device=...) builds it with the existing _decoder_part, after the runners, and caches it where the runners' decoder factory already looks. app.py calls it at module scope in the CUDA branch, after the two get_pipeline calls, so its .to("cuda") is intercepted and registered for packing with the other components before demo.launch(); a forked worker's runner then finds the instance through _decoder_part's cache. get_pipeline itself still builds no decoder. Whether the NATTEN fetch needs a GPU at load time: it does not. diffusers' processor calls kernels.get_kernel("shi-labs/natten", version=1): the version-to-ref lookup and the variant download are network and disk; the build variant is chosen f…ed13532