Spaces:
Running on Zero
Running on Zero
Commit History
2 days ago
-
docs: every sentence stays true after the deploy: README and the release notes say this is deploy 2's code and point to issue #32 for what the Space shows, and no doc says the Space still runs deploy 1, is not deployed, or has not been redeployed since August (#32)
Unverified signatureThe commit we deploy is also what origin/main is fast-forwarded to, and nothing can be pushed after the deploy without restarting the public Space, so a sentence that a deploy makes false would stay false. README: "Where things stand, 2026-11-17" says the README describes deploy 2's code, which replaces deploy 1 on the Space, points to SPACE_MODE_MATRIX 10.1-10.3 and the release notes, and to issue #32 for deploy 2's own check on the Space. The cold-start time points to #32 instead of "has not been measured"; the longest accepted reservation is "not measured by these runs"; the browser runs of 2026-11-16 "did not finish in that session" instead of "have not been run ... yet". docs/RELEASE_NOTES.md: "Deploy 2 - 2026-11-17", the code of this commit's tree, the Space result on #32 (also under "On the Space"); deploy 1 "ran" as 0cd54f2 until deploy 2, and main "was fast-forwarded" to 1e9fd20 that day. The other docs, the same way: SPACE_MODE_MATRIX's header and 10 intro, 10.2 and 10.3; …abc9150 -
docs: the reservation follows size and length: the README's table is the estimator's (the default run 60 s, needing 120 s of quota), the measured envelope is a stated limit, and SPACE_MODE_MATRIX §10.3 records the six size-and-length runs and the answer to the operator's question (#21)
Unverified signatureac68de5 made zerogpu.estimate_duration a fit of the Space's measured runs with a 1.3 margin per term; the docs still said a flat 180 s, x1.5 with Upscale, +60 with an adapter and +180 with Enhance. README: the reservation section says how a run is priced now and gives the estimator's own figures (computed from the code: the default 704x480, 5 s run reserves 60 s and needs 120 s free; +45 s with Upscale at that size, +40 s with adapters, +80 s with Enhance; 1280x704 at 10 s 145 s; 960x544 at 10 s with Upscale 240 s; a sound-only mode 45 s), that LTX25_GPU_DURATION_SECONDS is ignored and LTX25_GPU_DURATION_FLOOR_SECONDS sets a minimum; deploy 1's GPU windows stay as the history they are. The limits section states the envelope the Space refuses past (64,000 tokens per pass, 520 million returned pixel-frames) in the refusal's own words (tests/test_reservation.py). Enhance's "reserves 180 s more" is 80 s, Upscale's x1.5 is 45 s at the default size, and the Upscale paragraph gives what the …ca1f2a8 -
docs: wording in deploy 1's record: the README's adapter-on-the-Space sentence says Validate once, the matrix's 10.1 intro reads as one paragraph, and the release notes' browser runs open with a sentence (#25)
Unverified signaturec8c5965 -
docs: the small truths: the Space is public and runs deploy 1, the README says what the page shows about cost on a Mac in place of the comment that asked for it, the browser session on the fixed build is written, and a speaking face gets the page's own words (#29)
Unverified signatureREADME body: - "Deploys to a private Hugging Face Space" now says public: the repo and the Space were made public on 2026-11-16 (the YAML front matter's preload comment, which also explains the 401 in terms of a private Space, belongs to the package editing the front matter and is untouched); - "Where things stand" no longer says the Space runs the August build: it runs deploy 1, on which all eleven modes ran and were looked at, and it points at the matrix (10.1 the Space, 10.2 the Mac) and the release notes; - the HTML comment addressed to the lead is gone; in its place the sentence it asked for: on a Mac the cost panel reads "Runs on this Mac's GPU; nothing is metered" and nothing on the page speaks of quota, reserving, billing or paying (render.py; pinned by test_ui_device_words.py). This is also the README sentence #19 asks for; - the browser runs: the second session, on a build with the memory check fixed, finished text to video with sound through the page with the…da5f47d -
docs: deploy 1's Space run is in the repo: SPACE_MODE_MATRIX §10.1 call by call, PR #1's corrected texts as the release notes' Deploy 1 section, and the README's Space sentences carry what the Space measured (#25)
Unverified signature§10.1 was a placeholder waiting for the post-deploy Space run. It now records deploy 1's 24 calls (2026-11-16, 16:02-16:27 UTC, Space commit 0cd54f2, the tree of 1e9fd20): per call what was sent, what came back, what was seen and measured, and the GPU-s bound (the window x 2), each row named by its label under the run's evidence/space/matrix/, where every figure in the row can be found (checked by script: 245 figures, none missing). With it: the build and the start-up report, how the outputs were judged (and why the client's MISMATCH on the 13 video-with-sound calls was the client's), what the calls settle, the decoder and adapter A/B pairs, mode 11's colour by seed, and the findings the calls left open. The document's header, the never-written-up block and the section 8 follow-ups that waited on the Space (7, 8, 9, 12, 13) now say what deploy 1 showed. PR #1 is merged and stays as it is on GitHub. docs/RELEASE_NOTES.md holds its corrected body and comment as the "Deploy 1 - 2026-11-1…0646770
3 days ago
-
docs: the local matrix records the final pass: every mode, the adapter, the enhancer and the model-chosen length run again on the final code, identical wherever compared with a run made after the RoPE fix
Unverified signatureSPACE_MODE_MATRIX.md §10.2 gets the lead's final pass (run log, 2026-11-16 19:16-20:16): fourteen runs on e59944e, each with the inputs, prompt, size and seed of an earlier run and compared with it, plus F11b_ocean and F11c_rail for mode 11's new sample and second example. Modes 01, 03, 04, 09, 10, 11, the adapter, the enhancer and the model-chosen length are identical to their references (SSIM 1.000, sound 1.000); 02, 05, 06 and 07, compared with runs from one commit before the float64 RoPE fix, show the same scenes framed slightly differently (SSIM 0.78-0.94), the same words and sound, and frame 0 of mode 02 at 0.7507 as before; mode 08 has the same sound. Lip sync on these outputs: +0.845 in mode 07, -0.379 in mode 11. Three runs refused on e59944e by the memory check's early reading (mode 08 twice, the enhancer, the model-chosen length at the denoise check) ran on ba2d582, with the settled readings 3 to 13 GB below the first ones; their peaks (77.0, 77.6, 84.5 GB) and the pass's (6…0300944 -
docs: the local matrix records video to audio on a clip with no sound track at all, the run behind the mode's shipped sample
Unverified signatureRow 09 of SPACE_MODE_MATRIX §10.2 listed only the runs at b12415f, where the clip had a sound track that never reached the model. S09_captain_voice (at 7cf1547) ran on a copy of the same clip with no sound track: the generated voice's loudness follows the track the clip originally had at 0.926. It is the run the mode's sample output comes from.1132d15 -
docs: running locally is described as one code path with the Space: the same stage graph, with the local runner supplying only what is resident when
Unverified signatureSince c712e7e the local runner is a StageRunner: the decode policy, stage 2 and its refusals, the audio conditioning, the clip variation and the run facts are one piece of code, and the local side adds residency through one seam. The README's local section and PLAN §7.1, §7.4 and §8.7 now say so, and what follows from it: off CUDA the default decode is the conv VAE and the run says why, and an explicit diffusion decode or Upscale x2 is refused before dispatch except at very small sizes. The measured memory and time figures stay, with the day's full range (peaks 57-72 GB at 704x480 or 480x704, up to 193 frames) and the enhancer's runs added. SPACE_MODE_MATRIX §10.2 says which of its runs predate the change.1f05c04 -
docs: audio to video is said plainly not to lip-sync, with the measurements, and a face that speaks in time is sent to image + text to video with sound
Unverified signatureThe README's audio section gains "Speech and lips": in audio to video the mouth's opening does not follow the words (correlation between -0.38 and +0.07 in every run scored, no better than an unrelated track), while speech the model writes itself scores +0.84; prompting for lip sync with the quoted words did not change it; holding the audio differently while sampling plus cross-modal guidance reached at most +0.26 in a probe and is not in the app; the vendor's lip-sync recipes for LTX-2.3 use the non-distilled model. SPACE_MODE_MATRIX §10.2 gets the lead's three tables as STATUS has them (the reference and controls, the prompt-technique A/B on the app's own path, the sampling probe) and the conclusion; row 11 no longer reads as if the man spoke the recorded line, and points to them.b27082c -
docs: the audio modes are described as measured after the audio-encoding fix, and the API's noise_scale default is said to snap to "a little"
Unverified signatureThe known-issue box about the 9 dB loss is gone: the cause (the mel transform's window taken from the vocoder's config) is fixed at 7cf1547. In its place the README states what the lead measured on real weights afterwards: audio to audio returns sound at the input's own level, graded by the choice (same events at the same times 0.97 / 0.95 / 0.71 on speech, transcript exact at "a little"); video to audio follows what is visible (on a silent clip of a shouting man, a voice whose loudness follows the clip's original track at 0.92); audio to video carries the recording exactly and takes its length from it, with or without a prompt and a first-frame image. Through the API, noise_scale has to be passed for video to video and audio to audio: the published default 0.5 snaps to 0.42 in both. SPACE_MODE_MATRIX §10.2 rows 10 and 11 carry the runs at 7cf1547 and the fixed defect, including the black-and-white pictures it caused; PLAN §5.4 says the analysis window is a declared convention, not a …67c1514 -
docs: video to video is described as verified on real weights: a variation of the clip at three levels, with the prompt acting at the highest
Unverified signatureThe README, PLAN §3.3 and the local matrix (SPACE_MODE_MATRIX §10.2) now say what the lead's runs of the new mechanism showed (V04_* at 85405e9, S04_summer_l091 at 7cf1547): SSIM to the clip 0.78 / 0.65 / 0.36-0.40 at "a little", "moderately" and "a lot"; the prompt acts only at "a lot"; it runs without a prompt; a sideways-stored phone clip comes out upright and a 60 fps clip is read in real time; the length is the clip's. All eleven modes are therefore verified locally, and every count of them says so. The follow-up comments the first docs pass left are gone. The Matroska/WebM length defect (F70) is not listed as a known issue: it is fixed at 16bbbd3.41d0e94 -
docs: the enhancer is verified on real weights after its transformers 5.23.0 fix, and the 9 dB audio loss has a found cause
Unverified signatureFrom the lead's run log at 13:41: - the prompt enhancer, which could not load on transformers 5.23.0, now rewrites prompts through the app's own path on both routes (text, and image: it writes details only the supplied image shows); README and §10.2 no longer call it unchecked; - the ~9 dB loss in audio to audio, and in the audio audio-to-video's picture is made to, comes from the app's mel transform, which took its FFT size (512) from the vocoder's config; with 1024 the encode -> decode round trip comes back within 0.2 dB. A fix is in progress; the README's known-issue note and §10.2 rows 10 and 11 say so.7dd6ff6 -
docs: the local matrix calls the test inputs generated images, not photos
Unverified signatureThe images the conditioned modes were checked against were generated for the test; §10.2 said "photo" in five places.71b405c -
docs: the local matrix and the README follow the lead's later runs: mode 11 follows colour words, and the audio encode loses about 9 dB
Unverified signatureTwo results recorded after the matrix was written (lead's run log, 13:07 and 13:27): - audio to video without a first-frame image was black and white only when the prompt named no colours; a prompt naming colours gives colour, and the mode passes in every variant tried (prompt only, no prompt, first frame, non-speech audio, the operator's recording); - audio to audio's 7-9 dB loss is not the generator: the app's own audio encode -> decode round trip loses about 9 dB with no generation at all. It is an open defect, being investigated; it also shapes the audio that audio to video's picture is made to, but not the soundtrack in that file. SPACE_MODE_MATRIX §10.2 rows 10 and 11 say so, with the new run ids; the README names the known issue under the audio modes.1c9e1f6 -
docs: the mode matrix says what has been verified since the fixes, and that the Space has not been re-run
Unverified signatureSPACE_MODE_MATRIX.md is a living document, so its stale claims are corrected in place rather than boxed: - the title and the headline say the August modes *execute*; "generate, content-verified" was colour static; - stage 2's 25-49 frame ceiling was the `large` partition (49 passes on xlarge); the August cost labels were `large` runs; the ×2 multiplier is documented, not measured; three code shas, not two; spaces 0.51.3; the inert precision variable is gone from the Space; - the post-fix matrix of 2026-08-25 ran but was never written up and its files are lost; follow-ups 7, 8, 9, 11, 12 and 13 say where each stands now. §10 is rewritten. 10.1 is an honest placeholder to be written from the post-deploy Space run, naming the CUDA-only parts nobody has verified since the fixes (Upscale ×2, the diffusion decoder, bitsandbytes int8). 10.2 is the local real-weights matrix as the lead's run log records it on 2026-11-16: modes 01-03 and 05-11 verified with run ids and measurements,…342075f
-
docs: the app ran its whole life on half a GPU — `auto` cannot promote, and four documents said otherwise
Unverified signature`MIG_ALLOCATOR_SIGHTINGS.md` was a register of three NVML allocator asserts with no explanation and a ruling that nobody chase them until somebody read the headroom. `0a4ddbb` read it. The number was **47.375 GiB** — ZeroGPU's `large` — where every document in this repo assumed `xlarge`'s 94.971 GiB. ⛔ THE ROOT CAUSE, IN FOUR HOPS, ALL FROM THE PLATFORM'S OWN PACKAGE: * `spaces/zero/__init__.py:29-35` — `startup()` packs, clears the 42.00 GiB `xlarge_threshold`, and sets the module global `_default_gpu_size='xlarge'`. The promotion is real and it does fire. * `spaces/zero/client.py:109` — `get_duration_seconds()` is the **only** consumer of `default_gpu_size()` anywhere in the package, and it rebinds a **local**. * `spaces/zero/client.py:111-112` — that local feeds one branch: `if gpu_size != 'xlarge': duration_seconds *= duration_factor`. The promotion's entire observable effect is skipping a 1.5× multiplier on the *requested duration*. * `client.py:144` → `api.py:112-…caf1083 -
docs: the NVML allocator assert has been seen three times — collected, hypothesised, and deliberately not chased
Unverified signatureThe same `RuntimeError` has now killed a run on this Space on two separate days, in three different modules. Each was recorded where that day's work happened. Scattered, they are incidents. Side by side, they are a shape, and that is the only reason to collect them. `docs/MIG_ALLOCATOR_SIGHTINGS.md`, a register rather than an investigation: * **1 — 2026-08-25**, `text_to_video` 1216×704×121 stage 1 (R3, deploy 1). Denoise completed (~52 s); the VAE decode raised `NVML_SUCCESS == r INTERNAL ASSERT FAILED at "/pytorch/c10/cuda/CUDACachingAllocator.cpp":1165` inside `conv3d` via `autoencoder_kl_ltx2.py:108`. Fixed by tiled + framewise decode at `cdb3211`; the retry passed at 66.47 s / 6,258,833 B. * **2 — 2026-08-25**, stage 2 at 704×480×49 input (R4b). Same assert, in `LTX2VideoDiffusionDecoderModel`, which has no tiling API at the pinned diffusers SHA. Unresolved; the stage-2 ceiling sits between 25 and 49. * **3 — 2026-08-27**, the FIRST enhanced generation in a fresh w…06acd83
-
docs: every Space generation result was garbage — the invalidation, stated at the top
Unverified signatureThe deployed Space produced colour static for the entire project. Blanket int8 quantised the LTX-2.5 conditioning path (the eight LTX2AdaLayerNormSingle modulation heads plus the in/out projections), the denoiser stopped being steered, and the output became a correctly-shaped sample from the wrong distribution. Fixed at eb1aa9f, deployed e3b5333, verified by opening a frame. SPACE_MODE_MATRIX and SPACE_SESSION_2026-08-25 get a dated invalidation block at the very top, above existing content, per the operator's ruling. House rule honoured: nothing is rewritten, every measurement stands as history, the corrections are dated on top. Each block says the same six things: - Every generation result in the document was produced by a broken transformer. Space shas up to and including ff142d9 ran blanket int8. - MECHANICAL CLAIMS SURVIVE: all eleven modes execute; both fork-boundary directions carry their cargo; determinism and byte-reproducibility hold; tiled decode enables 1216x704x12…898dad2 -
docs: the 2026-08-25 session — byte identity sha256-verified, 1216x704 proven, the stage-2 ceiling found
Unverified signatureNew: docs/SPACE_SESSION_2026-08-25.md, the record of the day's campaign — 14 metered attempts plus two zero-GPU validates across two deploys (8e9f7a1, then ff142d9 carrying cdb3211/0be0014/b9ee967/2eaeb2c and the UI redesign), total <= ~675.5 GPU-s of 2400, the 1920 ceiling never approached. Every artifact is banked under a directory named by its sha256, and every hash in the doc was re-computed at write-up time. The headline closes the correction this morning's pass made: byte-level reproducibility, previously UNVERIFIED (no day-1 hash), is now VERIFIED in the happy direction — a fresh seed-1234 run hashed equal to the surviving 2026-08-22 artifact (b78803cc…) on a third container and a different code sha, then equal AGAIN through the redesigned UI on the tiled-decode build on a warm worker that had just carried an adapter. The condition pipeline reproduced its banked artifact byte-for-byte (ef8571e6…) — giving the conditioning discriminator a zero noise floor — and both LoRA routes …76c408d -
docs: the audit's stale claims annotated — measurements stand, corrections dated on top
Unverified signatureA four-agent audit swept every doc for claims events had falsified (~30, with file:line). Fixed per the house rule: history is annotated, never rewritten — dated blockquote corrections point at current reality with evidence. Six follow-up-table rows get in-cell appends with status flips (the file's own precedent for rows); no finding is deleted, no captured record reworded. The delicate one first. SPACE_MODE_MATRIX §5's 'byte-identical to the result banked the previous day' was never verified: no day-1 hash was ever recorded, the match was file size (39,146 B) plus every decoded statistic to three decimals, and the day-1 file is gone with its host — so byte identity is now unverifiable against it. The §5 caveat ('Do NOT upgrade this') was correct and reports upgraded past it; the error ran in that direction, not the doc's. Corrected claim: identical in size and in all decoded statistics, byte identity UNVERIFIED; a same-seed re-run sha256-compared against the surviving 2026-08-22 arti…fd85c87
-
docs: 11 of 11 on the Space, and what that still does not claim
Unverified signatureThe two video-INPUT modes were re-run at Space sha 287ba9c carrying d2eb6f2 and both pass, so follow-up #1 closes: 19.95s / 63,438 B / -19.5 dBFS for video_to_audio, 13.67s / 33,812 B / 9 distinct frames / mean |d| 17.277 for video_to_video. Both land inside the 13.67-21.59s band every successful generation occupies, against the 3.53-9.13s the failures took. The uint8 choice is recorded as the part that was load-bearing on the metered side rather than locally: upstream re-compresses a single-frame condition at the model CRF and rejects a non-uint8 array while doing it, so a float payload would have passed the offline gate and failed only on the Space. Fixture provenance is recorded with the result, because without it the numbers prove much less. The runner chains fixtures forward, and by the end of the matrix video.mp4 had drifted to audio_to_video's near-static 7,619 B output at mean |d| 0.905. Conditioning on that would have been unreadable in either direction. The fixture was rebu…17b589c -
docs: name why the picklability finding is load-bearing, not incidental
Unverified signature§4 records the fork-boundary fix as a pair and shows that each half alone is insufficient. What it did not say is why that is easy to get wrong. `gr.Error` is the obvious half. It is what the source points at, it is what anyone reading `exception_result` reaches for first, and it is a one-line base class change with a clear rationale. On its own it fails -- for exactly the reason the picklability measurement gives. So the measurement is not a second improvement stacked on the first. It is the thing that makes the obvious change insufficient, and neither half looks necessary until you are holding both. That is the argument for recording the pair rather than leaving the next person to find it one failed deploy at a time. Offline gate: 2140 passed, 13 skipped.fc566af -
docs: the fork-boundary fix is a pair, and half of it is a regression
Unverified signatureThree corrections to SPACE_MODE_MATRIX.md, plus four stale claims elsewhere. The fix for the fork boundary was recorded as one method: a `__reduce__` on LTXStudioError. Reading `spaces` rather than trying it shows that is wrong in both directions, and neither half works alone. `__reduce__` alone does nothing. ExceptionResult (types.py:28-31) carries only `traceback`, `error_cls` and `gradio_error`, and `exception_result` populates the third only when `isinstance(exc, gr.Error)`. Our exception object is therefore never transmitted at all, nothing is ever pickled, and `__reduce__` is never consulted. That also settles the open question of whether restoring picklability would restore the message: it would not, and no live run is needed to know it. A `gr.Error` subclass alone is worse than the status quo. It makes the object eligible for `gradio_error`, which must then cross a multiprocessing.Queue that our classes cannot pickle onto. `spaces` anticipates this at wrappers.py:272-275 and…f927c4e -
docs: nine of eleven modes generate, and the failure is video input
Unverified signatureThe matrix ran at 256x256x9, stage 2 off, seed 1234. Nine modes pass with content-verified artifacts; video_to_video and video_to_audio fail with 'DecodeError' at 3.53/8.12/9.13s against the 14-21s a real generation takes. The split is clean and it is along the INPUT KIND, not the pipeline: both audio-input modes pass, both video-input modes fail, and image_to_video, audio_to_video and video_to_video all run the same condition workflow. The hypothesis that the fixture's AAC track was the cause is recorded as REFUTED -- a video-only h264 fixture failed identically -- because a refuted hypothesis narrows the search for whoever fixes this. Also recorded, measured offline at zero GPU cost: 22 of our 27 error classes cannot be pickled. Their __init__ takes keyword-only arguments while super().__init__ stores one formatted string, so self.args is a 1-tuple pickle feeds back positionally and the constructor refuses it. That is a concrete mechanism for the observed loss -- the Space printed …99e3652