Spaces:
Running on Zero
Running on Zero
Commit History
3 days ago
-
fix: everything a user reads after Generate is in the UI's words, and audio too short for the set length is refused in them
Unverified signatureThe notes above Diagnostics now state facts that change what the user got, in plain words. A silent mode says the sound the model made was left out and names the mode that keeps it (video to video names none, since it has none). Audio to audio says the sound is the model's, made from the user's, and how much it was changed; audio to video says the sound is the user's, unchanged. Lengths and the trim are said in seconds. A requested level is mentioned only when it was snapped to a different level, never as "you asked for 0.42 ... 0.42 is the nearest". The upscaled size, the decoder and the prompt sent are the run record's and are no longer repeated as notes. The decoder policy, evictions, and a track an encoder offered that the mode does not declare go to the diagnostics. The upscale phase reads "upscaling ×2", and Upscale off CUDA is refused as "Upscale ×2 needs a CUDA GPU; this machine runs on cpu." A set length longer than the audio is refused before dispatch with the longest lengt…8a7fa0c
4 days ago
-
test: the exports and encoder contracts run every mode with real inputs
Unverified signature`run()` in test_exports_contract.py and `run_production()` in test_encoder_contract.py called `generate()` with no prompt, image, video or audio. That only worked because nothing checked a run's inputs. They now pass `conftest.inputs_for(mode_id)` (a prompt, a PIL image, a real mp4 or a real WAV for each kind the mode requires), with caller overrides still winning. The same goes for the three tests that named a nonexistent "an-upload.wav" / "a-song-the-user-cares-about.wav" (now `conftest.audio_file()`). The audio modes now really encode an upload, so: * both modules stub the loader with `audio_input_ready` (autouse) so the torch-less environment still runs them, and the joint-mode test in test_media_io.py requests it explicitly; * `VocoderPipe` and `VocoderlessPipe` carry a `FakeAudioVae`. Without one `_prepared_audio` refuses before the writer, and the vocoderless refusal those tests pin would never be reached. No assertion changed. Green at this commit in both environments…d12722b -
fix: the user's audio is muxed as 48 kHz stereo, the mp4 is encoded for viewing, and video+audio runs hand back a wav
Unverified signatureThree changes in `outputs.write_output`. The supplied track (F17). For `audio_to_video` the user's reconciled waveform was muxed at the upload's own rate and channel count. A mono 24 kHz file became mono 24 kHz AAC, `tests/verify_streams.py` (2 ch, 48 kHz) called a correct run a mismatch, and a non-standard rate killed the run at the very end: 37 kHz -> `avcodec_open2("aac") returned 22`. The writer now converts the supplied track to the rate the model's track would have had (the vocoder rate it is already handed) and to stereo before muxing: * mono is duplicated into both channels at the same level (FFmpeg's own mono->stereo would attenuate it 3 dB); * more than two channels get a plain downmix: front left/right at full level, the rest at -3 dB, LFE dropped, each side normalised so it cannot clip (FFmpeg's float downmix took 5.1 at 0.9 to 2.17); * the rate goes through `av`'s resampler, available in both environments, and the length is pinned to round(n * out / in) samples, s…20155a1
-
fix: `audio_to_video` hands back YOUR audio — the model's track is silence when audio is supplied
Unverified signatureA controlled discriminator on the live Space, same pipeline, same 512x960x121 geometry, same seed, one variable — whether an audio input was supplied: audio_to_video (audio IN) video+aac ✓ -88.3 dBFS digital silence text_to_audio_video (no audio) video+aac ✓ -24.2 dBFS real content 64 dB of separation. **Supplying audio suppresses the model's own audio generation.** That is a property of the model, not a defect in our plumbing: `4683a57`'s mux is correct and both files carry a proper 48 kHz stereo AAC track. But `4683a57` chose to mux the model's track, so a user who hands us audio has been getting back a file whose AAC stream is empty — the same emptiness the original mode-11 defect had, now wearing a stream header, which is arguably worse because it looks like a track until you play it. RULED, for `audio_to_video` only: mux the audio the user SUPPLIED. He provided it, he expects it back, and the model's is empty anyway. The six reasons at the mode-11 sp…e2f538a
-
fix: audio-conditioned video came back silent — mode 11 now muxes the model's own track
Unverified signature`audio_to_video` took an audio upload and returned an mp4 with ZERO audio streams. Confirmed twice on the live Space: the operator hit it, and a controlled run reproduced it as `audio: False` in the response tuple. A user who conditions on audio expects audio in the file. The registry declared exports=("video",), so `outputs._write_video` took its `include_audio` branch, discarded the generated track, and reported it through `dropped_audio_warning` — which advised "use the matching audio+video mode to keep it". Mode 11's `audio_counterpart` is None. THERE IS NO SUCH MODE. The one mitigation the drop had pointed at nothing, and sent the user hunting for a mode nobody built. RULED: THE MUXED TRACK IS THE AUDIO THE MODEL GENERATED, NOT THE AUDIO THE USER SUPPLIED. Both were defensible. The generated one is the one that cannot silently mislead. Six reasons, recorded in full at the mode-11 spec: 1. `audio_to_audio` already answers this. Mode 10 takes the same input, through the sa…4683a57
-
fix: bfloat16 audio must reach the encoder
Unverified signaturenumpy has no bfloat16, so _as_array's value.numpy() raised 'Got unsupported ScalarType BFloat16' and six of eleven modes generated successfully and died at the last step. WEIGHT_DTYPE is bfloat16 and is documented as not configurable, so this was the DEFAULT path for every audio-exporting mode, not an edge case. Downcast in _as_array, not in _as_audio_planes: the helper promises 'whatever we were given -> a numpy array', so a dtype it cannot convert is its problem, and fixing it at one caller would leave the other latent -- which is how a shared helper with two callers hid it. float32, not float16. A silent narrowing inside the module whose entire job is 'never silently wrong' would be its own defect. Selected by what numpy can represent rather than by dtype name, so a future dtype without a numpy equivalent takes the same route without anyone having to remember to add it. On the tests, which is the part that matters: every fixture in test_encoder_contract fed the real writer np.fl…c628a2a -
test: drive the real writer, and open the file back up
Every existing generate() spec injects a fake encoder, and a fake accepts whatever it is handed. So generate() could forward an argument set that outputs.write_output cannot write with, and all of them would still pass while the production path died for a mode nobody exercised. outputs refuses to invent two numbers on purpose (PLAN 5.4): the sampling rate off the vocoder, and the frame rate the pipeline generated at. Either one missing is a run that dies at the last step of an already-paid-for generation. So this runs encoder=None - the real av writer - for all 11 modes with real arrays, then demuxes the result. "Wrote an mp4" and "wrote an mp4 with the audio in it" are the same result to everything except the user's ears, so the joint modes' containers are opened back up and checked for a real audio stream with real samples and both channels intact. Muxing IS specifiable offline: av is a pinned runtime dependency and 2 frames of 64x64 costs no measurable time. Verified against HEA…2019e7f