Spaces:
Running on Zero
Running on Zero
Commit History
4 days ago
-
fix: the MPS watermark gate refuses exactly the settings PyTorch's allocator refuses, and says what really fails
Unverified signatureThe gate refused any high ratio set without a low one, and any low ratio above the high one, saying "every attempt to free GPU memory fails". That refused PyTorch's own default (1.7), its upper bound (2.0) and 0 (no limit, which PyTorch's own out-of-memory message suggests), so those users could not generate locally; and it let a high ratio above 2.0 through, which PyTorch refuses. The rule, measured on torch 2.11 and 2.15 with a 4-byte MPS allocation per setting and matching ATen/mps/MPSAllocator.h: the high ratio defaults to 1.7 and may be 0 (no limit) up to 2.0; the low ratio defaults to 1.4 and may be 0 up to the high one, or up to 2.0 when the high one is 0. A refused setting fails at the first allocation and on every GPU call after it, so the refusal now says that, names both variables and the value in force (a default when unset), and gives a working pair. Values are read as PyTorch reads them (C strtod: an empty value, "half" or "0,5" is 0, which means no limit), and MpsWaterm…6d7a452 -
fix: refuse an MPS watermark pair that makes every empty_cache() fail, before anything loads
Unverified signatureWith PYTORCH_MPS_HIGH_WATERMARK_RATIO below 1.4 and the low ratio unset, torch keeps its 1.4 low default and torch.mps.empty_cache() raises 'invalid low watermark ratio 1.4' — inside from_single_file and in the runner's own staging. memory.mps_watermarks() reads both ratios and names the problem and the fix; a high ratio without a low one at or below it, or a value that is not a number, is refused with a typed MpsWatermarkError. build_local_runner checks it on mps; scripts/local_generate.py prints both values and torch's recommended MPS maximum in the plan (dry run too) and exits 2 before touching anything. Nothing sets the variables.adad9b6 -
fix: the local runner gives model-file pages back — the committed gauge no longer counts weights twice
Unverified signatureMeasured by the lead on real weights: after a component moved to MPS, its memory-mapped safetensors pages stayed on the active queue for minutes, and active + wired + compressor (the gauge we keep) counted them, so G0 peaked ~22 GB over budget. Evicting them with mmap + msync(MS_INVALIDATE) dropped committed by the file's size in a fraction of a second. - memory.release_file_cache(paths): darwin msync(MS_INVALIDATE) per file, linux posix_fadvise(DONTNEED); resolves the HF cache's symlinks, walks directories, counts each file once, never raises. Data on disk untouched. - The runner moves every safetensors component (the small set, the text encoder, the transformer) tensor by tensor and releases the files' pages every ~2 GB, instead of one .to() that holds file pages plus the device copy; one release at the end, one line logged per release. The GGUF file is released after load_gguf_checkpoint (everything is copied by then) and after the move. Releasing the text encoder or …75d0c53
5 days ago
-
feat: a memory preflight for local runs — committed vm_stat memory plus the next step must fit the limit
Unverified signaturememory.py measures committed memory the way the operator's 90 GB rule is stated (vm_stat active + wired + compressor-occupied pages times the page size, never RSS, never 'available'), estimates the activations of one denoise + decode call from the video token count (MPS attention materialisation, live hidden tensors, the tiled conv-VAE decode, the float32 output, plus a fixed 1 GB; LTX25_ACTIVATION_CALIBRATION scales the size-dependent part), and judges a step: one describe() line naming every number, and require() raising a typed, plain-language refusal on FAIL. LTX25_MEMORY_LIMIT_GB overrides the 90 GB darwin default; Linux is report-only. Imports without torch.4ab8bee