techfreakworm commited on
Commit
92aea11
·
unverified ·
1 Parent(s): 66ca545

docs: the dtype is load-bearing for ZeroGPU's snapshot cleanup

Browse files

ZeroGPU frees the 72 GB HF snapshot after packing by reading
/proc/self/maps, mapping each tensor's data_ptr back to the blob it is
mmap'd from, and unlinking those. So the cleanup is conditional on the
tensors still BEING mmap'd — a dtype cast, a clone, an eager load or a
non-safetensors path materialises them into anonymous memory and there
is nothing left to capture. Then nothing is freed, 72 GB stays on disk
forever, and there is no error and no log line saying so.

torch_dtype has therefore stopped being a free parameter. The comment
sits at the risk site in get_pipeline rather than only in the plan,
because the next person to touch that line is the one who needs it and
they will not be reading §8.3 at the time.

Same class: the cache must stay under ~/.cache/huggingface/hub or the
autoprune pattern stops matching. That is a sharper reason not to move
HF_HOME than the preload one already recorded — it fails silently
rather than loudly.

TEST_PLAN corrections against the platform's own config:
xlarge_threshold is 42.00 GiB exactly, promoted from reported to
verified; duration_factor 1.5 is confirmed and gated on
gpu_size != 'xlarge', so it does not apply to us; and the 2× xlarge
billing multiplier is downgraded to UNVERIFIED because it appears
nowhere in the client. It sized the entire 7-run matrix. Budgeting the
pessimistic figure stays correct — being wrong costs unused quota,
while being wrong the other way costs a reservation failure
mid-matrix — and the first real run's billed seconds are the
measurement.

docs/PLAN.md CHANGED
@@ -1280,6 +1280,41 @@ normal GPU-box practice:
1280
 
1281
  HF's guidance, quoted: **models must be placed on cuda at the root module level.**
1282
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1283
  **Corollary: do NOT lazy-load models inside the decorated function.** HF documents that
1284
  placements are optimised for startup; lazy loading defeats the packer entirely.
1285
 
 
1280
 
1281
  HF's guidance, quoted: **models must be placed on cuda at the root module level.**
1282
 
1283
+ ##### ⚠️ The snapshot is freed after packing — but only if the mmap survives
1284
+
1285
+ Read from `spaces` 0.51.1 (`zero/torch/patching.py:395`) rather than inferred:
1286
+
1287
+ ```python
1288
+ with unmap_capture(mmap_addrs) as unmapped_paths:
1289
+ total_size = _pack(Config.zerogpu_offload_dir)
1290
+ for path in unmapped_paths:
1291
+ path.unlink(missing_ok=True)
1292
+ ```
1293
+
1294
+ `unmap_capture` reads `/proc/self/maps`, maps each tensor's `data_ptr()` back to
1295
+ **the blob file it is mmap'd from**, and unlinks those matching
1296
+ `~/.cache/huggingface/hub/models--*/blobs/*`.
1297
+
1298
+ **So the 72 GB snapshot IS reclaimed — conditionally.** Two preconditions, both
1299
+ easy to break and both silent when broken:
1300
+
1301
+ 1. **Tensors must still be mmap'd from the blobs when packing runs.** A dtype
1302
+ **cast**, a `.clone()`, an eager `low_cpu_mem_usage=False`, or a
1303
+ non-safetensors path materialises them into anonymous memory, leaving nothing
1304
+ in the map to capture. Nothing is freed, 72 GB stays forever, **no error and
1305
+ no log line**. This is why `torch_dtype` must equal the checkpoint's on-disk
1306
+ dtype rather than being a free parameter — recorded at the risk site in
1307
+ `pipelines.get_pipeline`.
1308
+ 2. **The cache must live under `~/.cache/huggingface/hub`**, or the autoprune
1309
+ pattern stops matching. **This is the sharper reason not to move `HF_HOME`** —
1310
+ §6.2 gives the preload reason, and this one is worse because it fails
1311
+ silently rather than loudly.
1312
+
1313
+ **And the unlink runs strictly AFTER `_pack()` completes**, so both copies
1314
+ coexist for the whole packing window — which is the 144.37 GB peak in §7.4.
1315
+ Steady state afterwards is fine at ~72 GB. **It is the peak that evicts us, not
1316
+ the residency.**
1317
+
1318
  **Corollary: do NOT lazy-load models inside the decorated function.** HF documents that
1319
  placements are optimised for startup; lazy loading defeats the packer entirely.
1320
 
docs/TEST_PLAN.md CHANGED
@@ -10,7 +10,7 @@ result from one does not answer the other.**
10
  | RAM | 124 GB | ~68 GB (community observation) |
11
  | disk | 136 GB free, persistent | 150 GB cap, **wiped on cold start** |
12
  | tensor packing | none | O_DIRECT to `~/.zerogpu/tensors` |
13
- | budget | time only | **2400 GPU-s/day**, 2× on `xlarge` |
14
  | status | **critical path** | **parked** |
15
 
16
  **Label every claim.** A mode that works on hinode is **HOST-PROVEN**; whether it
@@ -127,8 +127,8 @@ its economics do not apply to hinode and must not leak into Part A.
127
  | | |
128
  |---|---|
129
  | PRO quota | **2400 GPU-seconds/day**, fixed 24 h window starting at first use |
130
- | Our tier | **`xlarge`** — forced, because packed weights are 67.23 GiB and `xlarge_threshold` is 42.00 GiB (PLAN §8.6) |
131
- | Multiplier | **2×**. A 75 s run costs **150 GPU-seconds** |
132
  | Practical ceiling | **~16 runs/day** |
133
  | **HARD STOP** | **1920 GPU-seconds (80%)** |
134
 
@@ -136,6 +136,19 @@ its economics do not apply to hinode and must not leak into Part A.
136
  be fixed and retested on the same day. A test budget with no reserve is a test budget that
137
  guarantees the last bug found is the one you cannot verify fixed.
138
 
 
 
 
 
 
 
 
 
 
 
 
 
 
139
  ### B.1.1 Two metering paths — they are not the same number
140
 
141
  This distinction decides how we set `duration=`:
@@ -149,9 +162,10 @@ So: **keep `duration=` tight.** An over-generous request costs nothing in billin
149
  in schedulability — late in the day, a `duration=300` reservation fails against 200 s remaining
150
  while the actual 40 s run would have fit fine.
151
 
152
- Two related traps from PLAN §5.8: `large` applies a hidden **1.5× `duration_factor`** (so
153
- `duration=60` schedules as 90 s), while `xlarge` skips that factor but **doubles the request**
154
- instead. And the maximum duration is a **server-side, tier-dependent, unpublished cap** —
 
155
  discover it, never hardcode it.
156
 
157
  ---
 
10
  | RAM | 124 GB | ~68 GB (community observation) |
11
  | disk | 136 GB free, persistent | 150 GB cap, **wiped on cold start** |
12
  | tensor packing | none | O_DIRECT to `~/.zerogpu/tensors` |
13
+ | budget | time only | **2400 GPU-s/day**; an xlarge multiplier is *assumed*, see §B.1 |
14
  | status | **critical path** | **parked** |
15
 
16
  **Label every claim.** A mode that works on hinode is **HOST-PROVEN**; whether it
 
127
  | | |
128
  |---|---|
129
  | PRO quota | **2400 GPU-seconds/day**, fixed 24 h window starting at first use |
130
+ | Our tier | **`xlarge`** — forced, because packed weights are 67.23 GiB and `xlarge_threshold` is **42.00 GiB exactly** (`45097156608` bytes, **verified in the platform's own `spaces/zero/configs.json`**) |
131
+ | Multiplier | **assumed 2×** — ⚠️ **UNVERIFIED.** A 75 s run is budgeted at 150 GPU-seconds |
132
  | Practical ceiling | **~16 runs/day** |
133
  | **HARD STOP** | **1920 GPU-seconds (80%)** |
134
 
 
136
  be fixed and retested on the same day. A test budget with no reserve is a test budget that
137
  guarantees the last bug found is the one you cannot verify fixed.
138
 
139
+ > ⚠️ **The 2× multiplier is unverified, and it sized this entire matrix.**
140
+ > Reading the `spaces` 0.51.1 client turned up `duration_factor = 1.5` applied at
141
+ > `client.py:112` as `if gpu_size != 'xlarge': duration_seconds *= duration_factor` — so the
142
+ > **1.5× inflation applies to `large` and NOT to `xlarge`, which is what we get** — but **no 2×
143
+ > xlarge billing multiplier appears anywhere in the client.** It may well be server-side, which
144
+ > is why this is downgraded rather than deleted.
145
+ >
146
+ > Consequence if it does not exist: the 7-run budget of 930 GPU-seconds is really ~465, and the
147
+ > matrix is half as expensive as planned. **Budgeting the pessimistic number is correct** — being
148
+ > wrong about it costs unused quota, while being wrong the other way costs a reservation failure
149
+ > mid-matrix. Do not re-plan around the cheaper figure until a real run's *billed* seconds
150
+ > confirm it. **The first run is the measurement**: compare billed against requested.
151
+
152
  ### B.1.1 Two metering paths — they are not the same number
153
 
154
  This distinction decides how we set `duration=`:
 
162
  in schedulability — late in the day, a `duration=300` reservation fails against 200 s remaining
163
  while the actual 40 s run would have fit fine.
164
 
165
+ One trap from PLAN §5.8, now **confirmed at source**: `large` applies a hidden
166
+ **1.5× `duration_factor`** (so `duration=60` schedules as 90 s). `spaces/zero/client.py:112`
167
+ gates it on `gpu_size != 'xlarge'`, so it does **not** apply to us. The claim that `xlarge`
168
+ "doubles the request instead" is the unverified half — see the warning above. And the maximum duration is a **server-side, tier-dependent, unpublished cap** —
169
  discover it, never hardcode it.
170
 
171
  ---
ltx25_studio/pipelines.py CHANGED
@@ -207,6 +207,19 @@ def _base_pipeline(*, dtype=None, device: str | None = None):
207
  ) from exc
208
 
209
  try:
 
 
 
 
 
 
 
 
 
 
 
 
 
210
  built = LTX2ConditionPipeline.from_pretrained(
211
  model_files.REPO_ID,
212
  revision=model_files.REVISION,
 
207
  ) from exc
208
 
209
  try:
210
+ # ⚠️ `torch_dtype` MUST match the checkpoint's on-disk dtype. It is not a
211
+ # free parameter, and the reason is invisible from here.
212
+ #
213
+ # ZeroGPU frees the 72 GB HF snapshot after packing, but only by reading
214
+ # /proc/self/maps, mapping each tensor's data_ptr() back to the blob it is
215
+ # mmap'd from, and unlinking those. Anything that materialises tensors
216
+ # into anonymous memory -- a dtype CAST, a .clone(), an eager
217
+ # low_cpu_mem_usage=False, a non-safetensors path -- leaves nothing in the
218
+ # map to capture. Then nothing is freed, 72 GB stays on disk forever, and
219
+ # there is no error and no log line saying so.
220
+ #
221
+ # The checkpoint is bf16 and WEIGHT_DTYPE is bfloat16, so this is a
222
+ # no-op load rather than a conversion. Do not "helpfully" add a cast.
223
  built = LTX2ConditionPipeline.from_pretrained(
224
  model_files.REPO_ID,
225
  revision=model_files.REVISION,
ltx25_studio/preflight.py CHANGED
@@ -15,10 +15,13 @@ byte is touched:
15
  that failure cannot happen -- so we check the invariant instead of hoping for
16
  it, and a miss names the file AND its URL.
17
 
18
- Note we deliberately do NOT move ``HF_HOME`` to dodge the permission problem.
19
- ``preload_from_hub`` writes to the default cache and does not follow a moved
20
- ``HF_HOME``, so relocating it makes the app look somewhere empty and
21
- re-download 72 GB (PLAN §6.2).
 
 
 
22
 
23
  3. **Resource preflight.** Measure free RAM and free disk against the
24
  requirement computed from the resolved config, and raise an error that NAMES
@@ -87,8 +90,9 @@ def ensure_weights(
87
  log = logging.getLogger("ltx25_studio")
88
  paths = model_files.download_allow_list(components)
89
  total_gb = model_files.total_size_gb(components)
90
- log.info("fetching %d files (%.2f GB) at revision %s", len(paths), total_gb,
91
- model_files.REVISION)
 
92
  fetched: list[Path] = []
93
  for index, path in enumerate(paths, 1):
94
  # Progress is not decoration. This is a 72 GB cold start on every restart,
 
15
  that failure cannot happen -- so we check the invariant instead of hoping for
16
  it, and a miss names the file AND its URL.
17
 
18
+ Note we deliberately do NOT move ``HF_HOME``, and there are two reasons --
19
+ the second sharper than the first. ``preload_from_hub`` writes to the default
20
+ cache and does not follow a moved ``HF_HOME``, so relocating it makes the app
21
+ look somewhere empty and re-download 72 GB. **And ZeroGPU's post-pack cleanup
22
+ unlinks blobs matching** ``~/.cache/huggingface/hub/models--*/blobs/*``
23
+ **specifically** -- move the cache out from under that pattern and the 72 GB
24
+ snapshot is never freed, silently (PLAN §6.2, §8.3).
25
 
26
  3. **Resource preflight.** Measure free RAM and free disk against the
27
  requirement computed from the resolved config, and raise an error that NAMES
 
90
  log = logging.getLogger("ltx25_studio")
91
  paths = model_files.download_allow_list(components)
92
  total_gb = model_files.total_size_gb(components)
93
+ log.info(
94
+ "fetching %d files (%.2f GB) at revision %s", len(paths), total_gb, model_files.REVISION
95
+ )
96
  fetched: list[Path] = []
97
  for index, path in enumerate(paths, 1):
98
  # Progress is not decoration. This is a 72 GB cold start on every restart,