someone-in-the-world Claude Sonnet 5 commited on
Commit
f27c9e0
·
1 Parent(s): 4d8e705

Pad declared ZeroGPU duration with a cold-start buffer

Browse files

Every @spaces.GPU call runs in a fresh worker, so the int8 text_encoder
reload (up to ~30s) and full pipe-to-cuda move (up to ~14s) are paid on
every request. When the UI's gpu_duration slider only budgeted for the
user's expected diffusion time, ZeroGPU would kill the call mid-run with
"GPU task aborted" once the declared duration was exceeded (issue #7).
Since ZeroGPU bills by actual usage rather than the declared duration,
padding it costs nothing but queue priority.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +14 -1
app.py CHANGED
@@ -613,7 +613,20 @@ def _log_infer_error(e, t0, timer):
613
  timer.print_timings()
614
 
615
 
616
- @spaces.GPU(duration=lambda *a, **kw: int(a[8]) if len(a) > 8 else 60)
 
 
 
 
 
 
 
 
 
 
 
 
 
617
  def _infer_gpu(pil_images, prompt, seed, guidance_scale, steps, width, height, mode: Mode, gpu_duration=20):
618
  _cuda_ok = torch.cuda.is_available()
619
  timer = _InferTimer(_cuda_ok)
 
613
  timer.print_timings()
614
 
615
 
616
+ # Every @spaces.GPU call lands on a fresh worker (see comment above _FAST_PATH_MIN_GB), so
617
+ # _ensure_int8_text_encoder's reload and _place_pipe_on_device's pipe.to(cuda) are paid on
618
+ # every single request, not just a one-time warmup. Observed worst case: ~30s for the int8
619
+ # text_encoder load (HF hub/disk cache miss) + ~14s to move the pipe onto the device — before
620
+ # any diffusion work even starts. The gpu_duration slider only reflects the user's expectation
621
+ # of diffusion+decode time, so pad the declared duration with this buffer — it doesn't cost
622
+ # extra quota (billing is by real usage, not the declared duration — see
623
+ # huggingface.co/docs/hub/spaces-zerogpu) but declaring too little makes ZeroGPU kill the
624
+ # call mid-run with "GPU task aborted".
625
+ _COLD_START_BUFFER_S = 45
626
+ _MAX_GPU_DURATION_S = 120 # matches the gpu_duration slider's max in the UI
627
+
628
+
629
+ @spaces.GPU(duration=lambda *a, **kw: min(int(a[8]) + _COLD_START_BUFFER_S, _MAX_GPU_DURATION_S) if len(a) > 8 else 60)
630
  def _infer_gpu(pil_images, prompt, seed, guidance_scale, steps, width, height, mode: Mode, gpu_duration=20):
631
  _cuda_ok = torch.cuda.is_available()
632
  timer = _InferTimer(_cuda_ok)