docs/zero-gputeo.md

# Research: Building an Orchestrator/Queue System for Hugging Face ZeroGPU Beyond 120s

**Date:** 2026-06-06 **Status:** Research brief with honest feasibility assessment

---

## Executive Summary

**ZeroGPU is not viable for tasks that genuinely exceed \~120 seconds of continuous GPU work.** The hard limit is enforced at the infrastructure level — the forked GPU process is killed by the ZeroGPU scheduler, and this kill cannot be caught or intercepted. However, there is a meaningful middle ground: tasks that *conceptually* need more than 120s can be decomposed into discrete GPU calls of ≤120s each, with state persisted between calls. This requires careful architecture and comes with significant trade-offs. For production workloads that need unbounded GPU time, dedicated GPU hosting ($0.50–$5.00/hr) is the honest answer.

---

## 1\. ZeroGPU Limitations: What Actually Happens

### 1.1 The 120-Second Timeout

**Hard limit: 120 seconds.** The `@spaces.GPU(duration=...)` parameter accepts values up to 120 seconds reliably. Values between 120–240s sometimes partially work but are unreliable and may be rejected with `"ZeroGPU illegal duration"` errors. Values above 240s are rejected outright. ([HF Forums — illegal duration](https://discuss.huggingface.co/t/zerogpu-illegal-duration-the-requested-gpu-duration-300s-is-larger-than-the-maximum-allowed/140849), [zero-gpu-explorers discussions](https://huggingface.co/spaces/zero-gpu-explorers/README/discussions/34))

**What happens at timeout:** The ZeroGPU runtime *forks* the main process, attaches a GPU slice, runs the decorated function, and then kills the fork. At timeout, the fork is **killed externally** — this is not a Python exception you can catch with `try/except`. The process receives a kill signal (effectively SIGKILL), and the GPU slice is torn down. The Gradio UI sees a "GPU Task aborted" or "ZeroGPU worker error" message. ([HF blog — zerogpu-aoti](https://huggingface.co/blog/zerogpu-aoti), [HF Forums — GPU Task aborted](https://discuss.huggingface.co/t/when-i-try-to-use-zerogpu-it-keeps-saying-gpu-task-aborted/106618))

**Key implications:**

- No `try/finally` cleanup runs at timeout — the process is just dead  
- No graceful shutdown, no state serialization at timeout boundary  
- The main process (Gradio server) survives; only the fork dies  
- You **cannot** heartbeat or keepalive your way past the timeout

### 1.2 State Persistence Between Calls

**Model loading works across calls.** Models loaded at module level with `.to('cuda')` use ZeroGPU's CUDA emulation mode. The model weights reside in the main process's memory (CPU-side CUDA emulation). When `@spaces.GPU` is called, the fork inherits the process memory space including model weights, and a real GPU is attached. The model is effectively "warm" for each fork. ([Official docs](https://huggingface.co/docs/hub/spaces-zerogpu))

**What persists:**

- Model weights loaded at module level (inherited by each fork)  
- Global Python state (module-level variables, caches)  
- Files on disk (ephemeral — lost on Space restart)  
- Files in HF Storage Buckets (persistent across restarts)

**What does NOT persist:**

- GPU memory state (each fork starts with a fresh GPU)  
- CUDA context, kernels, compiled graphs (unless AoT-compiled and saved to disk)  
- In-progress tensor computations

### 1.3 Re-entrant / Multiple `@spaces.GPU` Calls

**Yes, you can call `@spaces.GPU` functions multiple times** from within the same request. Each call triggers a separate fork → GPU allocation → execute → kill cycle. The main process orchestrates these calls sequentially.

\# This IS valid \- two separate GPU calls in one request

@spaces.GPU(duration=60)

def encode\_text(prompt):

    return text\_encoder(prompt)

@spaces.GPU(duration=60)

def run\_diffusion(latents, steps):

    return unet(latents, num\_inference\_steps=steps)

def generate(prompt):

    latents \= encode\_text(prompt)     \# First GPU call

    images \= run\_diffusion(latents, 20\)  \# Second GPU call

    return images

**Caveat:** Each call consumes the user's GPU quota independently. Two 60s calls \= 120s of quota consumed. The user waits through two separate GPU queue \+ allocation cycles.

### 1.4 Duration Quota System

| Account Type | Daily GPU Quota | Queue Priority |
| :---- | :---- | :---- |
| Unauthenticated | 2 minutes | Low |
| Free account | 5 minutes | Medium |
| Pro ($9/mo) | 40 minutes | Highest |
| Team/Enterprise | 40–60 minutes | Highest |

Pro/Team/Enterprise users can extend beyond quota using credits at **$1 per 10 minutes** of GPU time. ([Official docs](https://huggingface.co/docs/hub/spaces-zerogpu))

**Important:** Quota is charged based on the *reserved* duration, not actual runtime. A `@spaces.GPU(duration=120)` call that finishes in 30s still consumes 120s of quota.

### 1.5 Hardware Specs

Current ZeroGPU hardware: **NVIDIA RTX Pro 6000 Blackwell** (successor to H200).

- `large` (default): half GPU, 48GB VRAM, 1× quota cost  
- `xlarge`: full GPU, 96GB VRAM, 2× quota cost

---

## 2\. Existing Solutions: How Big Demos Handle This

### 2.1 The AoT Compilation Strategy (Most Successful)

The official Hugging Face approach for video generation (Wan, LTX, Flux) is **not** to work around the timeout — it's to make inference fast *enough* to fit within 120s. The blog post [Make your ZeroGPU Spaces go brrr with ahead-of-time compilation](https://huggingface.co/blog/zerogpu-aoti) documents 1.3×–1.8× speedups using:

1. **PyTorch AoT compilation** (`torch.export` \+ `AOTInductor`) — compile once, reload instantly  
2. **FP8 quantization** — additional 1.2× speedup on H200's FP8 hardware  
3. **Flash Attention 3** — pre-built kernels via `kernels` library  
4. **Regional compilation** — compile repeated transformer blocks once, reduce cold-start from 6min to 30sec

**Real-world example:** The [LTX-Video Playground](https://huggingface.co/spaces/Lightricks/LTX-Video-Playground/blob/main/app.py) uses `@spaces.GPU(duration=120)` with dynamic duration estimation:

def get\_duration(steps, duration\_seconds, ...):

    if steps \> 4 and duration\_seconds \> 2:

        return 90

    elif steps \> 4 or duration\_seconds \> 2:

        return 75

    else:

        return 60

@spaces.GPU(duration=get\_duration)

def generate\_video(...):

    ...

**Real-world example:** The [Wan 2.2 AoTI Space](https://huggingface.co/spaces/r3gm/wan2-2-fp8da-aoti-preview/resolve/main/app.py) uses a sophisticated duration estimator:

def get\_inference\_duration(...):

    BASE\_FRAMES\_HEIGHT\_WIDTH \= 81 \* 832 \* 624

    BASE\_STEP\_DURATION \= 15

    factor \= num\_frames \* width \* height / BASE\_FRAMES\_HEIGHT\_WIDTH

    step\_duration \= BASE\_STEP\_DURATION \* factor \*\* 1.5

    gen\_time \= int(steps) \* step\_duration

    return max(60, min(gen\_time, 120))

### 2.2 The "Keep It Short" Pattern

Most production ZeroGPU Spaces simply **limit what users can request**:

- Cap resolution (512×512 instead of 1024+)  
- Cap step count (20–30 steps)  
- Cap video length (2–4 seconds)  
- Offer quality presets (Fast/Balanced/Quality) with different step counts

This is the dominant pattern. No major demo tries to exceed 120s.

### 2.3 cbensimon's ZeroGPU Queue

The Space [cbensimon/zero-gpu-queue](https://huggingface.co/spaces/cbensimon/zero-gpu-queue) is a minimal test of ZeroGPU queue mechanics. The related Gradio PRs ([\#4937](https://github.com/gradio-app/gradio/pull/4937), [\#5129](https://github.com/gradio-app/gradio/pull/5129)) fix Gradio's internal queue `max_size` and `concurrency_count` for ZeroGPU. This is about queue *management*, not about exceeding the timeout.

### 2.4 Agent Zero Orchestration

[ScottzillaSystems/agent-zero-orchestration](https://huggingface.co/ScottzillaSystems/agent-zero-fixed) uses `@spaces.GPU(duration=180)` for agent loops. This pushes past 120s but relies on the "sometimes works up to 240s" zone — **not reliable for production**.

---

## 3\. Architecture Patterns

### Pattern A: Checkpoint-and-Resume (Feasible, Complex)

**Concept:** Break work into chunks, save intermediate state to disk/Storage Bucket between `@spaces.GPU` calls.

┌─────────────────────────────────────────────────────────────┐

│                     Main Process (CPU)                       │

│  ┌──────────┐   ┌──────────┐   ┌──────────┐               │

│  │ Chunk 1  │──\>│ Chunk 2  │──\>│ Chunk N  │──\> Result     │

│  │ @GPU(60) │   │ @GPU(60) │   │ @GPU(60) │               │

│  └────┬─────┘   └────┬─────┘   └──────────┘               │

│       │              │                                       │

│       v              v                                       │

│  ┌──────────────────────────────┐                           │

│  │    HF Storage Bucket /data   │                           │

│  │    checkpoint\_step\_001.pt    │                           │

│  │    checkpoint\_step\_002.pt    │                           │

│  └──────────────────────────────┘                           │

└─────────────────────────────────────────────────────────────┘

**Pseudo-code:**

import spaces

import torch

import json

from pathlib import Path

from huggingface\_hub import CommitOperationAdd, HfApi

STATE\_DIR \= Path("/data/checkpoints")  \# Storage Bucket mount

@spaces.GPU(duration=60)

def run\_chunk(prompt\_embeds, chunk\_idx, total\_chunks):

    \# Load checkpoint if resuming

    state\_path \= STATE\_DIR / f"chunk\_{chunk\_idx}.pt"

    if state\_path.exists():

        latents \= torch.load(state\_path)

    else:

        latents \= torch.randn(1, 4, 64, 64, device="cuda")

    

    \# Run N diffusion steps for this chunk

    steps\_per\_chunk \= 10

    for step in range(steps\_per\_chunk):

        latents \= unet\_step(latents, prompt\_embeds, step \+ chunk\_idx \* steps\_per\_chunk)

    

    \# Save checkpoint

    torch.save(latents.cpu(), state\_path)

    

    return chunk\_idx \+ 1  \# Return next chunk index

def generate\_long(prompt, total\_steps=100):

    prompt\_embeds \= encode\_prompt(prompt)  \# CPU or quick GPU call

    

    chunk\_size \= 10  \# steps per GPU call

    total\_chunks \= total\_steps // chunk\_size

    chunk\_idx \= 0

    

    while chunk\_idx \< total\_chunks:

        chunk\_idx \= run\_chunk(prompt\_embeds, chunk\_idx, total\_chunks)

    

    \# Final decode

    return decode\_latents(chunk\_idx)

**Trade-offs:**

- ✅ Works within ZeroGPU constraints  
- ✅ Each chunk respects the timeout  
- ✅ Can resume from checkpoint if Space restarts  
- ❌ Latent tensor serialization to disk is slow (seconds per save)  
- ❌ Each GPU call goes through queue → allocate → fork → execute → kill cycle  
- ❌ User quota consumed per chunk (10 chunks × 60s \= 600s quota for 100-step generation)  
- ❌ Gradio UI must stay alive across multiple sequential GPU calls

**When it works:** Video generation where you can chunk frames (e.g., generate 4 frames per GPU call, accumulate).

### Pattern B: Step-Wise Execution via Generators (Feasible, Better UX)

**Concept:** Use Gradio's generator/streaming pattern to yield intermediate results while making sequential `@spaces.GPU` calls.

import gradio as gr

import spaces

import torch

pipe \= ...  \# loaded at module level

@spaces.GPU(duration=30)

def diffusion\_step(latents, prompt\_embeds, step):

    """Run ONE diffusion step per GPU call."""

    latents \= pipe.scheduler.scale\_model\_input(latents, step)

    noise\_pred \= pipe.unet(latents, step, encoder\_hidden\_states=prompt\_embeds).sample

    latents \= pipe.scheduler.step(noise\_pred, step, latents).prev\_sample

    return latents

def generate\_streaming(prompt, total\_steps=50):

    """Generator that yields intermediate images every N steps."""

    prompt\_embeds \= encode(prompt)  \# quick

    latents \= torch.randn(1, 4, 64, 64\)

    

    for step in range(total\_steps):

        latents \= diffusion\_step(latents, prompt\_embeds, step)

        

        if step % 5 \== 0:

            \# Decode and yield intermediate result

            image \= decode(latents)

            yield image, f"Step {step}/{total\_steps}"

    

    yield decode(latents), "Done\!"

gr.Interface(

    fn=generate\_streaming,

    inputs=\[gr.Text(), gr.Slider(10, 100)\],

    outputs=\[gr.Image(), gr.Text()\],

).launch()

**Trade-offs:**

- ✅ Streaming UI shows progress  
- ✅ Each step is a separate GPU call with its own timeout  
- ❌ **Extremely** quota-expensive: 50 steps × 30s duration \= 1500s quota for what should be a 20s job  
- ❌ Each step goes through GPU allocation queue (latency per step: 2–10s)  
- ❌ Model weights are loaded into GPU memory fresh for each fork (slow if not using AoT)

**Honest assessment:** This pattern is technically possible but **wasteful**. You're paying 25× in quota and 10× in wall-clock time for the privilege of streaming individual diffusion steps. The only scenario where it makes sense is if each "step" is itself a substantial computation (e.g., one frame of video generation that takes 30–60s).

### Pattern C: Dedicated Queue Space \+ Worker Spaces (Advanced, Fragile)

**Concept:** One Space acts as a queue manager (CPU-only), delegating work to multiple ZeroGPU worker Spaces via API calls.

┌──────────────────────────────┐

│   Queue Space (CPU-only)      │

│   \- SQLite / in-memory queue  │

│   \- REST API for job submit   │

│   \- Polling for results       │

│   \- Gradio UI for users       │

└──────────┬───────────────────┘

           │ HTTP API calls

           v

┌──────────────────────────────┐

│   Worker Space A (ZeroGPU)    │

│   @spaces.GPU(duration=120)  │

│   POST /run → process chunk   │

└──────────────────────────────┘

           ...

┌──────────────────────────────┐

│   Worker Space B (ZeroGPU)    │

│   @spaces.GPU(duration=120)  │

│   POST /run → process chunk   │

└──────────────────────────────┘

**How it works:**

1. User submits job via Queue Space's Gradio UI  
2. Queue Space writes job to HF Dataset or Storage Bucket  
3. Queue Space calls Worker Space's API endpoint (Gradio's REST API)  
4. Worker reads input from Storage, runs GPU computation, writes result to Storage  
5. Queue Space polls for result, assembles final output

**Pseudo-code (Queue Space):**

import gradio as gr

import requests

import time

from huggingface\_hub import HfApi

WORKER\_URL \= "https://your-worker-space.hf.space/api/predict"

JOB\_STORAGE \= "username/job-storage"  \# HF Dataset

def submit\_job(prompt, config):

    \# 1\. Write job to storage

    job\_id \= write\_job\_to\_dataset(JOB\_STORAGE, prompt, config)

    

    \# 2\. Call worker

    response \= requests.post(WORKER\_URL, json={

        "data": \[job\_id, prompt, config\]

    })

    

    \# 3\. Poll for completion

    while not job\_complete(JOB\_STORAGE, job\_id):

        time.sleep(2)

        yield None, "Processing..."

    

    \# 4\. Assemble result

    result \= read\_result\_from\_dataset(JOB\_STORAGE, job\_id)

    yield result, "Done\!"

gr.Interface(submit\_job, \[gr.Text(), gr.JSON()\], \[gr.Image(), gr.Text()\]).launch()

**Trade-offs:**

- ✅ Can parallelize across multiple workers  
- ✅ Queue management is clean  
- ❌ **10 ZeroGPU Space limit per Pro account** — can't scale to many workers  
- ❌ Each Space requires separate deployment and maintenance  
- ❌ Cross-Space communication adds latency (HTTP API \+ Storage I/O)  
- ❌ Each Space consumes the *user's* quota when they call it  
- ❌ Gradio's API client adds overhead; no native inter-Space RPC

### Pattern D: Hybrid CPU Orchestration \+ GPU Bursts (Most Practical)

**Concept:** Do everything except inference on CPU in the main process, using `@spaces.GPU` only for the actual computation kernel. Use HF Scheduled Jobs for orchestration if needed.

import gradio as gr

import spaces

import torch

\# Model loaded at module level (CUDA emulation handles .to('cuda'))

pipe \= DiffusionPipeline.from\_pretrained(...).to('cuda')

@spaces.GPU(duration=get\_duration)

def gpu\_kernel(prompt, steps, resolution, seed):

    """Single GPU call — all computation happens here."""

    generator \= torch.Generator("cuda").manual\_seed(seed)

    return pipe(

        prompt,

        num\_inference\_steps=steps,

        height=resolution,

        width=resolution,

        generator=generator,

    ).images\[0\]

def get\_duration(prompt, steps, resolution, seed):

    """Estimate duration to request minimal quota."""

    if resolution \<= 512 and steps \<= 20:

        return 30

    elif resolution \<= 768 and steps \<= 30:

        return 60

    else:

        return 120

\# Limit what users can request to fit within 120s

with gr.Blocks() as demo:

    prompt \= gr.Text()

    resolution \= gr.Dropdown(\[256, 512, 768\], value=512)

    steps \= gr.Slider(1, 50, value=20)

    output \= gr.Image()

    

    generate\_btn \= gr.Button("Generate")

    generate\_btn.click(gpu\_kernel, \[prompt, steps, resolution, gr.Number(42)\], output)

demo.launch()

**This is what the big demos actually do.** They don't exceed 120s — they constrain the problem to fit.

### Pattern E: HF Scheduled Jobs for Long Orchestration (Emerging)

HF now supports [Scheduled Jobs](https://huggingface.co/docs/hub/jobs-schedule) that can run on GPU hardware:

hf jobs scheduled uv run \--flavor a10g-small \--with torch @hourly python orchestrator.py

This is a relatively new feature and could be used for:

- Periodic batch processing  
- Pre-computing common outputs  
- Fine-tuning jobs that run in scheduled windows

However, this uses **paid dedicated GPU**, not ZeroGPU.

---

## 4\. Can You Pass Model State Between ZeroGPU Invocations?

**Yes, but with caveats:**

| Method | Works? | Speed | Notes |
| :---- | :---- | :---- | :---- |
| Global model `.to('cuda')` at module level | ✅ Yes | Fast | Weights in main process memory, inherited by fork |
| Pickle/torch.save to ephemeral disk | ✅ Yes | Slow | Lost on Space restart |
| Save to HF Storage Bucket | ✅ Yes | Very slow | Network I/O per save |
| Save to HF Dataset | ✅ Yes | Very slow | Git-based, not ideal for frequent writes |
| Keep tensor in global variable | ⚠️ Partial | Fast | Survives between calls but NOT across Space restarts |
| CUDA graph persistence | ❌ No | N/A | Each fork gets fresh GPU context |

**Recommendation:** For model weights, use the standard pattern (load at module level, `.to('cuda')`). For intermediate computation state, use ephemeral disk `/data/` for speed, with HF Storage Buckets only if you need persistence across restarts.

---

## 5\. The Gradio UI Timeout Problem

When chaining multiple `@spaces.GPU` calls, the Gradio UI must stay responsive. The key mechanisms:

1. **Generators (`yield`):** Gradio supports generator functions that yield intermediate results. The UI stays alive between yields.  
2. **`gr.Progress()`:** Shows a progress bar during long operations.  
3. **Queue mode:** Gradio's queue system handles concurrent users.

**Problem:** If the total time across all GPU calls exceeds \~5 minutes, users may see "connection timed out" from Gradio's WebSocket. ([HF Forums — connection timed out](https://discuss.huggingface.co/t/why-i-am-getting-connection-timed-out-from-gradio/133599))

**Solution:** Use generator pattern with periodic yields to keep the WebSocket alive, and poll for completion rather than blocking.

---

## 6\. Component Diagram: The Recommended Architecture

For tasks that genuinely need \>120s and you insist on using ZeroGPU:

┌────────────────────────────────────────────────────────────────────────┐

│                         USER'S BROWSER                                  │

│  ┌─────────────┐   ┌──────────────────┐   ┌──────────────────┐        │

│  │ Submit Job   │──\>│ See Progress      │──\>│ Download Result  │        │

│  └─────────────┘   │ (polling/SSE)     │   └──────────────────┘        │

│                     └────────┬─────────┘                                │

└──────────────────────────────┼─────────────────────────────────────────┘

                               │ WebSocket

┌──────────────────────────────┼─────────────────────────────────────────┐

│                    GRADIO SPACE (Main Process, CPU)                     │

│                     ┌────────┴─────────┐                                │

│                     │  Job Orchestrator  │                                │

│                     │  (Python generator)│                                │

│                     └────────┬──────────┘                                │

│                              │                                           │

│            ┌─────────────────┼─────────────────┐                        │

│            v                 v                  v                        │

│  ┌──────────────┐  ┌──────────────┐   ┌──────────────┐                 │

│  │ @GPU(60)     │  │ @GPU(60)     │   │ @GPU(60)     │  ← Sequential  │

│  │ Chunk 1      │  │ Chunk 2      │   │ Chunk N      │    GPU forks   │

│  │ encode+gen   │  │ generate     │   │ generate+dec │                 │

│  └──────┬───────┘  └──────┬───────┘   └──────┬───────┘                 │

│         │                  │                   │                         │

│         v                  v                   v                         │

│  ┌──────────────────────────────────────────────────────┐              │

│  │              HF Storage Bucket (/data/)              │              │

│  │  job\_123\_chunk\_1.pt → job\_123\_chunk\_2.pt → result.pt │              │

│  └──────────────────────────────────────────────────────┘              │

└────────────────────────────────────────────────────────────────────────┘

**Key design decisions:**

1. Single Space (not multi-Space orchestration) to avoid the 10-Space limit  
2. Generator pattern keeps UI alive between chunks  
3. State persisted to Storage Bucket between GPU calls  
4. Each chunk must complete in \<60s (leaving buffer before 120s hard limit)  
5. Dynamic duration estimator minimizes quota waste

---

## 7\. Honest Feasibility Assessment

### Tasks where ZeroGPU \+ orchestration COULD work:

| Task | Feasible? | Strategy |
| :---- | :---- | :---- |
| Video gen (4–8s clips, optimized) | ✅ Yes | AoT \+ FP8, fit in 120s |
| Batch image gen (10 images) | ✅ Yes | Sequential `@spaces.GPU` calls, 1 image each |
| Long text generation (LLM streaming) | ✅ Yes | Already works — streaming is natural |
| Image gen at high res (1024+) | ⚠️ Maybe | AoT compilation \+ FP8 might squeeze it in |
| Video gen (\>10s) | ⚠️ Difficult | Chunk frames, checkpoint between calls |
| Fine-tuning / training | ❌ No | Not what ZeroGPU is designed for |
| Batch processing (100+ items) | ❌ No | Quota exhaustion after a handful of items |
| Real-time video gen (\>30s) | ❌ No | Fundamentally incompatible |

### Tasks where ZeroGPU WILL NOT work:

- **Model fine-tuning** — requires minutes to hours of continuous GPU  
- **Large-scale batch processing** — quota burns too fast  
- **Training / LoRA adaptation** — optimizer state is hard to checkpoint between forks  
- **Real-time video longer than \~10s** — even AoT can't make it fast enough  
- **Multi-stage pipelines with tight dependencies** — inter-GPU-call latency kills throughput

---

## 8\. Cost Comparison: ZeroGPU vs. Alternatives

### ZeroGPU (if you could make it work)

| Tier | Cost | What You Get |
| :---- | :---- | :---- |
| Pro ($9/mo) | $9/month \+ $0.10/min over quota | 40 min/day GPU, H200-class, shared |
| Credits | $1 per 10 min GPU time | Extends beyond daily quota |
| Effective hourly rate | \~$6/hr (at credit rate) | But shared, queued, and time-limited |

### Dedicated GPU Hosting (No Timeout)

| Provider | GPU | Price/hr | Notes |
| :---- | :---- | :---- | :---- |
| HF Inference Endpoints | T4 16GB | $0.50/hr | Cheapest GPU, good for inference |
| HF Inference Endpoints | A10G 24GB | $1.00/hr | Good mid-range |
| HF Inference Endpoints | A100 80GB | $2.50–$5.00/hr | Serious workloads |
| RunPod Secure Cloud | A100 80GB | $2.17/hr | Cheapest A100 |
| RunPod Secure Cloud | H100 80GB | $3.35/hr | Cheapest H100 |
| Modal | A100 80GB | $2.10/hr | Python-native, $30/mo free credit |
| Fal.ai | A100 80GB | $0.99/hr | Cheapest per-GPU-second |
| Replicate | A100 80GB | $5.04/hr | Simplest API, most expensive |

Sources: [HF Endpoints pricing](https://huggingface.co/docs/inference-endpoints/en/pricing), [RunPod pricing](https://www.runpod.io/pricing), [HostFleet comparison](https://hostfleet.net/serverless-gpu-pricing-matrix-2026/), [Replicate pricing](https://replicate.com/pricing)

### Cost Analysis Example: Processing 100 video generations at 5 min each

| Option | Total GPU Time | Cost |
| :---- | :---- | :---- |
| ZeroGPU (Pro credits) | 500 min | $50 in credits \+ $9/mo subscription |
| HF Endpoints (A10G) | 500 min \= \~8.3 hrs | \~$8.30 (shut down between jobs) |
| RunPod Serverless (A100) | 500 min \= \~8.3 hrs | \~$18.00 |
| Fal.ai (A100) | 500 min \= \~8.3 hrs | \~$8.25 |
| Modal (A100) | 500 min \= \~8.3 hrs | \~$17.40 ($30 free credit covers it) |

**Conclusion:** For any sustained GPU workload, dedicated hosting is cheaper than ZeroGPU credits. ZeroGPU's value is the **free tier** (40 min/day for Pro) for demos and prototyping, not production workloads.

---

## 9\. Minimal-Cost Alternative Recommendations

### Tier 1: "I need free/cheap and can tolerate constraints"

**→ ZeroGPU with AoT compilation.** Optimize your model to fit in 120s. Use the strategies from the [official blog post](https://huggingface.co/blog/zerogpu-aoti):

1. AoT compile the transformer/denoiser  
2. Apply FP8 quantization  
3. Use Flash Attention 3  
4. Limit resolution, steps, video length in the UI  
5. Use dynamic duration to minimize quota waste

### Tier 2: "I need reliable long-running GPU but want to stay cheap"

**→ Fal.ai or RunPod Serverless.**

- Fal.ai A100 at $0.99/hr is the cheapest per-GPU-hour option available  
- RunPod Serverless with per-millisecond billing for bursty workloads  
- Both support custom Docker containers with your full pipeline  
- No timeout constraints, no queue system to fight with  
- Add a simple FastAPI \+ Redis queue on top for orchestration

### Tier 3: "I need production-grade inference at scale"

**→ HF Inference Endpoints or Modal.**

- HF Endpoints: managed, auto-scaling, integrates with HF Hub  
- Modal: Python-native, excellent DX, $30/mo free credit, proper autoscaling  
- Both support scale-to-zero to minimize idle costs  
- Add Celery/Redis or Modal's native queuing for task management

### Architecture for Tier 2/3:

┌─────────────┐     ┌──────────┐     ┌──────────────┐

│ Gradio UI   │────\>│ FastAPI   │────\>│ Redis Queue  │

│ (any host)  │\<────│ \+ Celery  │\<────│              │

└─────────────┘     └─────┬─────┘     └──────┬───────┘

                          │                   │

                          v                   v

                    ┌──────────────────────────────────┐

                    │  GPU Worker (RunPod/Modal/Fal)    │

                    │  \- Pulls from Redis queue        │

                    │  \- Runs full pipeline, no timeout │

                    │  \- Writes result to S3/HF Storage │

                    │  \- Marks job complete in Redis    │

                    └──────────────────────────────────┘

---

## 10\. Key Takeaways

1. **The 120s timeout is a hard, uncircumventable limit.** The forked process is killed externally — no Python exception, no graceful shutdown.  
     
2. **The big demos (Flux, Wan, LTX) do NOT exceed 120s.** They use AoT compilation \+ quantization to make inference fast enough, and constrain user inputs to fit within the budget.  
     
3. **Chunking/checkpointing is theoretically possible but impractical** for most use cases. Each chunk goes through the full GPU allocation pipeline (queue → allocate → fork → execute → kill), adding 5–15s overhead per chunk and burning quota at the *reserved* duration rate.  
     
4. **ZeroGPU is designed for demos, not production workloads.** The daily quota, per-call timeout, shared infrastructure, and queue-based scheduling all point to this being a demo/prototyping tool.  
     
5. **If your task genuinely needs \>120s of continuous GPU, use dedicated GPU hosting.** At $0.50–$5.00/hr, it's cheaper than ZeroGPU credits and has no timeout constraints.  
     
6. **The most promising pattern for "long" tasks on ZeroGPU** is: AoT compile \+ FP8 quantize to fit more work per second, use dynamic duration to minimize quota waste, and limit user-facing parameters to stay within 120s. For video: generate short clips (2–4s) and stitch client-side.  
     
7. **HF Storage Buckets** are the right persistence mechanism for checkpointing between GPU calls — mounted as local filesystem, persists across Space restarts, S3-like semantics.

---

## Sources

### Official Documentation

- [Spaces ZeroGPU: Dynamic GPU Allocation for Spaces](https://huggingface.co/docs/hub/spaces-zerogpu) — Official HF docs  
- [Make your ZeroGPU Spaces go brrr with ahead-of-time compilation](https://huggingface.co/blog/zerogpu-aoti) — HF Blog, AoT compilation guide  
- [Disk usage on Spaces / Storage Buckets](https://huggingface.co/docs/hub/spaces-storage) — HF Storage docs  
- [Schedule Jobs](https://huggingface.co/docs/hub/jobs-schedule) — HF Scheduled Jobs  
- [HF Inference Endpoints Pricing](https://huggingface.co/docs/inference-endpoints/en/pricing) — HF Endpoints pricing

### Community Sources

- [ZeroGPU illegal duration discussion](https://discuss.huggingface.co/t/zerogpu-illegal-duration-the-requested-gpu-duration-300s-is-larger-than-the-maximum-allowed/140849) — Confirms 120s hard limit  
- [HF ZeroGPU Space Hangs](https://discuss.huggingface.co/t/hf-zerogpu-space-hangs-no-output-in-the-logs/175410) — Debug patterns  
- [GPU Task aborted](https://discuss.huggingface.co/t/when-i-try-to-use-zerogpu-it-keeps-saying-gpu-task-aborted/106618) — Timeout behavior  
- [ZeroGPU quota discussion](https://discuss.huggingface.co/t/it-seems-use-60-sec-gpu-quota-instead-of-real-time-usage/175130) — Quota mechanics  
- [Debugging ZeroGPU Spaces (KB)](https://huggingface.co/datasets/John6666/knowledge_base_md_for_rag_1/blob/main/hf_spaces_debug_zerogpu_20251203.md) — Comprehensive debug guide

### Example Implementations

- [cbensimon/zero-gpu-queue](https://huggingface.co/spaces/cbensimon/zero-gpu-queue) — Queue mechanics test  
- [LTX-Video Playground](https://huggingface.co/spaces/Lightricks/LTX-Video-Playground/blob/main/app.py) — Dynamic duration  
- [Wan 2.2 AoTI](https://huggingface.co/spaces/r3gm/wan2-2-fp8da-aoti-preview/resolve/main/app.py) — Sophisticated duration estimation  
- [Agent Zero Orchestration](https://huggingface.co/ScottzillaSystems/agent-zero-fixed) — Multi-step agent on ZeroGPU  
- [ltx2.3-AIO-generator](https://github.com/techfreakworm/ltx2.3-AIO-generator) — Multi-mode video gen with presets  
- [spaces library source](https://pypi-browser.org/package/spaces/spaces-0.40.1-py3-none-any.whl/spaces/zero/gradio.py) — ZeroGPU internals

### Pricing Sources

- [HostFleet Serverless GPU Pricing Matrix (April 2026\)](https://hostfleet.net/serverless-gpu-pricing-matrix-2026/) — Cross-vendor comparison  
- [RunPod Pricing](https://www.runpod.io/pricing) — GPU cloud pricing  
- [Replicate Pricing](https://replicate.com/pricing) — Per-second inference pricing  
- [HuggingFace Inference API 2026](https://klymentiev.com/blog/huggingface-inference-api) — API pricing breakdown

---

## Provenance

| Claim | Source | Confidence |
| :---- | :---- | :---- |
| 120s hard duration limit | HF Forums, official docs | High |
| Process killed (SIGKILL-like) at timeout | HF blog ("kills the fork"), community reports | High |
| Duration \>120s sometimes works up to 240s | Community reports | Medium |
| Quota charged at reserved duration, not actual | HF Forums discussion | High |
| AoT gives 1.3–1.8× speedup | HF official blog post | High |
| FP8 gives additional 1.2× speedup | HF official blog post | High |
| Model weights persist across GPU calls | Official docs, spaces source | High |
| 10 ZeroGPU Spaces limit per Pro account | Official docs | High |
| Storage Buckets persist across Space restarts | Official docs | High |
| Pricing figures | Vendor pricing pages as of April 2026 | High |