README.md

metadata

title: GPU Capacity
emoji: 🛰️
colorFrom: green
colorTo: red
sdk: docker
app_port: 7860
pinned: false
hf_oauth: true
hf_oauth_scopes:
  - email
hf_oauth_authorized_org: huggingface

gpu-capacity

Near real-time AWS GPU on-demand capacity checker. Answers "where can I spawn a G5 right now?" by probing real capacity: it creates a 1-instance targeted On-Demand Capacity Reservation in each AZ and cancels it immediately. Success means the capacity actually exists — unlike spot placement scores or instance type offerings, which are proxies.

Usage

AWS_PROFILE=<profile> uv run gpu-capacity
# then open http://127.0.0.1:8300

One representative (smallest) size per GPU family is swept automatically when AUTO_REFRESH_MINUTES is set (the deployed Space uses 15); any other size can be probed on demand from the dashboard input or the API. Results are cached in state.json and shown with their age; they fade once older than STALE_AFTER_SECONDS (default 900).

On the Space, STATE_FILE lives in the mounted HF bucket (rtrm/gpu-capacity-storage on /data), so results survive restarts — and if the restored state is fresher than the refresh period, the startup sweep is skipped entirely.

API

• GET /api/availability/{instance_type} — cached availability for one type (?refresh=true probes synchronously first, ?region=eu-west-1 narrows)
• GET /api/state — full cached grid (types × regions, per-AZ detail)
• POST /api/probe — {"instance_type": "g5.12xlarge", "region": "eu-west-1"} (region optional → sweeps all configured regions; any type accepted)

Reading is public. Probing (POST /api/probe, refresh=true) requires a "Sign in with HF" session and is restricted to Hugging Face org members (hf_oauth_authorized_org gates the sign-in; the backend re-checks org membership / a verified @huggingface.co email). Sign in from the direct URL (https://rtrm-gpu-capacity.hf.space) — session cookies don't work inside the hf.co iframe.

Config

Types and regions: gpu_capacity/config.py. Env vars: AUTO_REFRESH_MINUTES (background sweep, default 0 = off), STALE_AFTER_SECONDS, HOST, PORT, STATE_FILE. AWS credentials come from the default boto3 chain (AWS_PROFILE locally, AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY secrets on the Space).

Cost & safety

• Reservations are billed per second at the on-demand rate; a probe lives ~1–3 s, so a full 8-region sweep for one type costs well under $0.01 (except p4d/p5 which are usually not ODCR-able anyway — they show up as unsupported, i.e. Capacity Blocks territory).
• Only immediate reservations are created (no StartDate/ CommitmentDuration), which are free to cancel at any time. Future-dated reservations and Capacity Blocks — the ones with commitments — go through different parameters/APIs and cannot be triggered accidentally.
• Every probe reservation gets EndDate = now + 30 min (auto-expires if the service dies between create and cancel), a purpose=capacity-probe tag, and a startup sweep cancels any leaked tagged reservation.

Reading the grid

status	meaning
available (n/m AZ)	reservation succeeded in n of m offered AZs
no capacity	AWS returned InsufficientInstanceCapacity everywhere
quota / denied	account quota or IAM/SCP blocks the check (not a capacity signal)
n/s	type not reservable via ODCR there (p5 → Capacity Blocks)
—	type not offered in that region

Note: results are account-specific (quotas, SCPs), which is what you want — "can this account get a G5 there".

Latency: an available answer comes back in ~1 s per region; a no capacity answer takes 10–20 s because EC2 genuinely attempts placement before returning InsufficientInstanceCapacity. Probe creates run with retries disabled — botocore would otherwise retry that error (HTTP 500) with backoff and multiply the wait.

IAM

ec2:CreateCapacityReservation, ec2:CancelCapacityReservation, ec2:DescribeCapacityReservations, ec2:DescribeInstanceTypeOfferings, ec2:CreateTags (on capacity reservations).