README.md

---
title: GPU Capacity
emoji: 🛰️
colorFrom: green
colorTo: red
sdk: docker
app_port: 7860
pinned: false
hf_oauth: true
hf_oauth_scopes:
  - email
hf_oauth_authorized_org: huggingface
---

# gpu-capacity

Near real-time AWS GPU on-demand capacity checker. Answers "where can I spawn a
G5 *right now*?" by probing real capacity: it creates a 1-instance targeted
On-Demand Capacity Reservation in each AZ and cancels it immediately. Success
means the capacity actually exists — unlike spot placement scores or instance
type offerings, which are proxies.

## Usage

```bash
AWS_PROFILE=<profile> uv run gpu-capacity
# then open http://127.0.0.1:8300
```

One representative (smallest) size per GPU family is swept automatically when
`AUTO_REFRESH_MINUTES` is set (the deployed Space uses 15); any other size can
be probed on demand from the dashboard input or the API. Results are cached in
`state.json` and shown with their age; they fade once older than
`STALE_AFTER_SECONDS` (default 900).

On the Space, `STATE_FILE` lives in the mounted HF bucket
(`rtrm/gpu-capacity-storage` on `/data`), so results survive restarts — and if
the restored state is fresher than the refresh period, the startup sweep is
skipped entirely.

### API

- `GET /api/availability/{instance_type}` — cached availability for one type
  (`?refresh=true` probes synchronously first, `?region=eu-west-1` narrows)
- `GET /api/state` — full cached grid (types × regions, per-AZ detail)
- `POST /api/probe` — `{"instance_type": "g5.12xlarge", "region": "eu-west-1"}`
  (`region` optional → sweeps all configured regions; any type accepted)

Reading is public. Probing (`POST /api/probe`, `refresh=true`) requires a
"Sign in with HF" session and is restricted to Hugging Face org members
(`hf_oauth_authorized_org` gates the sign-in; the backend re-checks org
membership / a verified @huggingface.co email). Sign in from the direct URL
(https://rtrm-gpu-capacity.hf.space) — session cookies don't work inside the
hf.co iframe.

### Config

Types and regions: `gpu_capacity/config.py`. Env vars: `AUTO_REFRESH_MINUTES`
(background sweep, default 0 = off), `STALE_AFTER_SECONDS`, `HOST`, `PORT`,
`STATE_FILE`. AWS credentials come from the default boto3 chain
(`AWS_PROFILE` locally, `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` secrets on
the Space).

## Cost & safety

- Reservations are billed per second at the on-demand rate; a probe lives
  ~1–3 s, so a full 8-region sweep for one type costs well under $0.01
  (except p4d/p5 which are usually not ODCR-able anyway — they show up as
  `unsupported`, i.e. Capacity Blocks territory).
- Only *immediate* reservations are created (no `StartDate`/
  `CommitmentDuration`), which are free to cancel at any time. Future-dated
  reservations and Capacity Blocks — the ones with commitments — go through
  different parameters/APIs and cannot be triggered accidentally.
- Every probe reservation gets `EndDate = now + 30 min` (auto-expires if the
  service dies between create and cancel), a `purpose=capacity-probe` tag, and
  a startup sweep cancels any leaked tagged reservation.

## Reading the grid

| status | meaning |
|---|---|
| `available` (n/m AZ) | reservation succeeded in n of m offered AZs |
| `no capacity` | AWS returned `InsufficientInstanceCapacity` everywhere |
| `quota` / `denied` | account quota or IAM/SCP blocks the check (not a capacity signal) |
| `n/s` | type not reservable via ODCR there (p5 → Capacity Blocks) |
| `—` | type not offered in that region |

Note: results are account-specific (quotas, SCPs), which is what you want —
"can *this* account get a G5 there".

Latency: an `available` answer comes back in ~1 s per region; a `no capacity`
answer takes 10–20 s because EC2 genuinely attempts placement before returning
`InsufficientInstanceCapacity`. Probe creates run with retries disabled —
botocore would otherwise retry that error (HTTP 500) with backoff and multiply
the wait.

## IAM

`ec2:CreateCapacityReservation`, `ec2:CancelCapacityReservation`,
`ec2:DescribeCapacityReservations`, `ec2:DescribeInstanceTypeOfferings`,
`ec2:CreateTags` (on capacity reservations).