Spaces:
Running
Running
| title: GPU Capacity | |
| emoji: 🛰️ | |
| colorFrom: green | |
| colorTo: red | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| hf_oauth: true | |
| hf_oauth_scopes: | |
| hf_oauth_authorized_org: huggingface | |
| # gpu-capacity | |
| Near real-time AWS GPU on-demand capacity checker. Answers "where can I spawn a | |
| G5 *right now*?" by probing real capacity: it creates a 1-instance targeted | |
| On-Demand Capacity Reservation in each AZ and cancels it immediately. Success | |
| means the capacity actually exists — unlike spot placement scores or instance | |
| type offerings, which are proxies. | |
| ## Usage | |
| ```bash | |
| AWS_PROFILE=<profile> uv run gpu-capacity | |
| # then open http://127.0.0.1:8300 | |
| ``` | |
| One representative (smallest) size per GPU family is swept automatically when | |
| `AUTO_REFRESH_MINUTES` is set (the deployed Space uses 15); any other size can | |
| be probed on demand from the dashboard input or the API. Results are cached in | |
| `state.json` and shown with their age; they fade once older than | |
| `STALE_AFTER_SECONDS` (default 900). | |
| On the Space, `STATE_FILE` lives in the mounted HF bucket | |
| (`rtrm/gpu-capacity-storage` on `/data`), so results survive restarts — and if | |
| the restored state is fresher than the refresh period, the startup sweep is | |
| skipped entirely. | |
| ### API | |
| - `GET /api/availability/{instance_type}` — cached availability for one type | |
| (`?refresh=true` probes synchronously first, `?region=eu-west-1` narrows) | |
| - `GET /api/state` — full cached grid (types × regions, per-AZ detail) | |
| - `POST /api/probe` — `{"instance_type": "g5.12xlarge", "region": "eu-west-1"}` | |
| (`region` optional → sweeps all configured regions; any type accepted) | |
| Reading is public. Probing (`POST /api/probe`, `refresh=true`) requires a | |
| "Sign in with HF" session and is restricted to Hugging Face org members | |
| (`hf_oauth_authorized_org` gates the sign-in; the backend re-checks org | |
| membership / a verified @huggingface.co email). Sign in from the direct URL | |
| (https://rtrm-gpu-capacity.hf.space) — session cookies don't work inside the | |
| hf.co iframe. | |
| ### Config | |
| Types and regions: `gpu_capacity/config.py`. Env vars: `AUTO_REFRESH_MINUTES` | |
| (background sweep, default 0 = off), `STALE_AFTER_SECONDS`, `HOST`, `PORT`, | |
| `STATE_FILE`. AWS credentials come from the default boto3 chain | |
| (`AWS_PROFILE` locally, `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` secrets on | |
| the Space). | |
| ## Cost & safety | |
| - Reservations are billed per second at the on-demand rate; a probe lives | |
| ~1–3 s, so a full 8-region sweep for one type costs well under $0.01 | |
| (except p4d/p5 which are usually not ODCR-able anyway — they show up as | |
| `unsupported`, i.e. Capacity Blocks territory). | |
| - Only *immediate* reservations are created (no `StartDate`/ | |
| `CommitmentDuration`), which are free to cancel at any time. Future-dated | |
| reservations and Capacity Blocks — the ones with commitments — go through | |
| different parameters/APIs and cannot be triggered accidentally. | |
| - Every probe reservation gets `EndDate = now + 30 min` (auto-expires if the | |
| service dies between create and cancel), a `purpose=capacity-probe` tag, and | |
| a startup sweep cancels any leaked tagged reservation. | |
| ## Reading the grid | |
| | status | meaning | | |
| |---|---| | |
| | `available` (n/m AZ) | reservation succeeded in n of m offered AZs | | |
| | `no capacity` | AWS returned `InsufficientInstanceCapacity` everywhere | | |
| | `quota` / `denied` | account quota or IAM/SCP blocks the check (not a capacity signal) | | |
| | `n/s` | type not reservable via ODCR there (p5 → Capacity Blocks) | | |
| | `—` | type not offered in that region | | |
| Note: results are account-specific (quotas, SCPs), which is what you want — | |
| "can *this* account get a G5 there". | |
| Latency: an `available` answer comes back in ~1 s per region; a `no capacity` | |
| answer takes 10–20 s because EC2 genuinely attempts placement before returning | |
| `InsufficientInstanceCapacity`. Probe creates run with retries disabled — | |
| botocore would otherwise retry that error (HTTP 500) with backoff and multiply | |
| the wait. | |
| ## IAM | |
| `ec2:CreateCapacityReservation`, `ec2:CancelCapacityReservation`, | |
| `ec2:DescribeCapacityReservations`, `ec2:DescribeInstanceTypeOfferings`, | |
| `ec2:CreateTags` (on capacity reservations). | |