README.md

metadata

title: GPU Detection Cluster
emoji: 🎮
colorFrom: blue
colorTo: purple
sdk: docker
app_port: 7860
pinned: false
models:
  - onnx-community/rfdetr_medium-ONNX
  - HuggingFaceTB/SmolVLM-500M-Instruct

WebGPU Cluster

A distributed inference grid that turns browsers with WebGPU into cluster nodes. Host RF-DETR object detection or SmolVLM image description in a Web Worker; a Node broker queues tasks and exposes them over HTTP so any client (curl, Python, Node, etc.) can call your GPU.

Repository: github.com/apssouza22/webgpu-video-cluster
Live demo (Hugging Face Space): apssouza22-webgpu-cluster.hf.space · Space repo

curl / Python / app  →  Node broker (task queue)  →  SSE  →  browser host (WebGPU)
                                                              ↓
                                                    RF-DETR · SmolVLM

Inference runs in the host’s browser on their hardware — not on the broker machine. The broker only coordinates tasks and fetches remote images.

Quick start

npm install
npm run dev

1. Open http://localhost:5180 (landing page)
2. Click Join the grid (or open http://localhost:5180/host.html), choose a model to share, pick a host id (e.g. my-gpu-node), and click Start hosting (loads the model on WebGPU; keep the tab open)
3. Open http://localhost:5180/monitor.html to see registered hosts and copy curl examples
4. From another terminal:

curl -X POST 'http://localhost:5180/v1/detect' \
  -H 'Content-Type: application/json' \
  -d '{
    "host": "my-gpu-node",
    "image_url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/coco_sample.png",
    "threshold": 0.5
  }'

Response:

{
  "task_id": "...",
  "host": "my-gpu-node",
  "threshold": 0.5,
  "detections": [
    { "label": "cat", "score": 0.92, "box": { "xmin": 10, "ymin": 20, "xmax": 200, "ymax": 180 } }
  ]
}

npm run dev runs Vite (port 5180) and the API broker (port 8787). Vite proxies /v1, /api, and /health to the broker.

Endpoints

Method	Path	Description
POST	/v1/detect	Object detection with RF-DETR (waits for host)
POST	/v1/describe	Image description with SmolVLM (waits for host)
GET	/v1/hosts	List registered hosts and online status
GET	/v1/models	Cluster models (id, label, implementation status)
GET	/v1/tasks/:id	Task status / results
GET	/health	Broker health check
—	/	Landing page — join or view the grid
—	/host.html	Browser host — register and share GPU
—	/monitor.html	Dashboard — live host list and curl examples
POST	/api/hosts/register	Called by the browser host
GET	/api/hosts/stream?host_id=	SSE — browser host receives tasks

POST /v1/detect body:

• host (required) — id from the browser host page
• image_url or image_base64 (required) — broker fetches URLs server-side
• threshold (optional, default 0.5)

POST /v1/describe body:

• host (required)
• image_url or image_base64 (required)
• instruction (optional, default "What do you see?")
• max_new_tokens (optional, default 100)

Scripts

Command	Description
npm run dev	Vite (5180) + API broker (8787)
npm run dev:api	API broker only
npm run dev:web	Vite only (proxies API when broker is running)
npm run build	Typecheck + production bundle to docs/ (base /webgpu-video-ai/ for GitHub Pages)
npm run build:space	Same bundle with base / for Hugging Face Spaces
npm run preview	Serve the production build locally
npm run start	Broker + static UI (SERVE_STATIC=1, PORT default 8787)
npm run start:api	Run broker without watch

Cluster models

Models are defined in shared/clusterModels.ts. The host page dropdown and GET /v1/models both read from that list. To add a model: add an entry there, wire loading in src/pages/addNode.ts (ensureModelLoaded), add a task handler under src/tasks/, and register broker routes in server/.

Model id	Endpoint	Worker
rfdetr-medium	POST /v1/detect	src/detection/detection.worker.ts
smolvlm-500m	POST /v1/describe	src/videodescription/videodescription.worker.ts

• RF-DETR: onnx-community/rfdetr_medium-ONNX via @huggingface/transformers with device: 'webgpu'
• SmolVLM: HuggingFaceTB/SmolVLM-500M-Instruct with quantized vision/decoder weights (same approach as the SmolVLM realtime WebGPU demo)

Models download from Hugging Face on first load. Inference runs in a Web Worker; the broker sends base64 images and the host converts them to VideoFrame for the worker.

Requirements

• Browser with WebGPU (Chrome or Edge desktop recommended)
• Dev server COOP/COEP headers in vite.config.ts (required for Transformers.js / WASM)

Project layout

Path	Role
server/	Express broker — task queue, host registry, SSE
src/pages/addNode.ts	Browser host — register, pull tasks, run inference
src/pages/clusterMonitor.ts	Monitor dashboard
src/detection/	RF-DETR worker and main-thread API
src/videodescription/	SmolVLM worker and main-thread API
shared/clusterModels.ts	Model catalog for UI and API

Hugging Face Space

Hosted at apssouza22/webgpu-cluster (Docker SDK, port 7860).

	URL
Landing	apssouza22-webgpu-cluster.hf.space
Host UI	apssouza22-webgpu-cluster.hf.space/host.html
Monitor	apssouza22-webgpu-cluster.hf.space/monitor.html

1. Open the host UI in Chrome or Edge (WebGPU required).
2. Choose a host id and model, then click Start hosting — keep the tab open.
3. Call the API on the same origin:

curl -X POST 'https://apssouza22-webgpu-cluster.hf.space/v1/detect' \
  -H 'Content-Type: application/json' \
  -d '{
    "host": "my-gpu-node",
    "image_url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/coco_sample.png",
    "threshold": 0.5
  }'

The Space container serves the broker and static files only; inference runs in the visitor’s browser.

Deploy a new version

npm run build:space
hf upload apssouza22/webgpu-cluster . . \
  --repo-type space \
  --exclude ".git/*" \
  --exclude "node_modules/*" \
  --commit-message "Your change summary"

Wait until the Space shows Running, then check curl https://apssouza22-webgpu-cluster.hf.space/health. Full steps: SPACES.md.

License

MIT