AGENTS.md

AGENTS.md

Guidance for AI agents working in this repository.

Commands

npm run dev           # Vite (5180) + detection broker API (8787)
npm run build         # Type-check + Vite bundle → docs/ (base /webgpu-video-ai/ for GitHub Pages)
npm run build:space   # Same, but base / for Hugging Face Spaces
npm run start         # Production broker + static UI (SERVE_STATIC=1, PORT from env)
npm run preview       # Serve the production build locally
npm run dev:web       # Vite only
npm run dev:api       # API broker only (8787)

No test suite exists — verify manually in the browser and via curl.

Architecture

Browser GPU detection cluster: RF-DETR (and optional SmolVLM) inference runs in Web Workers on WebGPU; a Node broker exposes HTTP so backends can submit jobs.

Data flow

curl / Python / app
  → Node broker (server/) — task queue, image fetch
  → SSE → browser host (src/pages/addNode.ts)
  → ObjectDetector / VideoDescriber → *.worker.ts (Transformers.js, WebGPU)
  → results POST back to broker → HTTP response

Key modules

Path	Role
src/pages/addNode.ts	Registers host, listens for SSE tasks, runs inference
src/pages/clusterMonitor.ts	Monitor dashboard (monitor.html)
src/detection/ObjectDetector.ts	Main-thread API to the detection worker
src/detection/detection.worker.ts	RF-DETR; model: onnx-community/rfdetr_medium-ONNX
src/videodescription/VideoDescriber.ts	SmolVLM image description worker API
server/index.ts	Express broker: /v1/detect, /v1/describe, host registration, SSE
shared/clusterModels.ts	Model ids exposed to host UI and GET /v1/models

Web Worker and COOP/COEP

Transformers.js uses device: 'webgpu'. The dev server and production broker set Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp (see vite.config.ts and server/index.ts).

Build outputs

Target	Command	Base path	Served by
GitHub Pages	npm run build	/webgpu-video-ai/	static host of docs/
Hugging Face Space	npm run build:space	/	Docker Space (Dockerfile)

Transformers.js is excluded from Vite optimizeDeps; models download at runtime from the Hub.

Hugging Face Space

Repo	apssouza22/webgpu-cluster
Live app	apssouza22-webgpu-cluster.hf.space
Monitor	apssouza22-webgpu-cluster.hf.space/monitor.html
SDK	Docker (app_port: 7860)

The Space container runs the Node broker and serves prebuilt static files from docs/. WebGPU inference runs in the visitor’s browser, not on Space hardware — a browser tab must stay open with Start hosting for API calls to succeed.

Deploy a new version

1. npm run build:space — refresh docs/ (base /; never use npm run build for Spaces).
2. hf upload apssouza22/webgpu-cluster . . --repo-type space --exclude ".git/*" --exclude "node_modules/*" --commit-message "…"
3. Wait for Space stage RUNNING; verify curl https://apssouza22-webgpu-cluster.hf.space/health.

Requires hf auth login. The Dockerfile does not run Vite; always build locally first. Use project .npmrc so package-lock.json stays on the public npm registry.

Full checklist: SPACES.md § Deploy a new version.

Space vs local dev

	Local (npm run dev)	Space
UI	http://localhost:5180	https://apssouza22-webgpu-cluster.hf.space/
API	proxied via Vite to :8787	same origin on :7860
Build	npm run build or build:space	npm run build:space only