AGENTS.md

# AGENTS.md

Guidance for AI agents working in this repository.

## Commands

```bash
npm run dev           # Vite (5180) + detection broker API (8787)
npm run build         # Type-check + Vite bundle → docs/ (base /webgpu-video-ai/ for GitHub Pages)
npm run build:space   # Same, but base / for Hugging Face Spaces
npm run start         # Production broker + static UI (SERVE_STATIC=1, PORT from env)
npm run preview       # Serve the production build locally
npm run dev:web       # Vite only
npm run dev:api       # API broker only (8787)
```

No test suite exists — verify manually in the browser and via `curl`.

## Architecture

**Browser GPU detection cluster:** RF-DETR (and optional SmolVLM) inference runs in Web Workers on WebGPU; a Node broker exposes HTTP so backends can submit jobs.

### Data flow

```
curl / Python / app
  → Node broker (server/) — task queue, image fetch
  → SSE → browser host (src/pages/addNode.ts)
  → ObjectDetector / VideoDescriber → *.worker.ts (Transformers.js, WebGPU)
  → results POST back to broker → HTTP response
```

### Key modules

| Path | Role |
|------|------|
| `src/pages/addNode.ts` | Registers host, listens for SSE tasks, runs inference |
| `src/pages/clusterMonitor.ts` | Monitor dashboard (`monitor.html`) |
| `src/detection/ObjectDetector.ts` | Main-thread API to the detection worker |
| `src/detection/detection.worker.ts` | RF-DETR; model: `onnx-community/rfdetr_medium-ONNX` |
| `src/videodescription/VideoDescriber.ts` | SmolVLM image description worker API |
| `server/index.ts` | Express broker: `/v1/detect`, `/v1/describe`, host registration, SSE |
| `shared/clusterModels.ts` | Model ids exposed to host UI and `GET /v1/models` |

### Web Worker and COOP/COEP

Transformers.js uses `device: 'webgpu'`. The dev server and production broker set `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` (see `vite.config.ts` and `server/index.ts`).

### Build outputs

| Target | Command | Base path | Served by |
|--------|---------|-----------|-------------|
| GitHub Pages | `npm run build` | `/webgpu-video-ai/` | static host of `docs/` |
| Hugging Face Space | `npm run build:space` | `/` | Docker Space (`Dockerfile`) |

Transformers.js is excluded from Vite `optimizeDeps`; models download at runtime from the Hub.

## Hugging Face Space

| | |
|--|--|
| **Repo** | [apssouza22/webgpu-cluster](https://huggingface.co/spaces/apssouza22/webgpu-cluster) |
| **Live app** | [apssouza22-webgpu-cluster.hf.space](https://apssouza22-webgpu-cluster.hf.space/) |
| **Monitor** | [apssouza22-webgpu-cluster.hf.space/monitor.html](https://apssouza22-webgpu-cluster.hf.space/monitor.html) |
| **SDK** | Docker (`app_port: 7860`) |

The Space container runs the Node broker and serves prebuilt static files from `docs/`. **WebGPU inference runs in the visitor’s browser**, not on Space hardware — a browser tab must stay open with **Start hosting** for API calls to succeed.

### Deploy a new version

1. `npm run build:space` — refresh `docs/` (base `/`; never use `npm run build` for Spaces).
2. `hf upload apssouza22/webgpu-cluster . . --repo-type space --exclude ".git/*" --exclude "node_modules/*" --commit-message "…"`
3. Wait for Space stage `RUNNING`; verify `curl https://apssouza22-webgpu-cluster.hf.space/health`.

Requires `hf auth login`. The `Dockerfile` does **not** run Vite; always build locally first. Use project `.npmrc` so `package-lock.json` stays on the public npm registry.

Full checklist: [SPACES.md § Deploy a new version](./SPACES.md#deploy-a-new-version).

### Space vs local dev

| | Local (`npm run dev`) | Space |
|--|----------------------|-------|
| UI | http://localhost:5180 | https://apssouza22-webgpu-cluster.hf.space/ |
| API | proxied via Vite to :8787 | same origin on :7860 |
| Build | `npm run build` or `build:space` | `npm run build:space` only |