# Piclets Discovery Server — Architecture This document explains *why* the backend is shaped the way it is. The short version: there is exactly one piece of shared state (a public dataset), exactly one trusted writer (this server), and a dumb client. The platform's limits push you toward that shape rather than fighting it. --- ## 1. The whole flow ``` player's browser (frontend Space) │ photo + player's HF access token (POST /scan) ▼ ┌───────────────────────────┐ │ Piclets Discovery Server │ free CPU Space, single replica │ (this Space) │ holds the ONLY dataset write token └───────────────────────────┘ │ forwards the player's token to 3 ZeroGPU Spaces (their quota, not ours) │ 1. identify object (VLM) ── cheap ──▶ normalize ──▶ DEDUP CHECK │ │ (only if new object) (if known: return ▼ existing, no commit) 2. design monster as JSON (LLM) ▼ 3. render art (T2I) ──▶ re-encode WebP ▼ acquire write lock ─▶ re-check dedup ▼ ONE commit: monster.json + art.webp + user.json + 4 index files ▼ return the monster to the browser Meanwhile ALL reads (dex, feed, leaderboard, a user's collection) go browser ─────────────▶ dataset CDN (/resolve/ URLs), never via this server. ``` The key move is that **stage 1 (identify) runs before the expensive stages**, so a repeat scan of an already-discovered object is recognised and returned for almost no GPU cost. Design and art only run for genuinely new objects. --- ## 2. Dataset layout (the database) Everything lives in one public dataset (`DATASET_REPO`). Files: | Path | What | | --------------------------- | ----------------------------------------------------------- | | `monsters/.json` | One canonical monster per normalized object name. | | `images/.webp` | That monster's art (re-encoded, ≤ 768px, quality 85). | | `users/.json` | A player's discoveries + summed rarity score. | | `index/monsters.json` | Array of monster summaries — the full dex. | | `index/feed.json` | Last 50 discoveries, newest first. | | `index/leaderboard.json` | Top 100 players by `total_rarity`. | | `index/stats.json` | Global totals. | `` = `normalize_object_name(descriptor)` — lowercased, articles dropped, punctuation stripped, lightly singularized, spaces → underscores. `"The Blue Pillows"` → `blue_pillow`. This is the dedup identity: same object → same key → same monster. **`monsters/.json`** ```json { "key": "coffee_mug", "descriptor": "ceramic coffee mug", "name": "Brewfin", "type": "cuisine", "appearance": "a round ceramic-bodied creature with a looping handle-tail, steam curling from its head, warm cream and cocoa colours", "description": "Brewfin dozes on warm surfaces and grumbles when its insides go cold.", "weight_kg": 1.2, "height_m": 0.25, "rarity": 34, "image_path": "images/coffee_mug.webp", "image_url": "https://huggingface.co/datasets//resolve/main/images/coffee_mug.webp", "discoverer": { "sub": "...", "username": "...", "name": "...", "picture": "..." }, "discovered_at": "2026-07-05T12:00:00+00:00" } ``` **`users/.json`** ```json { "sub": "6032...", "username": "fraser", "name": "Fraser", "picture": "https://...", "discoveries": ["coffee_mug", "eiffel_tower"], "total_rarity": 71, "discovery_count": 2, "joined_at": "2026-07-05T11:00:00+00:00", "last_seen": "2026-07-05T12:00:00+00:00" } ``` A player's **score is the sum of the rarity of the monsters they discovered.** The leaderboard entry is a denormalized `{sub, username, picture, total_rarity, discovery_count}` so the frontend can render it from one file. The four `index/*` files are **updated in the same commit as each new monster**, so a discovery is atomic: monster, art, the discoverer's record, and every aggregate view move together or not at all. --- ## 3. The limits that shape everything (verified) All figures below were checked against the official HF docs; URLs given so you can re-check (they drift). ### GPU: ZeroGPU daily quota, per account `https://huggingface.co/docs/hub/spaces-zerogpu` | Account | Daily GPU quota | Queue priority | | ------------- | --------------- | -------------- | | Anonymous | ~2 min | Low | | Free | ~5 min | Medium | | PRO | ~40 min (+ credits at $1 / 10 min) | Highest | - Quota is billed to **whichever token makes the call** — which is exactly why we forward the player's token. Their quota pays for their scans. - ZeroGPU is **Gradio-SDK only**, and **hosting your own** ZeroGPU Space requires **PRO** (max 10). Relevant only if you duplicate the model Spaces to pin versions. - A scan is 3 GPU calls; effective GPU time is on the order of ~30–60s (measure it). So a free player gets roughly a handful of *new* discoveries per day, anonymous 2–4, PRO many more. **This per-player daily ceiling is the headline constraint.** It's fine for a personal scanner plus a slowly-growing shared dex; it is not a high-volume-per-user design. ### Data: Hub rate limits, per 5-minute window `https://huggingface.co/docs/hub/rate-limits` | Bucket | Free | Anonymous (per IP) | PRO | | ------------------------------- | ------ | ------------------ | ------ | | **Resolvers** (`/resolve/` reads) | 5,000 | 3,000 | 12,000 | | **API** (incl. repo commits) | 1,000 | 500 | 2,500 | | **Pages** | 200 | 100 | 400 | - **Reads are `/resolve/` (resolver) URLs** — the highest limits, CDN-optimized, and counted **per client** (each browser's IP/token). So the frontend fetching monster JSON + images + index files directly means reads **never hit a central bottleneck** and never load this server. - **Writes are commits**, which HF rate-limits under "granular user action rate limits" — the **exact number is undocumented and changes**. Because this server commits with **one token**, that single account's commit budget is the true central write ceiling. - Mitigations, in order of importance: (1) always pass a token — the #1 cause of throttling; (2) use `huggingface_hub` ≥ 1.2.0, which parses the `RateLimit` header on a 429 and waits exactly the right time before retrying; (3) serialize commits (we do — single writer + lock); (4) commit only on new discoveries. ### Host: free CPU basic Space `https://huggingface.co/docs/hub/spaces-overview`, `.../spaces-gpus` - 2 vCPU, 16 GB RAM, 50 GB **non-persistent** disk. Free. - **Sleeps after 48h of inactivity** (fixed on free — you can't change the timer); any visitor wakes it, cold start up to ~a minute. Not billed while asleep. - **Single replica** — no horizontal scaling on free. This is a feature here: it guarantees one writer, so an in-process `threading.Lock` is sufficient to serialize commits with zero risk of git conflicts. No distributed locking, no external queue. --- ## 4. Design decisions (and the limit each one answers) - **Reads bypass the server entirely.** *(Resolver limits are per-client + CDN.)* The backend is orchestrate-and-write only; it exposes essentially one endpoint. - **Commit only on new monsters; repeat scans are read-only.** *(Undocumented commit ceiling on one token.)* Write rate tracks the rate of *new unique objects* discovered globally, not total scans — and that naturally decays as common objects get claimed. - **Identify the object before the expensive stages.** *(Tight per-player GPU quota.)* Dedup happens after the cheap VLM call, so repeat scans don't burn design/art GPU. - **Single in-process lock serializes writes.** *(Single replica.)* Memory is only mutated *after* the commit succeeds, so a failed commit never leaves the cache ahead of the dataset — no rollback logic needed. - **The player's photo is never stored.** It is only the input to the caption model; only the AI-generated art is persisted. Privacy and moderation win for free. - **Sign-in required to scan.** It gives us both a token to forward (for GPU) and an owner to attribute the discovery to. Anonymous users can still browse — reads are public. - **Monster spec is generated as JSON, not prose.** The concept model returns strict JSON (name, type, appearance, description, weight_kg, height_m, rarity); the server parses defensively and clamps every field. No brittle regex over markdown. - **`create_commit` with a list of operations = one commit per discovery.** (The older server used multiple `upload_file` calls = multiple commits per scan, which is worse for the write budget.) - **Framework = Gradio.** Reuses the proven forwarded-token pattern and the frontend already speaks `@gradio/client`. The framework barely matters because all heavy work is on external Spaces; the one thing that does matter — letting I/O-bound scans overlap — is handled with `queue(default_concurrency_limit=…)` and a per-event `concurrency_limit`. FastAPI would be a leaner alternative if you ever want explicit REST + async, but it buys little here. --- ## 5. The AI Spaces we call — API specs & how to swap them The AI layer in `app.py` is three isolated functions (`caption_object`, `generate_concept`, `generate_image`). Each takes the player's token, forwards it, and is the only code that knows a given Space's signature. **Swapping models = change the `*_SPACE` env vars and, if the signature differs, these three functions. Nothing else in the app depends on them.** The signatures below are the ones proven in the previous server. **Gradio Space APIs change, so verify before trusting** — for each Space: ```python from gradio_client import Client Client("/").view_api() # prints exact endpoint names + arg order ``` ### Stage 1 — caption / identify (`CAPTION_SPACE`, default `fancyfeast/joy-caption-alpha-two`) - Endpoint: `/stream_chat` - Positional args: `(image, caption_type, caption_length, extra_options, name_input, custom_prompt)` - we pass `(handle_file(path), "Descriptive", "short", [], "", )` - Returns: `(prompt_used, caption)` — we take index **1**, then trim to a short noun phrase used as the dedup key. ### Stage 2 — concept / design (`CONCEPT_SPACE`, default `amd/gpt-oss-120b-chatbot`) - Endpoint: `/chat` - Positional args: `(message, history, system_prompt, temperature)` - we pass `(, [], , 0.7)` - Returns: a response **string**; gpt-oss sometimes wraps it (`assistantfinal`, `**💬 Response:**`). `_extract_json()` strips that framing and any code fences, then parses the first `{...}` block. Every field is validated/clamped and `type` falls back to a keyword guess if the model returns something off-list. - The JSON prompt asks for exactly: `name`, `type` (one of the 10 categories), `appearance` (for the image model), `description`, `weight_kg`, `height_m`, `rarity` (1–100). See `CONCEPT_SYSTEM` / `_concept_prompt` in `app.py`. ### Stage 3 — image (`IMAGE_SPACE`, default `multimodalart/Qwen-Image-Fast`) - Endpoint: `/infer` - **This is the least-certain signature — confirm with `view_api()`.** Fast T2I Spaces (FLUX.1-schnell, Qwen-Image-Fast, …) all expose `/infer`, but the exact positional args (seed, steps, size, guidance, prompt-enhance) vary. We pass **only the prompt** and rely on the Space's defaults; if `view_api()` shows required positional args, add them in `generate_image`. - Output comes back as a local temp path, a URL, or a dict — `_read_image_result()` normalizes all three to bytes, then `_reencode_webp()` shrinks it. > **Stability vs cost.** Calling public third-party Spaces is free (the player's > quota) but you don't control their Gradio version or uptime — a signature can change > under you. Duplicating them into your own account lets you pin versions, but hosting > ZeroGPU requires PRO ($9/mo, ≤10 Spaces). That's the one recurring-cost decision and > it's optional; the `view_api()` check is your early-warning either way. --- ## 6. The one open item to verify Everything mechanical is proven **except**: that a forwarded **OAuth access token** (the kind the static frontend gets from "Sign in with Hugging Face") draws ZeroGPU quota exactly like a personal access token. The previous server proved token *forwarding* works when the token is a PAT/param; this just confirms OAuth tokens behave identically on the current Spaces. Test it in ~20 minutes: sign in on the frontend, grab the access token, and call one model Space with it via `gradio_client`. Confirm the call succeeds and the usage lands on that account's ZeroGPU quota (visible at `https://huggingface.co/settings/billing`). It's the last real unknown before trusting the end-to-end model. --- ## 7. Stress testing — what actually matters The GPU is **not this server's bottleneck** (external, per-user, and you can't stress it from one account without burning your own quota). So isolate the two real ceilings with `stress_test.py`: - **`commits`** — burst commits to a throwaway dataset to find the rate where HF starts throttling. Since one token serves *all* discoveries, that rate bounds global new-monster throughput. Watch for latencies climbing (the SDK sleeping off a 429). - **`reads`** — hammer a `/resolve/` URL; confirm it stays fast under concurrency and returns a CORS header the browser can use. This should never involve the server. - **`live`** *(optional, off by default)* — real end-to-end scans against the deployed Space to measure user-visible latency and concurrency. Spends GPU quota and creates real monsters, so keep `n` small. Also worth timing once: a **cold-start wake** after the Space has slept, so you know the worst-case first-scan latency after a quiet period. Interpreting it: you're looking for the commit rate at which throttling begins (your write ceiling) and confirmation that reads are effectively free and off-server. If the commit ceiling ever bites in practice, the mitigations are batching writes or moving the writer to a PRO/Team account with higher limits — but new-monster rate is self-limiting, so this is unlikely at hobby-to-moderate scale.