Spaces:
Sleeping
Sleeping
| # Piclets Discovery Server — Architecture | |
| This document explains *why* the backend is shaped the way it is. The short version: | |
| there is exactly one piece of shared state (a public dataset), exactly one trusted | |
| writer (this server), and a dumb client. The platform's limits push you toward that | |
| shape rather than fighting it. | |
| --- | |
| ## 1. The whole flow | |
| ``` | |
| player's browser (frontend Space) | |
| │ | |
| photo + player's HF access token (POST /scan) | |
| ▼ | |
| ┌───────────────────────────┐ | |
| │ Piclets Discovery Server │ free CPU Space, single replica | |
| │ (this Space) │ holds the ONLY dataset write token | |
| └───────────────────────────┘ | |
| │ | |
| forwards the player's token to 3 ZeroGPU Spaces (their quota, not ours) | |
| │ | |
| 1. identify object (VLM) ── cheap ──▶ normalize ──▶ DEDUP CHECK | |
| │ │ | |
| (only if new object) (if known: return | |
| ▼ existing, no commit) | |
| 2. design monster as JSON (LLM) | |
| ▼ | |
| 3. render art (T2I) ──▶ re-encode WebP | |
| ▼ | |
| acquire write lock ─▶ re-check dedup | |
| ▼ | |
| ONE commit: monster.json + art.webp + user.json + 4 index files | |
| ▼ | |
| return the monster to the browser | |
| Meanwhile ALL reads (dex, feed, leaderboard, a user's collection) go | |
| browser ─────────────▶ dataset CDN (/resolve/ URLs), never via this server. | |
| ``` | |
| The key move is that **stage 1 (identify) runs before the expensive stages**, so a | |
| repeat scan of an already-discovered object is recognised and returned for almost no | |
| GPU cost. Design and art only run for genuinely new objects. | |
| --- | |
| ## 2. Dataset layout (the database) | |
| Everything lives in one public dataset (`DATASET_REPO`). Files: | |
| | Path | What | | |
| | --------------------------- | ----------------------------------------------------------- | | |
| | `monsters/<key>.json` | One canonical monster per normalized object name. | | |
| | `images/<key>.webp` | That monster's art (re-encoded, ≤ 768px, quality 85). | | |
| | `users/<sub>.json` | A player's discoveries + summed rarity score. | | |
| | `index/monsters.json` | Array of monster summaries — the full dex. | | |
| | `index/feed.json` | Last 50 discoveries, newest first. | | |
| | `index/leaderboard.json` | Top 100 players by `total_rarity`. | | |
| | `index/stats.json` | Global totals. | | |
| `<key>` = `normalize_object_name(descriptor)` — lowercased, articles dropped, | |
| punctuation stripped, lightly singularized, spaces → underscores. `"The Blue | |
| Pillows"` → `blue_pillow`. This is the dedup identity: same object → same key → same | |
| monster. | |
| **`monsters/<key>.json`** | |
| ```json | |
| { | |
| "key": "coffee_mug", | |
| "descriptor": "ceramic coffee mug", | |
| "name": "Brewfin", | |
| "type": "cuisine", | |
| "appearance": "a round ceramic-bodied creature with a looping handle-tail, steam curling from its head, warm cream and cocoa colours", | |
| "description": "Brewfin dozes on warm surfaces and grumbles when its insides go cold.", | |
| "weight_kg": 1.2, | |
| "height_m": 0.25, | |
| "rarity": 34, | |
| "image_path": "images/coffee_mug.webp", | |
| "image_url": "https://huggingface.co/datasets/<repo>/resolve/main/images/coffee_mug.webp", | |
| "discoverer": { "sub": "...", "username": "...", "name": "...", "picture": "..." }, | |
| "discovered_at": "2026-07-05T12:00:00+00:00" | |
| } | |
| ``` | |
| **`users/<sub>.json`** | |
| ```json | |
| { | |
| "sub": "6032...", "username": "fraser", "name": "Fraser", "picture": "https://...", | |
| "discoveries": ["coffee_mug", "eiffel_tower"], | |
| "total_rarity": 71, "discovery_count": 2, | |
| "joined_at": "2026-07-05T11:00:00+00:00", "last_seen": "2026-07-05T12:00:00+00:00" | |
| } | |
| ``` | |
| A player's **score is the sum of the rarity of the monsters they discovered.** The | |
| leaderboard entry is a denormalized `{sub, username, picture, total_rarity, | |
| discovery_count}` so the frontend can render it from one file. | |
| The four `index/*` files are **updated in the same commit as each new monster**, so a | |
| discovery is atomic: monster, art, the discoverer's record, and every aggregate view | |
| move together or not at all. | |
| --- | |
| ## 3. The limits that shape everything (verified) | |
| All figures below were checked against the official HF docs; URLs given so you can | |
| re-check (they drift). | |
| ### GPU: ZeroGPU daily quota, per account | |
| `https://huggingface.co/docs/hub/spaces-zerogpu` | |
| | Account | Daily GPU quota | Queue priority | | |
| | ------------- | --------------- | -------------- | | |
| | Anonymous | ~2 min | Low | | |
| | Free | ~5 min | Medium | | |
| | PRO | ~40 min (+ credits at $1 / 10 min) | Highest | | |
| - Quota is billed to **whichever token makes the call** — which is exactly why we | |
| forward the player's token. Their quota pays for their scans. | |
| - ZeroGPU is **Gradio-SDK only**, and **hosting your own** ZeroGPU Space requires | |
| **PRO** (max 10). Relevant only if you duplicate the model Spaces to pin versions. | |
| - A scan is 3 GPU calls; effective GPU time is on the order of ~30–60s (measure it). | |
| So a free player gets roughly a handful of *new* discoveries per day, anonymous | |
| 2–4, PRO many more. **This per-player daily ceiling is the headline constraint.** | |
| It's fine for a personal scanner plus a slowly-growing shared dex; it is not a | |
| high-volume-per-user design. | |
| ### Data: Hub rate limits, per 5-minute window | |
| `https://huggingface.co/docs/hub/rate-limits` | |
| | Bucket | Free | Anonymous (per IP) | PRO | | |
| | ------------------------------- | ------ | ------------------ | ------ | | |
| | **Resolvers** (`/resolve/` reads) | 5,000 | 3,000 | 12,000 | | |
| | **API** (incl. repo commits) | 1,000 | 500 | 2,500 | | |
| | **Pages** | 200 | 100 | 400 | | |
| - **Reads are `/resolve/` (resolver) URLs** — the highest limits, CDN-optimized, and | |
| counted **per client** (each browser's IP/token). So the frontend fetching monster | |
| JSON + images + index files directly means reads **never hit a central bottleneck** | |
| and never load this server. | |
| - **Writes are commits**, which HF rate-limits under "granular user action rate | |
| limits" — the **exact number is undocumented and changes**. Because this server | |
| commits with **one token**, that single account's commit budget is the true central | |
| write ceiling. | |
| - Mitigations, in order of importance: (1) always pass a token — the #1 cause of | |
| throttling; (2) use `huggingface_hub` ≥ 1.2.0, which parses the `RateLimit` header | |
| on a 429 and waits exactly the right time before retrying; (3) serialize commits | |
| (we do — single writer + lock); (4) commit only on new discoveries. | |
| ### Host: free CPU basic Space | |
| `https://huggingface.co/docs/hub/spaces-overview`, `.../spaces-gpus` | |
| - 2 vCPU, 16 GB RAM, 50 GB **non-persistent** disk. Free. | |
| - **Sleeps after 48h of inactivity** (fixed on free — you can't change the timer); | |
| any visitor wakes it, cold start up to ~a minute. Not billed while asleep. | |
| - **Single replica** — no horizontal scaling on free. This is a feature here: it | |
| guarantees one writer, so an in-process `threading.Lock` is sufficient to serialize | |
| commits with zero risk of git conflicts. No distributed locking, no external queue. | |
| --- | |
| ## 4. Design decisions (and the limit each one answers) | |
| - **Reads bypass the server entirely.** *(Resolver limits are per-client + CDN.)* The | |
| backend is orchestrate-and-write only; it exposes essentially one endpoint. | |
| - **Commit only on new monsters; repeat scans are read-only.** *(Undocumented commit | |
| ceiling on one token.)* Write rate tracks the rate of *new unique objects* | |
| discovered globally, not total scans — and that naturally decays as common objects | |
| get claimed. | |
| - **Identify the object before the expensive stages.** *(Tight per-player GPU quota.)* | |
| Dedup happens after the cheap VLM call, so repeat scans don't burn design/art GPU. | |
| - **Single in-process lock serializes writes.** *(Single replica.)* Memory is only | |
| mutated *after* the commit succeeds, so a failed commit never leaves the cache ahead | |
| of the dataset — no rollback logic needed. | |
| - **The player's photo is never stored.** It is only the input to the caption model; | |
| only the AI-generated art is persisted. Privacy and moderation win for free. | |
| - **Sign-in required to scan.** It gives us both a token to forward (for GPU) and an | |
| owner to attribute the discovery to. Anonymous users can still browse — reads are | |
| public. | |
| - **Monster spec is generated as JSON, not prose.** The concept model returns strict | |
| JSON (name, type, appearance, description, weight_kg, height_m, rarity); the server | |
| parses defensively and clamps every field. No brittle regex over markdown. | |
| - **`create_commit` with a list of operations = one commit per discovery.** (The older | |
| server used multiple `upload_file` calls = multiple commits per scan, which is worse | |
| for the write budget.) | |
| - **Framework = Gradio.** Reuses the proven forwarded-token pattern and the frontend | |
| already speaks `@gradio/client`. The framework barely matters because all heavy work | |
| is on external Spaces; the one thing that does matter — letting I/O-bound scans | |
| overlap — is handled with `queue(default_concurrency_limit=…)` and a per-event | |
| `concurrency_limit`. FastAPI would be a leaner alternative if you ever want explicit | |
| REST + async, but it buys little here. | |
| --- | |
| ## 5. The AI Spaces we call — API specs & how to swap them | |
| The AI layer in `app.py` is three isolated functions (`caption_object`, | |
| `generate_concept`, `generate_image`). Each takes the player's token, forwards it, and | |
| is the only code that knows a given Space's signature. **Swapping models = change the | |
| `*_SPACE` env vars and, if the signature differs, these three functions. Nothing else | |
| in the app depends on them.** | |
| The signatures below are the ones proven in the previous server. **Gradio Space APIs | |
| change, so verify before trusting** — for each Space: | |
| ```python | |
| from gradio_client import Client | |
| Client("<owner>/<space>").view_api() # prints exact endpoint names + arg order | |
| ``` | |
| ### Stage 1 — caption / identify (`CAPTION_SPACE`, default `fancyfeast/joy-caption-alpha-two`) | |
| - Endpoint: `/stream_chat` | |
| - Positional args: `(image, caption_type, caption_length, extra_options, name_input, custom_prompt)` | |
| - we pass `(handle_file(path), "Descriptive", "short", [], "", <identify instruction>)` | |
| - Returns: `(prompt_used, caption)` — we take index **1**, then trim to a short noun | |
| phrase used as the dedup key. | |
| ### Stage 2 — concept / design (`CONCEPT_SPACE`, default `amd/gpt-oss-120b-chatbot`) | |
| - Endpoint: `/chat` | |
| - Positional args: `(message, history, system_prompt, temperature)` | |
| - we pass `(<JSON prompt>, [], <system>, 0.7)` | |
| - Returns: a response **string**; gpt-oss sometimes wraps it (`assistantfinal`, | |
| `**💬 Response:**`). `_extract_json()` strips that framing and any code fences, then | |
| parses the first `{...}` block. Every field is validated/clamped and `type` falls | |
| back to a keyword guess if the model returns something off-list. | |
| - The JSON prompt asks for exactly: `name`, `type` (one of the 10 categories), | |
| `appearance` (for the image model), `description`, `weight_kg`, `height_m`, | |
| `rarity` (1–100). See `CONCEPT_SYSTEM` / `_concept_prompt` in `app.py`. | |
| ### Stage 3 — image (`IMAGE_SPACE`, default `multimodalart/Qwen-Image-Fast`) | |
| - Endpoint: `/infer` | |
| - **This is the least-certain signature — confirm with `view_api()`.** Fast T2I | |
| Spaces (FLUX.1-schnell, Qwen-Image-Fast, …) all expose `/infer`, but the exact | |
| positional args (seed, steps, size, guidance, prompt-enhance) vary. We pass **only | |
| the prompt** and rely on the Space's defaults; if `view_api()` shows required | |
| positional args, add them in `generate_image`. | |
| - Output comes back as a local temp path, a URL, or a dict — `_read_image_result()` | |
| normalizes all three to bytes, then `_reencode_webp()` shrinks it. | |
| > **Stability vs cost.** Calling public third-party Spaces is free (the player's | |
| > quota) but you don't control their Gradio version or uptime — a signature can change | |
| > under you. Duplicating them into your own account lets you pin versions, but hosting | |
| > ZeroGPU requires PRO ($9/mo, ≤10 Spaces). That's the one recurring-cost decision and | |
| > it's optional; the `view_api()` check is your early-warning either way. | |
| --- | |
| ## 6. The one open item to verify | |
| Everything mechanical is proven **except**: that a forwarded **OAuth access token** | |
| (the kind the static frontend gets from "Sign in with Hugging Face") draws ZeroGPU | |
| quota exactly like a personal access token. The previous server proved token | |
| *forwarding* works when the token is a PAT/param; this just confirms OAuth tokens | |
| behave identically on the current Spaces. | |
| Test it in ~20 minutes: sign in on the frontend, grab the access token, and call one | |
| model Space with it via `gradio_client`. Confirm the call succeeds and the usage lands | |
| on that account's ZeroGPU quota (visible at `https://huggingface.co/settings/billing`). | |
| It's the last real unknown before trusting the end-to-end model. | |
| --- | |
| ## 7. Stress testing — what actually matters | |
| The GPU is **not this server's bottleneck** (external, per-user, and you can't stress | |
| it from one account without burning your own quota). So isolate the two real ceilings | |
| with `stress_test.py`: | |
| - **`commits`** — burst commits to a throwaway dataset to find the rate where HF | |
| starts throttling. Since one token serves *all* discoveries, that rate bounds global | |
| new-monster throughput. Watch for latencies climbing (the SDK sleeping off a 429). | |
| - **`reads`** — hammer a `/resolve/` URL; confirm it stays fast under concurrency and | |
| returns a CORS header the browser can use. This should never involve the server. | |
| - **`live`** *(optional, off by default)* — real end-to-end scans against the deployed | |
| Space to measure user-visible latency and concurrency. Spends GPU quota and creates | |
| real monsters, so keep `n` small. | |
| Also worth timing once: a **cold-start wake** after the Space has slept, so you know | |
| the worst-case first-scan latency after a quiet period. | |
| Interpreting it: you're looking for the commit rate at which throttling begins (your | |
| write ceiling) and confirmation that reads are effectively free and off-server. If the | |
| commit ceiling ever bites in practice, the mitigations are batching writes or moving | |
| the writer to a PRO/Team account with higher limits — but new-monster rate is | |
| self-limiting, so this is unlikely at hobby-to-moderate scale. | |