ARCHITECTURE.md

Piclets Discovery Server — Architecture

This document explains why the backend is shaped the way it is. The short version: there is exactly one piece of shared state (a public dataset), exactly one trusted writer (this server), and a dumb client. The platform's limits push you toward that shape rather than fighting it.

---

1. The whole flow

                        player's browser (frontend Space)
                                     │
                 photo + player's HF access token (POST /scan)
                                     ▼
                        ┌───────────────────────────┐
                        │   Piclets Discovery Server  │   free CPU Space, single replica
                        │        (this Space)         │   holds the ONLY dataset write token
                        └───────────────────────────┘
                                     │
         forwards the player's token to 3 ZeroGPU Spaces (their quota, not ours)
                                     │
        1. identify object (VLM)  ── cheap ──▶  normalize ──▶ DEDUP CHECK
                                     │                              │
                          (only if new object)                (if known: return
                                     ▼                          existing, no commit)
        2. design monster as JSON (LLM)
                                     ▼
        3. render art (T2I) ──▶ re-encode WebP
                                     ▼
                     acquire write lock ─▶ re-check dedup
                                     ▼
                ONE commit: monster.json + art.webp + user.json + 4 index files
                                     ▼
                     return the monster to the browser

        Meanwhile ALL reads (dex, feed, leaderboard, a user's collection) go
        browser ─────────────▶ dataset CDN (/resolve/ URLs), never via this server.

The key move is that stage 1 (identify) runs before the expensive stages, so a repeat scan of an already-discovered object is recognised and returned for almost no GPU cost. Design and art only run for genuinely new objects.

---

2. Dataset layout (the database)

Everything lives in one public dataset (DATASET_REPO). Files:

Path	What
monsters/<key>.json	One canonical monster per normalized object name.
images/<key>.webp	That monster's art (re-encoded, ≤ 768px, quality 85).
users/<sub>.json	A player's discoveries + summed rarity score.
index/monsters.json	Array of monster summaries — the full dex.
index/feed.json	Last 50 discoveries, newest first.
index/leaderboard.json	Top 100 players by total_rarity.
index/stats.json	Global totals.

<key> = normalize_object_name(descriptor) — lowercased, articles dropped, punctuation stripped, lightly singularized, spaces → underscores. "The Blue Pillows" → blue_pillow. This is the dedup identity: same object → same key → same monster.

monsters/<key>.json

{
  "key": "coffee_mug",
  "descriptor": "ceramic coffee mug",
  "name": "Brewfin",
  "type": "cuisine",
  "appearance": "a round ceramic-bodied creature with a looping handle-tail, steam curling from its head, warm cream and cocoa colours",
  "description": "Brewfin dozes on warm surfaces and grumbles when its insides go cold.",
  "weight_kg": 1.2,
  "height_m": 0.25,
  "rarity": 34,
  "image_path": "images/coffee_mug.webp",
  "image_url": "https://huggingface.co/datasets/<repo>/resolve/main/images/coffee_mug.webp",
  "discoverer": { "sub": "...", "username": "...", "name": "...", "picture": "..." },
  "discovered_at": "2026-07-05T12:00:00+00:00"
}

users/<sub>.json

{
  "sub": "6032...", "username": "fraser", "name": "Fraser", "picture": "https://...",
  "discoveries": ["coffee_mug", "eiffel_tower"],
  "total_rarity": 71, "discovery_count": 2,
  "joined_at": "2026-07-05T11:00:00+00:00", "last_seen": "2026-07-05T12:00:00+00:00"
}

A player's score is the sum of the rarity of the monsters they discovered. The leaderboard entry is a denormalized {sub, username, picture, total_rarity, discovery_count} so the frontend can render it from one file.

The four index/* files are updated in the same commit as each new monster, so a discovery is atomic: monster, art, the discoverer's record, and every aggregate view move together or not at all.

---

3. The limits that shape everything (verified)

All figures below were checked against the official HF docs; URLs given so you can re-check (they drift).

GPU: ZeroGPU daily quota, per account

https://huggingface.co/docs/hub/spaces-zerogpu

Account	Daily GPU quota	Queue priority
Anonymous	~2 min	Low
Free	~5 min	Medium
PRO	~40 min (+ credits at $1 / 10 min)	Highest

• Quota is billed to whichever token makes the call — which is exactly why we forward the player's token. Their quota pays for their scans.
• ZeroGPU is Gradio-SDK only, and hosting your own ZeroGPU Space requires PRO (max 10). Relevant only if you duplicate the model Spaces to pin versions.
• A scan is 3 GPU calls; effective GPU time is on the order of ~30–60s (measure it). So a free player gets roughly a handful of new discoveries per day, anonymous 2–4, PRO many more. This per-player daily ceiling is the headline constraint. It's fine for a personal scanner plus a slowly-growing shared dex; it is not a high-volume-per-user design.

Data: Hub rate limits, per 5-minute window

https://huggingface.co/docs/hub/rate-limits

Bucket	Free	Anonymous (per IP)	PRO
Resolvers (/resolve/ reads)	5,000	3,000	12,000
API (incl. repo commits)	1,000	500	2,500
Pages	200	100	400

• Reads are /resolve/ (resolver) URLs — the highest limits, CDN-optimized, and counted per client (each browser's IP/token). So the frontend fetching monster JSON + images + index files directly means reads never hit a central bottleneck and never load this server.
• Writes are commits, which HF rate-limits under "granular user action rate limits" — the exact number is undocumented and changes. Because this server commits with one token, that single account's commit budget is the true central write ceiling.
• Mitigations, in order of importance: (1) always pass a token — the #1 cause of throttling; (2) use huggingface_hub ≥ 1.2.0, which parses the RateLimit header on a 429 and waits exactly the right time before retrying; (3) serialize commits (we do — single writer + lock); (4) commit only on new discoveries.

Host: free CPU basic Space

https://huggingface.co/docs/hub/spaces-overview, .../spaces-gpus

• 2 vCPU, 16 GB RAM, 50 GB non-persistent disk. Free.
• Sleeps after 48h of inactivity (fixed on free — you can't change the timer); any visitor wakes it, cold start up to ~a minute. Not billed while asleep.
• Single replica — no horizontal scaling on free. This is a feature here: it guarantees one writer, so an in-process threading.Lock is sufficient to serialize commits with zero risk of git conflicts. No distributed locking, no external queue.

---

4. Design decisions (and the limit each one answers)

• Reads bypass the server entirely. (Resolver limits are per-client + CDN.) The backend is orchestrate-and-write only; it exposes essentially one endpoint.
• Commit only on new monsters; repeat scans are read-only. (Undocumented commit ceiling on one token.) Write rate tracks the rate of new unique objects discovered globally, not total scans — and that naturally decays as common objects get claimed.
• Identify the object before the expensive stages. (Tight per-player GPU quota.) Dedup happens after the cheap VLM call, so repeat scans don't burn design/art GPU.
• Single in-process lock serializes writes. (Single replica.) Memory is only mutated after the commit succeeds, so a failed commit never leaves the cache ahead of the dataset — no rollback logic needed.
• The player's photo is never stored. It is only the input to the caption model; only the AI-generated art is persisted. Privacy and moderation win for free.
• Sign-in required to scan. It gives us both a token to forward (for GPU) and an owner to attribute the discovery to. Anonymous users can still browse — reads are public.
• Monster spec is generated as JSON, not prose. The concept model returns strict JSON (name, type, appearance, description, weight_kg, height_m, rarity); the server parses defensively and clamps every field. No brittle regex over markdown.
• create_commit with a list of operations = one commit per discovery. (The older server used multiple upload_file calls = multiple commits per scan, which is worse for the write budget.)
• Framework = Gradio. Reuses the proven forwarded-token pattern and the frontend already speaks @gradio/client. The framework barely matters because all heavy work is on external Spaces; the one thing that does matter — letting I/O-bound scans overlap — is handled with queue(default_concurrency_limit=…) and a per-event concurrency_limit. FastAPI would be a leaner alternative if you ever want explicit REST + async, but it buys little here.

---

5. The AI Spaces we call — API specs & how to swap them

The AI layer in app.py is three isolated functions (caption_object, generate_concept, generate_image). Each takes the player's token, forwards it, and is the only code that knows a given Space's signature. Swapping models = change the *_SPACE env vars and, if the signature differs, these three functions. Nothing else in the app depends on them.

The signatures below are the ones proven in the previous server. Gradio Space APIs change, so verify before trusting — for each Space:

from gradio_client import Client
Client("<owner>/<space>").view_api()      # prints exact endpoint names + arg order

Stage 1 — caption / identify (CAPTION_SPACE, default fancyfeast/joy-caption-alpha-two)

• Endpoint: /stream_chat
• Positional args: (image, caption_type, caption_length, extra_options, name_input, custom_prompt)
  • we pass (handle_file(path), "Descriptive", "short", [], "", <identify instruction>)
• Returns: (prompt_used, caption) — we take index 1, then trim to a short noun phrase used as the dedup key.

Stage 2 — concept / design (CONCEPT_SPACE, default amd/gpt-oss-120b-chatbot)

• Endpoint: /chat
• Positional args: (message, history, system_prompt, temperature)
  • we pass (<JSON prompt>, [], <system>, 0.7)
• Returns: a response string; gpt-oss sometimes wraps it (assistantfinal, **💬 Response:**). _extract_json() strips that framing and any code fences, then parses the first {...} block. Every field is validated/clamped and type falls back to a keyword guess if the model returns something off-list.
• The JSON prompt asks for exactly: name, type (one of the 10 categories), appearance (for the image model), description, weight_kg, height_m, rarity (1–100). See CONCEPT_SYSTEM / _concept_prompt in app.py.

Stage 3 — image (IMAGE_SPACE, default multimodalart/Qwen-Image-Fast)

• Endpoint: /infer
• This is the least-certain signature — confirm with view_api(). Fast T2I Spaces (FLUX.1-schnell, Qwen-Image-Fast, …) all expose /infer, but the exact positional args (seed, steps, size, guidance, prompt-enhance) vary. We pass only the prompt and rely on the Space's defaults; if view_api() shows required positional args, add them in generate_image.
• Output comes back as a local temp path, a URL, or a dict — _read_image_result() normalizes all three to bytes, then _reencode_webp() shrinks it.

Stability vs cost. Calling public third-party Spaces is free (the player's quota) but you don't control their Gradio version or uptime — a signature can change under you. Duplicating them into your own account lets you pin versions, but hosting ZeroGPU requires PRO ($9/mo, ≤10 Spaces). That's the one recurring-cost decision and it's optional; the view_api() check is your early-warning either way.

---

6. The one open item to verify

Everything mechanical is proven except: that a forwarded OAuth access token (the kind the static frontend gets from "Sign in with Hugging Face") draws ZeroGPU quota exactly like a personal access token. The previous server proved token forwarding works when the token is a PAT/param; this just confirms OAuth tokens behave identically on the current Spaces.

Test it in ~20 minutes: sign in on the frontend, grab the access token, and call one model Space with it via gradio_client. Confirm the call succeeds and the usage lands on that account's ZeroGPU quota (visible at https://huggingface.co/settings/billing). It's the last real unknown before trusting the end-to-end model.

---

7. Stress testing — what actually matters

The GPU is not this server's bottleneck (external, per-user, and you can't stress it from one account without burning your own quota). So isolate the two real ceilings with stress_test.py:

• commits — burst commits to a throwaway dataset to find the rate where HF starts throttling. Since one token serves all discoveries, that rate bounds global new-monster throughput. Watch for latencies climbing (the SDK sleeping off a 429).
• reads — hammer a /resolve/ URL; confirm it stays fast under concurrency and returns a CORS header the browser can use. This should never involve the server.
• live (optional, off by default) — real end-to-end scans against the deployed Space to measure user-visible latency and concurrency. Spends GPU quota and creates real monsters, so keep n small.

Also worth timing once: a cold-start wake after the Space has slept, so you know the worst-case first-scan latency after a quiet period.

Interpreting it: you're looking for the commit rate at which throttling begins (your write ceiling) and confirmation that reads are effectively free and off-server. If the commit ceiling ever bites in practice, the mitigations are batching writes or moving the writer to a PRO/Team account with higher limits — but new-monster rate is self-limiting, so this is unlikely at hobby-to-moderate scale.