docs/ARCHITECTURE.md

Architecture and growth path

Request boundaries

The FastAPI parent owns authentication, CSRF/origin checks, private uploads, quotas, provenance, publication and public catalogue projection. Gradio is mounted only when inference is enabled, and accepts an opaque, server-reserved generation UUID as its only input. It never accepts a client path, URL, author, result file or metadata row. Its file/upload/proxy/stream endpoints are blocked. The function checks the cookie-derived owner again after queueing. A conditional journal update claims a generation exactly once.

Real HF OIDC requests only openid profile; Authlib handles state, PKCE, nonce and ID-token verification, followed by userinfo subject comparison. Session cookies contain a random opaque value; the database stores its hash. OAuth access tokens are encrypted with a separate Fernet key and are not sent to the browser. Every mutating request has a session-bound CSRF token and exact configured Origin. Authenticated mutations revalidate HF userinfo; publication revalidates again after expensive preparation and before each commit attempt. Logout only needs the current cookie and CSRF so an expired HF token does not prevent logout.

Anonymous creation is allowed, but anonymous contribution is refused by the API. OAuth rotates the session cookie and atomically binds anonymous image/generation ownership to the verified HF subject. Cross-account access uses neither username parameters nor browser claims. A second account cannot access another user's private files. Private paths are derived solely from server-issued UUIDs.

State and publication

SQLite runs in WAL mode with BEGIN IMMEDIATE, full synchronization, unique generation IDs, request keys and content hashes. This is a single-instance pilot with a durable private volume, not an eventually consistent distributed database. Feature flags fail closed if real OAuth, dedicated writer configuration or durable state acknowledgement is missing.

Publication has a durable reservation before networking. flock serializes publishers and withdrawal/cleanup-sensitive files within one host; the Hub commit's parent_commit provides compare-and-swap against other repository writers. Each retry re-reads the remote head. A lost successful response is reconciled by the reserved public asset UUID and verified SHA/author. Content hashes prevent byte-identical duplicates across accounts. Near-duplicate geometry is not detected.

Files are copied into private staging, revalidated and hashed. One atomic Hub commit contains the original PLY, both actual-render previews, canonical metadata, canonical JSONL export, manifest and runtime-package inventory. No public row is written separately. Pre-uploaded unreferenced Hub blobs are not treated as published objects. Source images are never in the commit operations.

The dataset writer is distinct from model read access. The repository adapter introspects its fine-grained scope and refuses broad/unknown descriptors; the exact introspection response still needs validation with the operator's actual restricted credential. There is no fallback to a broad token. Repository IDs and file paths cannot be supplied by contributors.

Rendering and validation

Fixed-layout PLY only: 17 little-endian float32 fields, SH0 coefficients, logit opacity, logarithmic scales, WXYZ quaternion, Y-up. Limits: 80 MiB; 32,768–1,048,576 Gaussians in multiples of 32; 4 KiB header; exact payload length; no faces, lists, extra properties, comments, external references, trailing bytes or non-finite numbers. Resource-profile bounds also restrict coordinates, normals, scales, opacity and quaternions. The actual pinned upstream 262,144-splat backpack passes the validator with its original SHA unchanged. These numeric thresholds still need a broader generated-object corpus.

The validator has a 10-second CPU limit, address-space limit and a 15-second wall deadline. Rendering runs in a separate process group, allows only local validated input plus the built renderer, strips authentication/model/writer secrets from its environment, blocks external browser requests, limits dimensions to 1024², RSS to 3 GiB and wall time to 120 seconds, and kills the process group on completion/failure. Linux process accounting is required; absence fails closed. It is a resource boundary, not a substitute for a hardened deployment container.

Spark 2.4.0 extended splats and Three 0.180.0 share one scene/camera module between preview and interactive rendering. The original PLY is never converted, resaved or replaced by a GLB. Spark's internal representation is not lossless for every numeric field, so fidelity remains an acceptance gate. There is no claim that browser rendering is pixel-identical to IDOL's original renderer.

The preview is transparent RGBA, alpha bounds define a padded square crop, and the same camera/source plane/crop is reused by Scenario's spatial inspector. The main Create stage uses a centred square camera plane. Input photos are only labelled inputs. A failed preview leaves the validated PLY downloadable; retrying a preview does not rerun inference. Publication remains impossible until both previews exist and validate.

Refresh without changing navigation

The public API reads a revision-pinned collection snapshot every 15 seconds and supports ETag/304. It never serves a partially prepared local record. All exposed source URLs resolve an immutable commit. The client merges by stable IDs and reuses unchanged object references/URLs. Sequence slots are monotonically allocated; withdrawal leaves a gap. Search has a separate stable slot map per query. Refresh does not set zoom, view, query or selection; removal only clears selection of the removed object. Publishing returns a direct object link and an origin-checked message explicitly centres that object when requested.

The viewer's CSS, D3 movement/zoom, image cutouts, inspector geometry, basket and download mechanisms are retained. Create and Explore remain mounted; hiding Explore sends an explicit visibility signal to pause its active renderer. Catalogue refresh can continue while hidden without moving the user's camera.

Capacity and the large-collection goal

This first release is a controlled pilot, capped at 10,000 public records / 32 MiB manifest. Current defaults: 5 publication reservations/account/day; 20 generations/account/day (3 anonymous); 10 image uploads/session-owner/day; 200 uploads and 100 generations globally per rolling 24 hours. Private files expire after 24 hours. These are configuration defaults, not claims of throughput.

Before 10,000 objects, replace full-manifest rewrites and one-commit-per-object growth with:

1. PostgreSQL transactions/outbox and a durable work queue, immutable object storage, isolated validator/render workers, shared account quotas and publication leases.
2. Versioned immutable JSONL/Parquet shards and paginated/delta catalogue snapshots, with an atomically replaced small root manifest. Preserve record schema, stable IDs and append-only spatial sequence slots.
3. Search indexes and region/viewport-aware catalogue queries, while keeping pinned download manifests and resumable bulk downloads.
4. Internally generated storage packs when needed for Hub file/directory limits. Packs are a storage implementation, never an untrusted archive-upload path. The original asset remains independently addressable through a verified extraction/download service.
5. A moderation queue, abuse signals and duplicate-geometry heuristics, separated from binary validation and clear about their confidence. Define maintainer coverage and response targets before advertising them.

No claim of “largest collection” is made by the current build. That is the product direction; reliable provenance, usable downloads and scalable catalogue contracts are the foundations.