deploy/hf/README.md

Hugging Face Spaces

Where the DB lives

• Do not commit store_slim.db to the Space git repo.
• Attach a Storage Bucket (or dataset volume) and mount it at e.g. /data.
• Set KINK_STORE_PATH=/data/store_slim.db so the SQLite file lives on the volume.

Why user accounts survive deploys

SQLite stores users in the same file as the catalog. If that file is on persistent storage, new users and plays are written into that file and remain after image updates or container restarts. If the DB only existed on ephemeral disk, it would reset when the Space restarts.

Free alternative: snapshot user-state to a private Hub dataset

If you do not want to pay for HF Persistent Storage, set these Space variables instead:

• KINK_USER_SNAPSHOT_REPO=<owner>/<dataset> — a private dataset repo for the snapshot.
• HF_TOKEN=<write-token> — same secret used by the catalog/Space deploy.
• Optional: KINK_USER_SNAPSHOT_FILENAME (default user_state.db), KINK_USER_SNAPSHOT_INTERVAL_S (default 60).

Behaviour:

• On cold boot, after the catalog opens, _restore_user_snapshot_on_boot pulls the latest snapshot from the dataset and upserts it into the live SQLite — but only when the local user tables are empty. Failures are logged and ignored so a Hub outage cannot block boot.
• A background task in lifespan recomputes the user-state fingerprint every KINK_USER_SNAPSHOT_INTERVAL_S seconds; when it changes, it dumps USER_TABLES to a slim SQLite and uploads via huggingface_hub.upload_file. A final push runs on lifespan shutdown.
• Tables shipped: user, userpreference, playpreference, rolepreference, scenariopreference, promptdismissal, partnerlink, partnerlinkrequest, partnergroup, partnergroupmember. Catalog tables (kink, play_fts*, similarityedge, fetlife*, kinkscenarioparent) stay on the ephemeral image.
• Trade-off: up to ~`interval` seconds of writes can be lost on a hard crash. Multi-replica Spaces would race on push — keep cpu-basic (single replica) or add an external mutex before scaling.

First boot (empty volume)

Pick one way to supply the full store_slim.db (multi‑GB):

1. KINK_CATALOG_URL — HTTPS URL to the file on any host you control (see Free hosting below). Optional: KINK_CATALOG_URL_BEARER for Authorization: Bearer … on private HTTP URLs.

2. Backblaze B2 (private OK) — same Space repository secrets as parser scripts/hf_deploy.py: B2_KEY_ID, B2_APPLICATION_KEY, B2_BUCKET, B2_REGION. Set KINK_B2_OBJECT_KEY to the uploaded object path (default kink/catalog/store_slim.db). The app uses boto3 against the B2 S3 endpoint on first boot; unset KINK_CATALOG_URL when using this path.

3. KINK_HF_DATASET_REPO (+ optional KINK_HF_DATASET_FILENAME, KINK_HF_DATASET_REVISION) — download from a Hub dataset (see scripts/upload_catalog_to_hub.py).

deploy/hf/entrypoint.sh only copies the tiny bundled seed when full-catalog mode is off and none of URL, B2 (B2_BUCKET / B2_KEY_ID), or Hub dataset env is configured; otherwise the store path stays empty until ensure_store_db() runs.

After first boot, the file on the bucket (or /tmp) is the source of truth; remove URL/dataset env vars if you do not want re-download attempts (an existing file takes precedence).

Free hosting (looked up for a ~3 GB store_slim.db)

Not a substitute for a file host: Pinecone (Weaviate, Qdrant, etc.) are vector databases for similarity search over embeddings. They do not store your SQLite file as-is. Putting the catalog there would mean rebuilding the product around vectors + APIs, not flipping one env var.

Options that fit “free” and avoid adding financial info (no card on signup):

Option	Why it fits	Caveat
Hugging Face Hub (public dataset + LFS)	A normal free Hub account hosts large LFS objects; no separate object‑store vendor; already wired via KINK_HF_DATASET_REPO + upload_catalog_to_hub.py.	Hub ToS / fair use; cold‑start download time.
Backblaze B2	Same as above for free tier fit. Use scripts/upload_catalog_to_b2.py with [b2] from parser secrets.toml; optional env KINK_B2_CATALOG_BUCKET / --bucket targets a dedicated bucket if another bucket hit a storage cap. After upload: scripts/verify_b2_catalog_http.py. Public catalog: set KINK_CATALOG_URL to the printed native file URL. Private catalog: add B2_* secrets on the Space + KINK_B2_OBJECT_KEY (no public URL needed).	Anonymous GET returns 401 on private buckets—that is expected; the Space uses authenticated S3 download. If upload fails with storage cap exceeded, raise the cap or use a bucket with headroom.

Cloudflare R2: generous documented free tier, but Cloudflare often still asks for payment details on the account even when usage stays at $0—skip it if your requirement is no financial info at all.

Oracle Cloud “Always Free” and similar clouds: sometimes useful, but many require a card for account creation even when the tier is free—check their current signup flow before counting on it.

Usually a poor fit at ~3 GB: GitHub/Git LFS repo limits, Supabase Storage free caps, ad‑hoc “free upload” sites (size caps, fragile hotlinks).

For the Space, set KINK_CATALOG_URL (public HTTPS), or B2_* + KINK_B2_OBJECT_KEY (private B2), or Hub without a third‑party object store.

Environment

Variable	Role
KINK_STORE_PATH	Full path to SQLite on the mounted volume (required on HF)
KINK_HF_REQUIRE_FULL_CATALOG	0 in the Docker image (bundled seed). Set 1 on the Space when you supply a full catalog via URL, B2, or Hub (no tiny bundled fallback).
KINK_CATALOG_URL	If set, wins over B2 and Hub: stream-download the DB from this HTTPS URL
KINK_CATALOG_URL_BEARER	Optional bearer token for private HTTP object URLs
B2_KEY_ID, B2_APPLICATION_KEY, B2_BUCKET, B2_REGION	Backblaze S3 credentials (Space secrets; same names as parser). Used when KINK_CATALOG_URL is unset.
KINK_B2_OBJECT_KEY	Object key inside the bucket (default kink/catalog/store_slim.db)
KINK_HF_DATASET_REPO	Hub dataset id (used when KINK_HF_REQUIRE_FULL_CATALOG=1, or with KINK_HF_USE_HUB_DATASET=1 while full catalog is off)
KINK_HF_DATASET_FILENAME	Remote filename in the dataset (e.g. store_slim.db)
KINK_HF_USE_HUB_DATASET	Set 1 only if you want a Hub download while KINK_HF_REQUIRE_FULL_CATALOG=0. Otherwise a leftover KINK_HF_DATASET_REPO Space variable is ignored so the bundled seed is used (avoids anonymous Hub traffic and OOMs).
HF_TOKEN	If the Hub dataset is private
KINK_SKIP_HEAVY_WARM	1 in the published Docker image skips only PPR / similarity-graph warm-up (saves RAM on cpu-basic). The in-memory catalog is still built at process start so the first recommendations request is not blocked. Set to 0 on larger hardware for full graph warm-up (better cold recs for heavy graph users, higher memory).
KINK_CORS_ORIGINS	Your Space URL for browser API calls
KINK_RECSYS_SOURCE	Optional: graph (default), ots, or hybrid when not using Backend(..., recsys_settings=...). Overrides recsys/settings.json if present.
KINK_RECSYS_CANDIDATES_PATH	Optional: absolute path to OTS candidates.json (from scripts/recsys_benchmark.py).

OTS / Cornac: beside the SQLite file, you can add recsys/settings.json and recsys/candidates.json on the same volume (copy deploy/recsys/settings.example.json as a starting point). See repo docs/recsys_ots.md. GET /health reports recsys (source, paths, whether the cache file exists).

Build with Dockerfile in the repo root or point the Space at this project.

Release checklist

1. Tests: from repo root, python -m pytest -q (or make test). Partner HTTP probes are also covered in CI via tests/test_adversarial_partner_probes_script.py; locally: make adv-probes-http.
2. Catalog / B2 / Hub bootstrap (only when the SQLite artifact or Space secrets change):
   • B2 upload + Space secrets: scripts/sync_hf_space_b2_catalog.py
   • Smaller Hub dataset bootstrap for the Space: scripts/deploy_hf_bootstrap_catalog.py
3. Push code and smoke the live app: HF_TOKEN=... HF_SPACE_REPO=owner/slug python scripts/publish_hf_space.py --verify (scripts/publish_hf_space.py waits for RUNNING and runs scripts/verify_hf_stack.py, including Cross-Origin-Embedder-Policy: credentialless on /health).
4. COEP on *.hf.space: the Hub merges custom_headers from the repo root README.md frontmatter onto responses. That block must set cross-origin-embedder-policy: credentialless (see root README), or the Space can show require-corp and block CDN catalog images even when api.py is correct.

Routine code-only changes need steps 1 and 3 only (step 4 is satisfied once the README on the Hub includes custom_headers).