# Hugging Face Spaces

## Where the DB lives

- **Do not** commit `store_slim.db` to the Space git repo.
- Attach a **Storage Bucket** (or dataset volume) and mount it at e.g. `/data`.
- Set **`KINK_STORE_PATH=/data/store_slim.db`** so the SQLite file lives on the volume.

## Why user accounts survive deploys

SQLite stores users in the **same file** as the catalog. If that file is on **persistent storage**, new users and plays are written into that file and remain after image updates or container restarts. If the DB only existed on ephemeral disk, it would reset when the Space restarts.

### Free alternative: snapshot user-state to a private Hub dataset

If you do not want to pay for HF Persistent Storage, set these Space variables instead:

- `KINK_USER_SNAPSHOT_REPO=<owner>/<dataset>` — a private dataset repo for the snapshot.
- `HF_TOKEN=<write-token>` — same secret used by the catalog/Space deploy.
- Optional: `KINK_USER_SNAPSHOT_FILENAME` (default `user_state.db`), `KINK_USER_SNAPSHOT_INTERVAL_S` (default 60).

Behaviour:

- On cold boot, after the catalog opens, `_restore_user_snapshot_on_boot` pulls the latest snapshot from the dataset and upserts it into the live SQLite — but only when the local user tables are empty. Failures are logged and ignored so a Hub outage cannot block boot.
- A background task in `lifespan` recomputes the user-state fingerprint every `KINK_USER_SNAPSHOT_INTERVAL_S` seconds; when it changes, it dumps `USER_TABLES` to a slim SQLite and uploads via `huggingface_hub.upload_file`. A final push runs on lifespan shutdown.
- Tables shipped: `user`, `userpreference`, `playpreference`, `rolepreference`, `scenariopreference`, `promptdismissal`, `partnerlink`, `partnerlinkrequest`, `partnergroup`, `partnergroupmember`. Catalog tables (`kink`, `play_fts*`, `similarityedge`, `fetlife*`, `kinkscenarioparent`) stay on the ephemeral image.
- Trade-off: up to ~`interval` seconds of writes can be lost on a hard crash. Multi-replica Spaces would race on push — keep cpu-basic (single replica) or add an external mutex before scaling.

## First boot (empty volume)

Pick **one** way to supply the full `store_slim.db` (multi‑GB):

1. **`KINK_CATALOG_URL`** — HTTPS URL to the file on **any** host you control (see **Free hosting** below). Optional: **`KINK_CATALOG_URL_BEARER`** for `Authorization: Bearer …` on private HTTP URLs.

2. **Backblaze B2 (private OK)** — same **Space repository secrets** as **`parser`** `scripts/hf_deploy.py`: **`B2_KEY_ID`**, **`B2_APPLICATION_KEY`**, **`B2_BUCKET`**, **`B2_REGION`**. Set **`KINK_B2_OBJECT_KEY`** to the uploaded object path (default `kink/catalog/store_slim.db`). The app uses **`boto3`** against the B2 S3 endpoint on first boot; unset **`KINK_CATALOG_URL`** when using this path.

3. **`KINK_HF_DATASET_REPO`** (+ optional **`KINK_HF_DATASET_FILENAME`**, **`KINK_HF_DATASET_REVISION`**) — download from a Hub **dataset** (see `scripts/upload_catalog_to_hub.py`).

**`deploy/hf/entrypoint.sh`** only copies the tiny bundled seed when full-catalog mode is off **and** none of URL, B2 (`B2_BUCKET` / `B2_KEY_ID`), or Hub dataset env is configured; otherwise the store path stays empty until **`ensure_store_db()`** runs.

After first boot, the file on the bucket (or `/tmp`) is the source of truth; remove URL/dataset env vars if you do not want re-download attempts (an existing file takes precedence).

## Free hosting (looked up for a ~3 GB `store_slim.db`)

**Not a substitute for a file host:** **Pinecone** (Weaviate, Qdrant, etc.) are **vector databases** for similarity search over embeddings. They do **not** store your SQLite file as-is. Putting the catalog there would mean **rebuilding the product** around vectors + APIs, not flipping one env var.

**Options that fit “free” and avoid adding financial info (no card on signup):**

| Option | Why it fits | Caveat |
|--------|----------------|--------|
| **Hugging Face Hub** (public dataset + LFS) | A normal free Hub account hosts large LFS objects; no separate object‑store vendor; already wired via `KINK_HF_DATASET_REPO` + `upload_catalog_to_hub.py`. | Hub ToS / fair use; cold‑start download time. |
| **Backblaze B2** | Same as above for free tier fit. Use **`scripts/upload_catalog_to_b2.py`** with **`[b2]`** from **`parser`** `secrets.toml`; optional env **`KINK_B2_CATALOG_BUCKET`** / **`--bucket`** targets a **dedicated** bucket if another bucket hit a storage cap. After upload: **`scripts/verify_b2_catalog_http.py`**. **Public catalog:** set **`KINK_CATALOG_URL`** to the printed native file URL. **Private catalog:** add **`B2_*`** secrets on the Space + **`KINK_B2_OBJECT_KEY`** (no public URL needed). | Anonymous GET returns **401** on private buckets—that is expected; the Space uses **authenticated S3 download**. If upload fails with **storage cap exceeded**, raise the cap or use a bucket with headroom. |

**Cloudflare R2:** generous [documented free tier](https://developers.cloudflare.com/r2/pricing/), but **Cloudflare often still asks for payment details on the account** even when usage stays at $0—skip it if your requirement is **no financial info at all**.

**Oracle Cloud “Always Free”** and similar clouds: sometimes useful, but many **require a card** for account creation even when the tier is free—check their current signup flow before counting on it.

**Usually a poor fit at ~3 GB:** GitHub/Git LFS repo limits, Supabase Storage free caps, ad‑hoc “free upload” sites (size caps, fragile hotlinks).

For the Space, set **`KINK_CATALOG_URL`** (public HTTPS), or **`B2_*`** + **`KINK_B2_OBJECT_KEY`** (private B2), or **Hub** without a third‑party object store.

## Environment

| Variable | Role |
|----------|------|
| `KINK_STORE_PATH` | Full path to SQLite on the mounted volume (required on HF) |
| `KINK_HF_REQUIRE_FULL_CATALOG` | **`0` in the Docker image** (bundled seed). Set **`1`** on the Space when you supply a full catalog via URL, B2, or Hub (no tiny bundled fallback). |
| `KINK_CATALOG_URL` | If set, **wins** over B2 and Hub: stream-download the DB from this HTTPS URL |
| `KINK_CATALOG_URL_BEARER` | Optional bearer token for private HTTP object URLs |
| `B2_KEY_ID`, `B2_APPLICATION_KEY`, `B2_BUCKET`, `B2_REGION` | Backblaze S3 credentials (Space secrets; same names as **parser**). Used when `KINK_CATALOG_URL` is unset. |
| `KINK_B2_OBJECT_KEY` | Object key inside the bucket (default `kink/catalog/store_slim.db`) |
| `KINK_HF_DATASET_REPO` | Hub dataset id (used when `KINK_HF_REQUIRE_FULL_CATALOG=1`, or with `KINK_HF_USE_HUB_DATASET=1` while full catalog is off) |
| `KINK_HF_DATASET_FILENAME` | Remote filename in the dataset (e.g. `store_slim.db`) |
| `KINK_HF_USE_HUB_DATASET` | Set **`1`** only if you want a Hub download while **`KINK_HF_REQUIRE_FULL_CATALOG=0`**. Otherwise a leftover `KINK_HF_DATASET_REPO` Space variable is **ignored** so the bundled seed is used (avoids anonymous Hub traffic and OOMs). |
| `HF_TOKEN` | If the Hub dataset is private |
| `KINK_SKIP_HEAVY_WARM` | **`1` in the published Docker image** skips only **PPR / similarity-graph** warm-up (saves RAM on **cpu-basic**). The **in-memory catalog** is still built at process start so the first recommendations request is not blocked. Set to **`0`** on larger hardware for full graph warm-up (better cold recs for heavy graph users, higher memory). |
| `KINK_CORS_ORIGINS` | Your Space URL for browser API calls |
| `KINK_RECSYS_SOURCE` | Optional: `graph` (default), `ots`, or `hybrid` when not using `Backend(..., recsys_settings=...)`. Overrides `recsys/settings.json` if present. |
| `KINK_RECSYS_CANDIDATES_PATH` | Optional: absolute path to OTS `candidates.json` (from `scripts/recsys_benchmark.py`). |

**OTS / Cornac:** beside the SQLite file, you can add `recsys/settings.json` and `recsys/candidates.json` on the same volume (copy [`deploy/recsys/settings.example.json`](../../deploy/recsys/settings.example.json) as a starting point). See repo [`docs/recsys_ots.md`](../../docs/recsys_ots.md). `GET /health` reports `recsys` (source, paths, whether the cache file exists).

Build with `Dockerfile` in the repo root or point the Space at this project.

## Release checklist

1. **Tests:** from repo root, `python -m pytest -q` (or `make test`). Partner HTTP probes are also covered in CI via `tests/test_adversarial_partner_probes_script.py`; locally: `make adv-probes-http`.
2. **Catalog / B2 / Hub bootstrap (only when the SQLite artifact or Space secrets change):**
   - B2 upload + Space secrets: [`scripts/sync_hf_space_b2_catalog.py`](../../scripts/sync_hf_space_b2_catalog.py)
   - Smaller Hub dataset bootstrap for the Space: [`scripts/deploy_hf_bootstrap_catalog.py`](../../scripts/deploy_hf_bootstrap_catalog.py)
3. **Push code and smoke the live app:** `HF_TOKEN=... HF_SPACE_REPO=owner/slug python scripts/publish_hf_space.py --verify` ([`scripts/publish_hf_space.py`](../../scripts/publish_hf_space.py) waits for RUNNING and runs [`scripts/verify_hf_stack.py`](../../scripts/verify_hf_stack.py), including **`Cross-Origin-Embedder-Policy: credentialless`** on `/health`).
4. **COEP on `*.hf.space`:** the Hub merges [`custom_headers`](https://huggingface.co/docs/hub/spaces-config-reference) from the repo **root `README.md`** frontmatter onto responses. That block must set `cross-origin-embedder-policy: credentialless` (see root README), or the Space can show `require-corp` and block CDN catalog images even when [`api.py`](../../api.py) is correct.

Routine **code-only** changes need steps **1** and **3** only (step **4** is satisfied once the README on the Hub includes `custom_headers`).