docs/SPEC.md

# SupraDashboard — Product Spec (v0)

> CB[7] (cucurbit[7]uril) host–guest binding-affinity (logKa) reasoning dashboard.
> Gradio app → Hugging Face **Docker** Space. Inference via a CLIPROXYAPI
> OpenAI-compatible endpoint (codex closedbook GPT-5.5). Public GitHub repo,
> **unpublished** research → no private prompts/data committed.

---

## 1. Problem & goal

We have an LLM reasoning system that predicts CB[7] host–guest binding affinity
(logKa) from precomputed molecular features, producing a chain-of-thought with
numbered binding-driver rules, a one-line summary, and a numeric prediction. The
research is unpublished and the reasoning quality must be judged by domain experts,
not by automated metrics alone. **The goal of v0 is a lightweight internal review
tool**: pick a guest, render its structure, run three reasoning trajectories
(physics-guided / chemistry-guided / combined) over precomputed features, show each
trajectory's predicted logKa + `<tldr>` + the combined `<think>` reasoning (rule
headers color-coded), and capture a structured expert reasoning-quality rating plus
a free-text comment — so we accumulate expert judgments on whether the model's
reasoning is chemically sound.

## 2. Target users & job-to-be-done

- **Primary users:** ~3 domain-expert supramolecular chemists (internal collaborators).
- **NOT** a public consumer app; no anonymous traffic, no scale concerns.
- **Job-to-be-done:** *"For this guest, is the model's reasoning chemically sound,
  and where does it go wrong?"* — read the structure + features + reasoning, then
  record a calibrated quality rating and pinpoint flawed feature interpretations in a
  comment, fast enough to review many guests in a sitting.

## 3. v0 scope

### MUST have
- Guest selection from a dropdown populated by the private HF feature dataset.
- RDKit 2D structure render from the guest SMILES.
- Three reasoning trajectories per guest — **physics-guided**, **chemistry-guided**,
  **combined** — each calling the CLIPROXYAPI endpoint over that guest's precomputed
  features (physics/chemistry trajectories receive feature subsets; combined gets all).
- Parse and display from each trajectory: predicted `logKa` (`<answer>`), one-line
  `<tldr>`; show the **combined** trajectory's `<think>` reasoning with numbered
  binding-driver rule headers bolded and color-coded.
- A predicted-logKa comparison table (combined / physics / chemistry).
- Expert rating widget with the 5-point scale (Absolutely right / Mostly right /
  Partially right (mixed) / Mostly wrong / Absolutely wrong) + free-text comment,
  persisted to a feedback store.
- Config **entirely by env vars / Space Secrets** — no secrets in code or repo.
- Password login wall via `APP_AUTH` (Gradio `auth`); recommended on by default.
- Production prompts loaded at runtime from `PROMPT_DIR` (never committed); repo ships
  only a generic public-safe prompt scaffold.
- Features pulled at runtime from a **private** HF dataset via `HF_TOKEN`.
- Loading / busy state during the (slow) inference call so reviewers aren't left
  staring at a frozen UI.
- Persistent feedback on Spaces (point `FEEDBACK_DB` at the `/data` mount).
- An admin/export path to retrieve collected ratings (CSV/rows).

### Out of scope / deferred (design-for-later, do NOT build in v0)
- **Live feature computation** (docking / xtb / RDKit conformer generation). v0 uses
  **precomputed** features only. The Docker Space base is chosen so xtb/docking can be
  added later — keep `data_loader` the single seam where "precomputed vs live" is
  decided.
- Multi-user accounts / per-reviewer identity & auth roles (shared password only in
  v0; see Open Questions on reviewer attribution).
- Editing/curating features or prompts from the UI.
- Inter-rater agreement analytics, dashboards over collected ratings, model retraining
  hooks.
- Public/anonymous deployment, rate limiting, abuse protection.
- Batch / bulk scoring of many guests in one action.
- Result caching across sessions (see Open Questions — may be promoted into v0).

## 4. User flow (happy path)

1. Reviewer opens the Space, passes the `APP_AUTH` password wall.
2. App shows a health line (proxy configured? dataset loaded?) and a guest dropdown.
3. Reviewer picks a guest and clicks **Run prediction**.
4. UI enters a loading state; the three trajectories run and the proxy returns text.
5. App renders: guest 2D structure; predicted-logKa table (combined/physics/chemistry);
   physics `<tldr>`; chemistry `<tldr>`; combined `<think>` reasoning with color-coded
   numbered rule headers.
6. Reviewer reads the reasoning, picks a 5-point rating, and writes a comment.
7. Reviewer clicks **Submit feedback**; a confirmation with the saved row id appears.
8. Reviewer picks the next guest and repeats.

## 5. Functional requirements

- **FR1** — The app SHALL load the guest list from the private HF dataset (`HF_DATASET`,
  default `SupraBench/physics_feature`) using `HF_TOKEN`, with a `LOCAL_FEATURES` CSV
  fallback for offline dev. Dropdown labels use `guest_name`, falling back to `inchikey`.
- **FR2** — On guest selection, the app SHALL render a 2D RDKit structure from the
  guest's SMILES, and SHALL degrade gracefully (no crash, empty image) on missing/
  invalid SMILES or absent RDKit.
- **FR3** — The app SHALL build three prompts per guest: physics (physics feature
  subset), chemistry (complementary subset), combined (all features), using the
  scaffold prompts unless `PROMPT_DIR/{physics,chemistry,combined}.txt` overrides exist.
- **FR4** — The app SHALL call the CLIPROXYAPI OpenAI-compatible endpoint
  (`CLIPROXY_BASE_URL`, `CLIPROXY_API_KEY`, `CLIPROXY_MODEL`) once per trajectory and
  return the raw completion text.
- **FR5** — The app SHALL parse `<answer>` → numeric logKa, `<tldr>` → summary, and
  `<think>` → reasoning from each completion, tolerating missing tags (show `—`).
- **FR6** — The app SHALL display a logKa comparison table for the three trajectories
  and render the combined `<think>` with numbered rule headers (`N. Title — …` or
  `N. Title: …`) bolded and color-coded by rule index.
- **FR7** — The app SHALL present the 5-point reasoning-quality rating and a free-text
  comment box, and SHALL refuse submission with a visible warning if no rating is chosen.
- **FR8** — On submit, the app SHALL persist `{ts (UTC ISO), inchikey, guest_name,
  rating, comment, reviewer}` to the feedback store and confirm with the row id.
- **FR9** — Feedback SHALL persist across Space restarts when `FEEDBACK_DB` points at the
  persistent-storage mount; otherwise it MAY be ephemeral (documented).
- **FR10** — The app SHALL enforce a password login wall when `APP_AUTH` is set
  (comma-separated `user:pass` pairs) and run open when it is blank.
- **FR11** — The app SHALL show a health/status line indicating whether the proxy is
  configured and whether the dataset loaded, surfacing config errors as readable text
  rather than crashing on startup.
- **FR12** — There SHALL be a path to export all collected feedback rows (e.g. an
  admin export tab / CSV download), gated behind the same access control.

## 6. Non-functional requirements

- **NFR1 (latency / async).** codex closedbook inference is **slow** (multi-second to
  tens-of-seconds per call) and v0 runs **three** calls per guest. The UI MUST show an
  explicit loading/busy state on **Run prediction** and MUST NOT appear frozen. Long
  request timeouts MUST be configured on the proxy client; a per-call failure SHOULD
  surface as readable error text without losing already-rendered panels. (Sequential vs
  concurrent execution and caching are Open Questions that directly affect perceived
  latency.)
- **NFR2 (secrets).** No secret (`CLIPROXY_API_KEY`, `HF_TOKEN`) ever appears in code,
  logs, or the repo. All config is env vars locally (`.env`, gitignored) / Space Secrets
  in prod. Only `.env.example` (placeholders) is committed.
- **NFR3 (no data/prompt leakage).** The public repo MUST contain no unpublished data
  and no production prompts — features come from the private HF dataset at runtime;
  production prompts mount via `PROMPT_DIR`. CI/repo hygiene MUST keep feature CSVs and
  prompt files out of git.
- **NFR4 (access control).** Because the research is unpublished, the deployed Space
  SHOULD keep `APP_AUTH` set and/or be a private Space. Access is a shared password for
  the ~3 reviewers in v0.
- **NFR5 (graceful degradation).** Missing dataset, unconfigured proxy, missing RDKit,
  or malformed model output MUST degrade to readable status/placeholder, never an
  uncaught crash at startup or per-guest.
- **NFR6 (portability).** Runs identically locally (`python app.py`, port 7860) and on a
  HF Docker Space; RDKit native libs provided in the Dockerfile.

## 7. Acceptance criteria (verifier checklist)

- [ ] With valid `.env`, `python app.py` serves the dashboard on :7860.
- [ ] The guest dropdown is populated from the (private HF or `LOCAL_FEATURES`) dataset.
- [ ] Selecting a guest + Run prediction renders the 2D structure for a valid SMILES.
- [ ] Three trajectories execute and a logKa table shows combined/physics/chemistry values
      (or `—` when a tag is missing).
- [ ] Physics and chemistry `<tldr>` lines render; combined `<think>` renders with
      numbered rule headers color-coded.
- [ ] A loading/busy indicator is visible while inference runs.
- [ ] Submitting without a rating shows a warning and does not persist a row.
- [ ] Submitting with a rating persists a row and confirms a row id; the row survives a
      restart when `FEEDBACK_DB` is on persistent storage.
- [ ] With `APP_AUTH` set, the app requires login; blank disables it.
- [ ] With the proxy or dataset misconfigured, the app starts and shows a readable
      status error instead of crashing.
- [ ] `git grep` finds no real API keys, `HF_TOKEN`, production prompts, or feature CSVs
      in the repo; only `.env.example` placeholders are present.
- [ ] Collected feedback can be exported (rows/CSV) by an admin path.

## 8. Open questions / decisions needed from the user

1. **CLIPROXYAPI endpoint + model string.** Exact production `CLIPROXY_BASE_URL` and the
   `CLIPROXY_MODEL` string that maps to codex closedbook GPT-5.5 (scaffold default is
   `gpt-5.5-codex`). Also: expected per-call latency and a sane client timeout.
2. **HF dataset contract.** Confirm `HF_DATASET` repo id, the **filename** (`data_loader`
   hardcodes `physics_feature.csv`), and that the columns include `inchikey`,
   `guest_name`, `smiles` plus the 22 feature columns. Is the schema stable, or should
   loading be column-tolerant?
3. **Sequential vs concurrent trajectories.** v0 currently runs the three calls in a
   loop (sequential → ~3× latency). Run them concurrently to cut wait time? (Affects NFR1
   and proxy rate limits.)
4. **Caching.** Cache inference results per (guest, prompt-version) so re-opening a guest
   is instant and we don't re-bill the proxy? If yes, in-memory vs persistent, and how is
   it invalidated when `PROMPT_DIR` prompts change?
5. **Reviewer attribution.** The feedback schema has a `reviewer` column but the UI never
   sets it. Do we want per-reviewer attribution (e.g. from the `APP_AUTH` login) for
   inter-rater analysis, or is anonymous v0 fine?
6. **Deployment privacy.** Private Space vs public Space + `APP_AUTH` only? And where does
   persistent storage / `FEEDBACK_DB` live, and who exports the ratings?
7. **Prompt delivery.** How do production prompts reach `PROMPT_DIR` on the Space (build-
   time mount, secret, or pulled from a private repo)? Confirm the expected
   `physics.txt / chemistry.txt / combined.txt` contract.