docs/SPEC.md

SupraDashboard — Product Spec (v0)

CB[7] (cucurbit[7]uril) host–guest binding-affinity (logKa) reasoning dashboard. Gradio app → Hugging Face Docker Space. Inference via a CLIPROXYAPI OpenAI-compatible endpoint (codex closedbook GPT-5.5). Public GitHub repo, unpublished research → no private prompts/data committed.

---

1. Problem & goal

We have an LLM reasoning system that predicts CB[7] host–guest binding affinity (logKa) from precomputed molecular features, producing a chain-of-thought with numbered binding-driver rules, a one-line summary, and a numeric prediction. The research is unpublished and the reasoning quality must be judged by domain experts, not by automated metrics alone. The goal of v0 is a lightweight internal review tool: pick a guest, render its structure, run three reasoning trajectories (physics-guided / chemistry-guided / combined) over precomputed features, show each trajectory's predicted logKa + <tldr> + the combined <think> reasoning (rule headers color-coded), and capture a structured expert reasoning-quality rating plus a free-text comment — so we accumulate expert judgments on whether the model's reasoning is chemically sound.

2. Target users & job-to-be-done

• Primary users: ~3 domain-expert supramolecular chemists (internal collaborators).
• NOT a public consumer app; no anonymous traffic, no scale concerns.
• Job-to-be-done: "For this guest, is the model's reasoning chemically sound, and where does it go wrong?" — read the structure + features + reasoning, then record a calibrated quality rating and pinpoint flawed feature interpretations in a comment, fast enough to review many guests in a sitting.

3. v0 scope

MUST have

• Guest selection from a dropdown populated by the private HF feature dataset.
• RDKit 2D structure render from the guest SMILES.
• Three reasoning trajectories per guest — physics-guided, chemistry-guided, combined — each calling the CLIPROXYAPI endpoint over that guest's precomputed features (physics/chemistry trajectories receive feature subsets; combined gets all).
• Parse and display from each trajectory: predicted logKa (<answer>), one-line <tldr>; show the combined trajectory's <think> reasoning with numbered binding-driver rule headers bolded and color-coded.
• A predicted-logKa comparison table (combined / physics / chemistry).
• Expert rating widget with the 5-point scale (Absolutely right / Mostly right / Partially right (mixed) / Mostly wrong / Absolutely wrong) + free-text comment, persisted to a feedback store.
• Config entirely by env vars / Space Secrets — no secrets in code or repo.
• Password login wall via APP_AUTH (Gradio auth); recommended on by default.
• Production prompts loaded at runtime from PROMPT_DIR (never committed); repo ships only a generic public-safe prompt scaffold.
• Features pulled at runtime from a private HF dataset via HF_TOKEN.
• Loading / busy state during the (slow) inference call so reviewers aren't left staring at a frozen UI.
• Persistent feedback on Spaces (point FEEDBACK_DB at the /data mount).
• An admin/export path to retrieve collected ratings (CSV/rows).

Out of scope / deferred (design-for-later, do NOT build in v0)

• Live feature computation (docking / xtb / RDKit conformer generation). v0 uses precomputed features only. The Docker Space base is chosen so xtb/docking can be added later — keep data_loader the single seam where "precomputed vs live" is decided.
• Multi-user accounts / per-reviewer identity & auth roles (shared password only in v0; see Open Questions on reviewer attribution).
• Editing/curating features or prompts from the UI.
• Inter-rater agreement analytics, dashboards over collected ratings, model retraining hooks.
• Public/anonymous deployment, rate limiting, abuse protection.
• Batch / bulk scoring of many guests in one action.
• Result caching across sessions (see Open Questions — may be promoted into v0).

4. User flow (happy path)

1. Reviewer opens the Space, passes the APP_AUTH password wall.
2. App shows a health line (proxy configured? dataset loaded?) and a guest dropdown.
3. Reviewer picks a guest and clicks Run prediction.
4. UI enters a loading state; the three trajectories run and the proxy returns text.
5. App renders: guest 2D structure; predicted-logKa table (combined/physics/chemistry); physics <tldr>; chemistry <tldr>; combined <think> reasoning with color-coded numbered rule headers.
6. Reviewer reads the reasoning, picks a 5-point rating, and writes a comment.
7. Reviewer clicks Submit feedback; a confirmation with the saved row id appears.
8. Reviewer picks the next guest and repeats.

5. Functional requirements

• FR1 — The app SHALL load the guest list from the private HF dataset (HF_DATASET, default SupraBench/physics_feature) using HF_TOKEN, with a LOCAL_FEATURES CSV fallback for offline dev. Dropdown labels use guest_name, falling back to inchikey.
• FR2 — On guest selection, the app SHALL render a 2D RDKit structure from the guest's SMILES, and SHALL degrade gracefully (no crash, empty image) on missing/ invalid SMILES or absent RDKit.
• FR3 — The app SHALL build three prompts per guest: physics (physics feature subset), chemistry (complementary subset), combined (all features), using the scaffold prompts unless PROMPT_DIR/{physics,chemistry,combined}.txt overrides exist.
• FR4 — The app SHALL call the CLIPROXYAPI OpenAI-compatible endpoint (CLIPROXY_BASE_URL, CLIPROXY_API_KEY, CLIPROXY_MODEL) once per trajectory and return the raw completion text.
• FR5 — The app SHALL parse <answer> → numeric logKa, <tldr> → summary, and <think> → reasoning from each completion, tolerating missing tags (show —).
• FR6 — The app SHALL display a logKa comparison table for the three trajectories and render the combined <think> with numbered rule headers (N. Title — … or N. Title: …) bolded and color-coded by rule index.
• FR7 — The app SHALL present the 5-point reasoning-quality rating and a free-text comment box, and SHALL refuse submission with a visible warning if no rating is chosen.
• FR8 — On submit, the app SHALL persist {ts (UTC ISO), inchikey, guest_name, rating, comment, reviewer} to the feedback store and confirm with the row id.
• FR9 — Feedback SHALL persist across Space restarts when FEEDBACK_DB points at the persistent-storage mount; otherwise it MAY be ephemeral (documented).
• FR10 — The app SHALL enforce a password login wall when APP_AUTH is set (comma-separated user:pass pairs) and run open when it is blank.
• FR11 — The app SHALL show a health/status line indicating whether the proxy is configured and whether the dataset loaded, surfacing config errors as readable text rather than crashing on startup.
• FR12 — There SHALL be a path to export all collected feedback rows (e.g. an admin export tab / CSV download), gated behind the same access control.

6. Non-functional requirements

• NFR1 (latency / async). codex closedbook inference is slow (multi-second to tens-of-seconds per call) and v0 runs three calls per guest. The UI MUST show an explicit loading/busy state on Run prediction and MUST NOT appear frozen. Long request timeouts MUST be configured on the proxy client; a per-call failure SHOULD surface as readable error text without losing already-rendered panels. (Sequential vs concurrent execution and caching are Open Questions that directly affect perceived latency.)
• NFR2 (secrets). No secret (CLIPROXY_API_KEY, HF_TOKEN) ever appears in code, logs, or the repo. All config is env vars locally (.env, gitignored) / Space Secrets in prod. Only .env.example (placeholders) is committed.
• NFR3 (no data/prompt leakage). The public repo MUST contain no unpublished data and no production prompts — features come from the private HF dataset at runtime; production prompts mount via PROMPT_DIR. CI/repo hygiene MUST keep feature CSVs and prompt files out of git.
• NFR4 (access control). Because the research is unpublished, the deployed Space SHOULD keep APP_AUTH set and/or be a private Space. Access is a shared password for the ~3 reviewers in v0.
• NFR5 (graceful degradation). Missing dataset, unconfigured proxy, missing RDKit, or malformed model output MUST degrade to readable status/placeholder, never an uncaught crash at startup or per-guest.
• NFR6 (portability). Runs identically locally (python app.py, port 7860) and on a HF Docker Space; RDKit native libs provided in the Dockerfile.

7. Acceptance criteria (verifier checklist)

• With valid .env, python app.py serves the dashboard on :7860.
• The guest dropdown is populated from the (private HF or LOCAL_FEATURES) dataset.
• Selecting a guest + Run prediction renders the 2D structure for a valid SMILES.
• Three trajectories execute and a logKa table shows combined/physics/chemistry values (or — when a tag is missing).
• Physics and chemistry <tldr> lines render; combined <think> renders with numbered rule headers color-coded.
• A loading/busy indicator is visible while inference runs.
• Submitting without a rating shows a warning and does not persist a row.
• Submitting with a rating persists a row and confirms a row id; the row survives a restart when FEEDBACK_DB is on persistent storage.
• With APP_AUTH set, the app requires login; blank disables it.
• With the proxy or dataset misconfigured, the app starts and shows a readable status error instead of crashing.
• git grep finds no real API keys, HF_TOKEN, production prompts, or feature CSVs in the repo; only .env.example placeholders are present.
• Collected feedback can be exported (rows/CSV) by an admin path.

8. Open questions / decisions needed from the user

1. CLIPROXYAPI endpoint + model string. Exact production CLIPROXY_BASE_URL and the CLIPROXY_MODEL string that maps to codex closedbook GPT-5.5 (scaffold default is gpt-5.5-codex). Also: expected per-call latency and a sane client timeout.
2. HF dataset contract. Confirm HF_DATASET repo id, the filename (data_loader hardcodes physics_feature.csv), and that the columns include inchikey, guest_name, smiles plus the 22 feature columns. Is the schema stable, or should loading be column-tolerant?
3. Sequential vs concurrent trajectories. v0 currently runs the three calls in a loop (sequential → ~3× latency). Run them concurrently to cut wait time? (Affects NFR1 and proxy rate limits.)
4. Caching. Cache inference results per (guest, prompt-version) so re-opening a guest is instant and we don't re-bill the proxy? If yes, in-memory vs persistent, and how is it invalidated when PROMPT_DIR prompts change?
5. Reviewer attribution. The feedback schema has a reviewer column but the UI never sets it. Do we want per-reviewer attribution (e.g. from the APP_AUTH login) for inter-rater analysis, or is anonymous v0 fine?
6. Deployment privacy. Private Space vs public Space + APP_AUTH only? And where does persistent storage / FEEDBACK_DB live, and who exports the ratings?
7. Prompt delivery. How do production prompts reach PROMPT_DIR on the Space (build- time mount, secret, or pulled from a private repo)? Confirm the expected physics.txt / chemistry.txt / combined.txt contract.