README.md

---
title: NOVA
emoji: ✨
colorFrom: purple
colorTo: indigo
sdk: gradio
sdk_version: 6.20.0
app_file: nova_app.py
python_version: "3.12"
pinned: false
short_description: Frame a research idea, find the papers, chat with them.
---

# NOVA — Research, guided by SONIC

A Gradio app that stitches two projects together, unchanged:

- **`app/`** — the research pipeline: an INTENT graph that frames your messy idea into
  Problem / Objective / Additional Context, then a SEARCH + CLUSTER graph that fetches
  from arXiv + Semantic Scholar + OpenAlex, reranks with SPECTER, and clusters by approach.
- **`chatbot_core/`** — single-PDF Q&A: `vectorizeer.py` chunks a paper by section and
  embeds it, `Qa.py` answers over it with page citations.

NOVA is the product. SONIC is the assistant persona that talks you through it.

```
your raw idea
   -> INTENT agent frames it
   -> you review/edit the framing
   -> SEARCH agent fetches + reranks + clusters
   -> paper cards, grouped by approach
   -> "Chat it out" on any card -> PDF fetched, vectorized, Q&A with citations
```

## Layout

| Path | What it is |
| --- | --- |
| `nova_app.py` | The whole UI: stage machine, event wiring, entrypoint |
| `ui/` | Presentation + glue — theme, SONIC, paper/PDF helpers, model loading |
| `app/`, `chatbot_core/` | Backend. Imported and driven, **never modified** |

`ui/paths.py` must be imported first — it does the `sys.path` + `chdir` + `.env` wiring
that makes `import app...` and `from vectorizeer import ...` resolve.

## Run it

```bash
cd NOVA
python -m venv .venv && source .venv/bin/activate

# CPU-only box: install the CPU torch wheel FIRST, or pip resolves the CUDA build
# and drags in ~2.5 GB of nvidia-*/cuda-* wheels you'll never execute. NOVA pins
# device="cpu" and never asks for a GPU.
pip install torch --index-url https://download.pytorch.org/whl/cpu

pip install -r requirements.txt
cp .env.example .env      # then fill it in
python nova_app.py        # http://localhost:7860
```

Verified end-to-end on **Python 3.14** with torch 2.13.0+cpu. Anything ≥3.10 works.
Doing the CPU-torch line first pulls in **zero** nvidia packages and lands at a 2.0 GB
venv; skip it and you get roughly double that, for a GPU the app never touches.

### Keys

`GROQ_API_KEY`, `SECOND_GROQ_API_KEY` and `TAVILY_API_KEY` are required — both graphs
read them at *import* time, so a missing key fails the boot splash, not the first search.
The rest in `.env.example` are optional and only raise rate limits.

## First boot is slow, once

NOVA loads three models totalling ~2 GB:

| Model | Loaded by | Size |
| --- | --- | --- |
| `allenai-specter` | the search graph, at import | 440 MB |
| `BAAI/bge-base-en-v1.5` | the chatbot's embeddings | 438 MB |
| `BAAI/bge-reranker-base` | the chatbot's cross-encoder | 1.1 GB |

They download from HuggingFace once and cache in `~/.cache/huggingface`. Set `HF_TOKEN`
to avoid the anonymous rate limit on that first pull.

Only the search graph loads **synchronously** behind the splash — the two chatbot models
warm on a background thread, since you can't click "Chat it out" until you've framed an
intent and run a search anyway. If the thread hasn't finished by then, the click simply
blocks on the same lock rather than loading twice.

## Deploying to Hugging Face Spaces

The YAML front-matter at the top of this file *is* the Space config — `app_file` points
at `nova_app.py`, so nothing needs renaming. The free CPU tier (16 GB RAM) is the target.

**1. Create the Space.** [huggingface.co/new-space](https://huggingface.co/new-space) →
SDK **Gradio**, hardware **CPU basic (free)**. Don't initialize it with anything.

**2. Push this folder as the Space's repo root.** `nova_app.py` must land at the top
level, not inside a `NOVA/` subfolder:

```bash
cd NOVA
git init && git branch -M main
git remote add space https://huggingface.co/spaces/<your-username>/NOVA
git add -A && git commit -m "NOVA on Gradio"
git push --force space main
```

Push over HTTPS and use an [access token](https://huggingface.co/settings/tokens) with
**write** scope as the password — your account password won't work.

**3. Add the keys** under *Settings → Variables and secrets*, as **Secrets** (not public
variables): `GROQ_API_KEY`, `SECOND_GROQ_API_KEY`, `TAVILY_API_KEY`. Spaces injects them
as env vars, which is exactly what the code reads — `load_dotenv()` finding no `.env` is
fine and expected. Add `HF_TOKEN` too, to lift the anonymous rate limit on the first
model pull.

Miss a key and the Space still boots, then shows a named error on the splash telling you
which one — it won't hang.

**4. First boot takes a few minutes** while ~2 GB of weights download. The server binds
the port *before* loading anything (models load on first visit, via `demo.load`), so the
Space goes green early and the splash does the waiting.

### Hardware: ZeroGPU or CPU basic

This repo is configured for **ZeroGPU**, and runs unmodified on **CPU basic** too — it
detects which it's on via `SPACES_ZERO_GPU` and adapts. Note HF only lets free accounts
pick the free tier *at Space creation*; you can't downgrade into it later.

ZeroGPU imposes two hard rules, and both shape the code:

1. **torch must be 2.11.0 / 2.10.0 / 2.9.1 / 2.8.0**, plain CUDA build — the `+cpu` local
   version is rejected. Hence the exact pin in `requirements.txt`.
2. **At least one `@spaces.GPU` function must exist at import**, or the Space dies with
   *"No @spaces.GPU function detected during startup"*.

A GPU exists **only inside** an `@spaces.GPU` call. Everything outliving that window must
be CPU-resident, which is why `get_embeddings`, `get_retriever` and the SPECTER model now
take an explicit `device` (defaulting to `cpu`) instead of auto-detecting. Auto-detect is
the trap: torch reports a GPU at import on ZeroGPU, so a model would load onto `cuda` and
then fail on first use out in a LangGraph node.

So exactly one thing runs on the GPU — the bulk chunk embedding in
[ui/gpu.py](ui/gpu.py), the slowest step in the app. It builds and **persists** the
vectorstore, then the caller re-opens it on CPU; what crosses the GPU boundary is the
file on disk, never a cuda-resident object.

### Verified

- `sdk_version: 6.20.0` — the code needs Gradio 6 (`launch()` takes `theme`/`css`/`js`;
  `Chatbot` dropped `type=`), and 6.20.0 is what Spaces currently serves.
- `python_version: "3.12"` — the full dependency set resolves cleanly there with
  `torch==2.11.0`.
- Off ZeroGPU, `@spaces.GPU` is a transparent no-op, so local runs and CPU hardware are
  unaffected — the CPU path keeps the threaded progress animation, and only the ZeroGPU
  path calls the GPU inline (it must run on the caller's thread, not one we spawn).

### Why not Streamlit Community Cloud

That's what this rewrite escapes. It caps at roughly 1 GB of RAM, so ~2 GB of weights got
the container OOM-killed mid-load; it restarted, re-entered boot, and sat on the splash
forever. Any host needs **≥ 4 GB RAM**.

## Notes for the next person

- **State.** Streamlit re-ran the script top-to-bottom and branched on
  `session_state.stage`. Gradio builds the component graph once, so a stage is a
  `gr.Column` and every handler returns the `visible` flag for all seven. Same state
  machine, declared instead of re-derived.
- **The card grid** is inside `@gr.render` because each card's button needs a real
  handler closing over its paper key. It redraws off `clusters_state.change` — a
  `gr.State` reassigned to a fresh list, since `gr.render` can't see a dict mutated
  in place.
- **Models are process-global**, not per-session, behind a double-checked lock in
  `ui/agents.py`. Gradio serves from a thread pool, so without the lock two
  simultaneous first-visitors would each kick off a 2 GB load.
- **The two graphs share `SECOND_GROQ_API_KEY`** (see `app/modules/search/graph.py`) —
  `GROQ_API_KEY` is read but only `second_api_key` is actually passed to both `ChatGroq`
  instances. Pre-existing upstream behaviour, left alone. Set both keys.