deploy/DEPLOY.md

# Deploying Fugee — Hugging Face Space + Modal

The demo runs in two pieces: the **Gradio UI on a free HF CPU Space**, calling the
**LLM + embeddings on a GPU Ollama endpoint on Modal**. Same code, same model
(`lfm2.5:8b`), just on rented GPU.

```
HF Space (free CPU)  ──HTTPS, proxy-auth──▶  Modal (L4 GPU, ollama serve)
app/app.py                                    lfm2.5:8b + nomic-embed-text
```

Run the steps in order. Steps that only the maintainer can do (auth, secrets) are
marked **[you]**.

---

## 0. Prerequisites (already done in this repo)

- `deploy/modal_app.py` is deployed and **kept warm**
  (`MODAL_MIN_CONTAINERS=1`) → endpoint `https://hf-labs--fugee-ollama-serve.modal.run`.
- A Modal **Proxy Auth Token** exists; its id/secret are in `.env.modal`
  (gitignored) as `MODAL_KEY` / `MODAL_SECRET`.
- Models cached in the Modal Volume (`modal run deploy/modal_app.py::download_models`).

Re-check the endpoint is warm before a demo:

```bash
set -a; . ./.env.modal; set +a
curl -s -o /dev/null -w "%{http_code}\n" \
  https://hf-labs--fugee-ollama-serve.modal.run/api/version \
  -H "Modal-Key: $MODAL_KEY" -H "Modal-Secret: $MODAL_SECRET"     # expect 200
```

---

## 1. **[you]** Authenticate the HF CLI

Create a token with **write** access at <https://huggingface.co/settings/tokens>,
then:

```bash
hf auth login        # paste the write token
```

---

## 2. **[you]** Create the Space in the hackathon org

Web: **New Space** → Owner `build-small-hackathon`, name `fugee`, SDK **Gradio**,
hardware **CPU basic (free)**, **Public**. Or CLI:

```bash
hf repo create build-small-hackathon/fugee --repo-type space --space-sdk gradio
```

(If the flag name differs on your CLI version, run `hf repo create --help`.)

---

## 3. Push the code to the Space

The Space is its own git repo. Add it as a remote and push `main` (force, to
replace the starter README HF created on the first deploy):

```bash
git remote add space https://huggingface.co/spaces/build-small-hackathon/fugee
git push space main --force
```

This pushes everything except gitignored files — code, `countries.json`,
`countries_enriched.json`, bundled fonts, etc. It does **not** include the binary
data the HF Hub requires Xet/LFS for — the 23 MB RAG index and the source PDFs —
which are uploaded next. The Space starts building on push.

---

## 4. Upload the binary data (RAG index + source PDFs)

The HF Hub requires binary files to use Xet/LFS, which `hf upload` does
automatically (no local git-lfs needed).

The RAG index — needed at runtime by `guideline_search`:

```bash
hf upload build-small-hackathon/fugee \
  specs/data/guidelines_index.json specs/data/guidelines_index.json \
  --repo-type space
```

The 15 source UNHCR PDFs — not needed at runtime, but uploaded so the index's
provenance is visible on the Space (upload the whole folder):

```bash
hf upload build-small-hackathon/fugee \
  specs/data/guidelines specs/data/guidelines \
  --repo-type space
```

---

## 5. **[you]** Set the Space secrets & variables

Space → **Settings → Variables and secrets**. The app reads these from the
environment (its `.env` loader never overrides real env vars).

**Secrets** (encrypted):

| Name | Value |
|------|-------|
| `OLLAMA_HOST` | `https://hf-labs--fugee-ollama-serve.modal.run` |
| `MODAL_KEY` | your Modal Proxy Auth Token id (`wk-…`) |
| `MODAL_SECRET` | your Modal Proxy Auth Token secret (`ws-…`) |

**Variables** (public):

| Name | Value |
|------|-------|
| `MODEL_ID` | `lfm2.5:8b` |
| `MODEL_PROVIDER` | `ollama` |
| `NUM_CTX` | `16384` |

> `MODEL_ID` **must** be set — the code default is `qwen2.5:7b`, which isn't on
> the Modal endpoint. `OLLAMA_HOST` + `MODAL_KEY` + `MODAL_SECRET` are required for
> the Space to reach the model. The Modal **account** token never goes here — it
> stays in `~/.modal.toml` on your machine.

Setting a secret restarts the Space.

---

## 6. Verify the live Space

Open `https://huggingface.co/spaces/build-small-hackathon/fugee`, wait for the
build (watch the **Logs** tab), then walk a full flow: language → interview →
assessment → recommendations → documents. First model call may take a few seconds
even though Modal is warm.

If PDF generation errors in the logs, it's a missing system lib — confirm
`packages.txt` installed (Pango/HarfBuzz/Noto fonts).

---

## 7. Submit & costs

- **Submission:** on the hackathon *submit* page, enter `fugee` under
  `build-small-hackathon/` → it loads the README and writes track/badge tags into
  the frontmatter → commit that back (`git pull space main` then push, or edit on HF).
- **GPU limit:** the "10 ZeroGPU apps per user" rule does **not** apply — we use
  Modal, not ZeroGPU.
- **After the demo window**, stop the Modal meter:

  ```bash
  MODAL_MIN_CONTAINERS=0 modal deploy deploy/modal_app.py   # scale to zero
  # or fully stop:
  modal app stop fugee-ollama
  ```

  Scale-to-zero keeps the deployment but costs nothing while idle (cold start
  ~40 s on the next request).