docs/deploy-fly.md

# Deploying rag-psych to Fly.io

Architecture: **Fly.io for the API container, Neon for Postgres + pgvector.**
Both have free tiers that cover the demo. The API auto-stops when idle so
the bill stays at $0 between visits.

> **Critical:** never run `--sources dsm5` against the remote DB. The DSM
> chunks are licensed for local personal use only and must stay in your
> laptop's `pgdata` volume. The `scripts/seed_remote.sh` helper rejects
> any attempt to ingest DSM remotely.

---

## 1. Provision Postgres on Neon (free)

1. Sign up at [console.neon.tech](https://console.neon.tech) (GitHub auth, free).
2. Create a new project. Region: pick one near your Fly region (we'll use
   `ord` below; Neon's `aws-us-east-2` is closest).
3. In the project dashboard, go to **Settings → Extensions** and enable
   `vector`. (One toggle. Neon ships pgvector pre-installed; you just
   activate it.)
4. Copy the **Connection string** from the dashboard. It looks like:
   ```
   postgresql://USER:PASSWORD@ep-xyz-12345.us-east-2.aws.neon.tech/rag_psych?sslmode=require
   ```
5. Apply our schema. From your laptop:
   ```bash
   psql 'postgresql://USER:PASSWORD@ep-xyz-12345.us-east-2.aws.neon.tech/rag_psych?sslmode=require' \
     -f ingest/schema.sql
   ```

---

## 2. Install Fly's CLI and authenticate

```bash
brew install flyctl              # macOS; see fly.io/docs/hands-on/install-flyctl/ for others
fly auth signup                  # or: fly auth login (browser)
```

Free trial credit covers the demo for the first month. After that the
runtime cost is dominated by uptime, which scale-to-zero pushes near zero.

---

## 3. Launch the Fly app (no deploy yet)

From the repo root:

```bash
fly launch --no-deploy --copy-config --name rag-psych-<your-suffix>
```

When prompted:
- **Region**: pick the same one your Neon DB is closest to (e.g. `ord`)
- **Postgres**: **No** (we're using Neon, not Fly Postgres)
- **Redis**: No
- **Settings**: keep the existing `fly.toml` (the `--copy-config` flag preserves it)

Open the generated `fly.toml` and update the `app = "..."` line to match
the unique name Fly assigned (the launcher overwrites it).

---

## 4. Set secrets

These never appear in `fly.toml` or the image. They live encrypted in
Fly's secret store and are injected as env vars at runtime.

```bash
fly secrets set \
  DATABASE_URL='postgresql://USER:PASSWORD@ep-xyz.us-east-2.aws.neon.tech/rag_psych?sslmode=require' \
  ANTHROPIC_API_KEY='sk-ant-api03-...' \
  ANTHROPIC_MODEL='claude-haiku-4-5' \
  NCBI_EMAIL='you@example.com' \
  ICD_CLIENT_ID='...' \
  ICD_CLIENT_SECRET='...' \
  EVAL_PASSWORD="$(python3 -c 'import secrets; print(secrets.token_urlsafe(16))')" \
  CORS_ORIGIN='https://rag-psych-<your-suffix>.fly.dev'
```

> ⚠️ **Rotate your local `.env` keys before deploying.** The
> `ANTHROPIC_API_KEY` and `EVAL_PASSWORD` currently in `.env` should be
> regenerated for production — assume the old ones are compromised
> (anything that touches a chat with an LLM is). Console:
> [console.anthropic.com/settings/keys](https://console.anthropic.com/settings/keys).

---

## 5. Deploy

```bash
fly deploy
```

First push uploads ~3.5 GB (models baked into the image — see
`api/Dockerfile`). Subsequent deploys only push the layers that changed,
so code edits redeploy in seconds.

When it finishes:

```bash
fly status                        # check machine health
fly logs                          # follow startup logs
open https://rag-psych-<your-suffix>.fly.dev/health
```

A `{"status":"ok"}` response means the API is up. The first `/query`
will pay a 5–10 s cold-start while the embedder + reranker load.

---

## 6. Seed the remote database

The Neon DB is empty after step 1. Run ingest from your laptop against it:

```bash
DATABASE_URL='postgresql://USER:PASSWORD@ep-xyz.us-east-2.aws.neon.tech/rag_psych?sslmode=require' \
  ./scripts/seed_remote.sh
```

This takes 10–15 minutes wall time, mostly the PubMed `efetch` loop on
the first run. The cached JSON files in `data/cache/` mean re-runs are
near-instant.

After it finishes, hit the deployed UI:

```bash
open https://rag-psych-<your-suffix>.fly.dev/ui
```

---

## 7. Production hardening (recommended before sharing publicly)

| Item | How |
|---|---|
| Anthropic spend cap | Drop to **$5/week** at console.anthropic.com → Settings → Limits before sharing the URL |
| `/eval` password | Confirm `EVAL_PASSWORD` is random + long. Never the local-dev value. |
| Rate limit | Already 30/min/IP. If you see abuse, drop to 10/min in `api/main.py`'s `@limiter.limit` decorator and `fly deploy` |
| CORS | Only the Fly subdomain in production. If you remove the localhost dev origin from the Fly secret, dev still works locally because your `.env` has its own value |
| Healthcheck grace | `fly.toml` already gives 30 s for cold-start. If you see flapping, bump to 60 s |
| Auto-stop | Already on (`auto_stop_machines = "stop"`). Verify with `fly status` — idle machines should report `stopped` |

---

## 8. Day-2 ops cheatsheet

```bash
fly status                       # which machines are running
fly logs                         # tail combined logs
fly logs --instance <id>         # one machine
fly ssh console                  # shell into a running instance
fly secrets list                 # what's set (values not shown)
fly secrets unset SOME_KEY       # remove
fly scale memory 4096            # bump RAM if rerank latency is bad
fly scale count 2                # add a second machine for redundancy
fly apps destroy rag-psych-<...> # nuke everything (careful)
```

---

## 9. Things you'll feel in production that you don't on localhost

- **First request is slow** (5–10 s) when the machine just woke from
  auto-stop. Subsequent requests in the same minute are fast.
- **Latency floor is higher** because every request crosses the public
  internet (Fly ↔ Neon) instead of localhost. Expect ~2× the times shown
  in `eval/results/*.json`.
- **Anthropic costs scale with usage.** A naive demo URL hit by 100 curious
  visitors at 5 queries each = ~$2 of Haiku tokens. Spend cap protects you.
- **No DSM in answers.** If a query that worked locally suddenly returns
  the refusal string in production, you're hitting the DSM-shaped hole in
  the public corpus — that's expected.

---

## Why not Fly Postgres?

Fly's managed Postgres is fine but you'd have to install pgvector via a
custom Postgres image and manage it yourself, then pay $5/mo for the
smallest instance. Neon's free tier is functionally equivalent for our
scale (~30K chunks) and has zero setup beyond toggling the extension.

If you want everything inside Fly's network later (lower latency,
fewer external dependencies), swap `DATABASE_URL` to a Fly Postgres
endpoint and re-run step 6. The application code doesn't care.