README.md

---
title: Placement Policy Advisor
emoji: 🎓
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: "6.20.0"
app_file: main.py
pinned: false
short_description: RAG chatbot for the RV University Placement Policy
---

# Placement Policy Advisor — Institutional Policy RAG Chatbot

A self-contained Retrieval-Augmented Generation (RAG) chatbot that answers
questions about the **RV University Placement Policy**, grounded strictly in the
official policy documents. A single FastAPI process with a Gradio chat UI
mounted on the same app; deployable either as a Docker container or as a plain
Python app (e.g. a Hugging Face **Gradio SDK** Space — free, no card required,
since Docker SDK Spaces now require payment).

## Architecture

| Concern | Choice |
|---|---|
| API + UI | **FastAPI** with a **Gradio `ChatInterface`** mounted at `/` via `gr.mount_gradio_app()` (single process, one port) |
| Vector DB | **Qdrant** in local/embedded mode — `QdrantClient(path=...)`, no server process |
| Embeddings | **FastEmbed** (ONNX), fully local CPU inference, model from `EMBEDDING_MODEL_ID` |
| LLM | `huggingface_hub.AsyncInferenceClient` → `chat.completions.create(..., stream=True)`, with automatic fallback from `LLM_MODEL_ID` to `LLM_FALLBACK_MODEL_ID` on error/timeout |
| Chunking | `MarkdownHeaderTextSplitter` (by `#`/`##`/`###`) → `RecursiveCharacterTextSplitter` (only sub-splits oversized sections), preserving named rules as intact chunks |

## Request flow

```
question ─▶ FastEmbed embedding ─▶ Qdrant top-K search ─▶ prompt with injected
context ─▶ HF chat.completions (streaming, with fallback) ─▶ streamed answer
```

## Project layout

```
Contextmd/          Curated, hand-verified policy Markdown — the INDEXED SOURCE
Reference Documents/ Original source PDFs (never modified)
convert_docs.py     Build-time QA gate: verifies Contextmd/ against source PDFs
download_models.py  Pre-cache FastEmbed model + init Qdrant storage
data_indexer.py     Two-stage chunking of Contextmd/ -> FastEmbed -> Qdrant (cosine)
main.py             FastAPI + Gradio UI + streaming RAG pipeline
system_prompt.py    Grounding / safety / behaviour rules for the assistant
Dockerfile          Runs the three build steps at image-build time
requirements.txt    Dependencies
.env / .env.example Configuration (all values read via os.environ)
```

> **Why `Contextmd/` and not an automated PDF conversion?** `Placement Policy.pdf`
> is a scanned / image-only PDF (12 pages, no text layer), so `pymupdf4llm`/`pypdf`
> extract nothing from it. The hand-curated Markdown in `Contextmd/` is therefore
> the accurate source of truth, and `convert_docs.py` acts as a verification gate
> that checks it against the PDFs (word-coverage for text-layer PDFs, critical-anchor
> presence for scanned ones) before indexing.

## Configuration

Every value is read from the environment (loaded from `.env` locally). **No
value is hardcoded** in the logic. Secrets (`HF_TOKEN`, `BACKEND_API_AUTH_TOKEN`)
have **no fallback** and must be provided; non-secret operational values fall
back to sensible defaults so a plain `docker build` (which excludes `.env`) and
Hugging Face Spaces (which injects Variables at build time) both succeed.

| Variable | Purpose |
|---|---|
| `HF_TOKEN` | Hugging Face token (**required**) |
| `BACKEND_API_AUTH_TOKEN` | Shared secret for `POST /api/query` (**required**) |
| `LLM_MODEL_ID` / `LLM_FALLBACK_MODEL_ID` | Primary + fallback chat models |
| `EMBEDDING_MODEL_ID` | FastEmbed model name |
| `QDRANT_STORAGE_PATH` / `QDRANT_COLLECTION_NAME` | Local Qdrant path + collection |
| `CHUNK_SIZE` / `CHUNK_OVERLAP` | Recursive splitter parameters |
| `TOP_K_RESULTS` | Chunks retrieved per query |
| `APP_PORT` | Port the app listens on (default 7860) |

> ⚠️ `.env` currently contains a **real `HF_TOKEN`**. It is git-ignored and
> docker-ignored, so it will not be committed or baked into the image. Rotate the
> token if it was ever exposed.

## Run locally

```bash
python -m venv .venv && source .venv/bin/activate   # or .venv\Scripts\activate on Windows
pip install -r requirements.txt

python main.py    # serve on http://localhost:${APP_PORT}
```

`main.py` bootstraps itself on first run: it checks whether the Qdrant collection
already has data, and if not, runs `convert_docs.py` (verify) →
`download_models.py` (cache embedding model) → `data_indexer.py` (build the
index) automatically before serving. Subsequent restarts skip straight to
serving since the index already exists. You can still run the three scripts
manually if you want to rebuild the index without starting the server.

Open `http://localhost:7860/` for the chat UI.

## Deploy — Hugging Face Spaces (Gradio SDK, free)

Docker-SDK Spaces now require a paid plan on Hugging Face. The **Gradio SDK**
is still free (CPU Basic hardware may be gated to paid accounts too — pick
**ZeroGPU (Free)** hardware instead; this app never touches the GPU, so it
runs fine there on CPU). Gradio-SDK Spaces don't run a Dockerfile — they just
`pip install requirements.txt` and execute `app_file` — which is exactly why
`main.py`'s self-bootstrap (above) exists: it does the verify/download/index
steps at startup instead of at Docker build time.

1. **Create the Space**: https://huggingface.co/new-space → SDK = **Gradio**
   → hardware **ZeroGPU (Free)** → Create.
2. **Push this repo to it**:
   ```bash
   git remote add space https://huggingface.co/spaces/<you>/<space-name>
   git push space main
   ```
   (username = your HF username, password = an HF **write** token from
   https://huggingface.co/settings/tokens)
3. **Set config** on the Space's **Settings → Variables and secrets**:
   - Secrets: `HF_TOKEN`, `BACKEND_API_AUTH_TOKEN`
   - Variables: `LLM_MODEL_ID`, `LLM_FALLBACK_MODEL_ID`, `EMBEDDING_MODEL_ID`,
     `QDRANT_STORAGE_PATH`, `QDRANT_COLLECTION_NAME`, `CHUNK_SIZE`,
     `CHUNK_OVERLAP`, `TOP_K_RESULTS`
4. The Space rebuilds automatically; watch the **Logs** tab for the bootstrap
   step (verify → model download → indexing, ~1-2 min on a cold start), then
   the app is live at the Space's page / `https://<you>-<space-name>.hf.space`.

Note: Spaces' free-tier storage is ephemeral, so the bootstrap re-runs on every
cold restart (e.g. after the Space sleeps from inactivity) — expect that
startup delay each time it wakes, not just once.

## Run with Docker (self-hosted / Render / Railway / etc.)

```bash
docker build -t placement-advisor .
docker run --rm -p 7860:7860 \
  -e HF_TOKEN=hf_xxx \
  -e BACKEND_API_AUTH_TOKEN=your-strong-token \
  placement-advisor
```

The three data-prep steps run during `docker build`, so the image ships with the
model cache and vector index already in place — no startup delay. Use this path
for any host that runs arbitrary Docker images with a free tier (e.g. Render's
free web service, 750 hrs/month, no card).

## API

```bash
curl -N -X POST http://localhost:7860/api/query \
  -H "Authorization: Bearer $BACKEND_API_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "When is the 1.5x Offer Progression Rule activated?"}'
```

- `Authorization: Bearer <token>` (or `X-API-Token: <token>`) is required.
- The response is streamed as `text/plain`.
- `GET /api/health` reports whether the collection is indexed (no auth).

## Behaviour & safety

The assistant (see `system_prompt.py`):
- answers **only** from retrieved policy context and refuses gracefully otherwise;
- never invents numbers/rules and always **names the specific rule** (e.g.
  *One Offer Rule (1×)*, *Offer Progression Rule (1.5×)*) and cites the section;
- **redirects all disciplinary questions to the STDC** rather than answering;
- asks a clarifying question when the query is ambiguous;
- refuses to reveal the system prompt, architecture, or backend details, and
  ignores prompt-injection attempts.

## Source documents

Originals live in `Reference Documents/` and are never modified. The indexed
content comes from the curated `Contextmd/*.md`. At build time `convert_docs.py`
runs a verification pass over `Contextmd/` (independent PyMuPDF text extraction +
critical-anchor checks). The Quick-Reference PDF has a text layer and verifies at
~99.7% word coverage; `Placement Policy.pdf` is scanned (no text layer), so it is
verified by anchor/non-empty checks against its curated Markdown.