# Team test runbook — `dynamic-finetuned`

The full product: dynamic baseline + fine-tuned classifier (fusion) + LLM
extraction + traceability + features. Here's how to run and what to verify.

## 1. Get the branch + the model
```bash
git fetch && git checkout dynamic-finetuned
# the fine-tuned weights are gitignored — drop the folder in by hand:
#   backend/finetune/legalbert-sent-cuad/   (LegalBERT 0.70, demo)
#   backend/finetune/deberta-sent-cuad/     (DeBERTa 0.67, deploy — clean licence)
```

## 2. Run it
```bash
./run.sh        # backend :8000, frontend :5173
```
Pick the classifier with env vars (backend):
| Goal | Command |
|---|---|
| Demo, most accurate | `CLASSIFIER=fusion CORE_LLM_BACKEND=ollama uvicorn app.main:app --port 8000` |
| Deploy, clean licence | `CLASSIFIER=fusion LEGALBERT_MODEL_DIR=finetune/deberta-sent-cuad uvicorn app.main:app --port 8000` |
| No models / safest | `CLASSIFIER=rules CORE_LLM_BACKEND=rules uvicorn app.main:app --port 8000` |
| LLM extraction | also: `ollama serve` + `ollama pull qwen2.5:7b` once |

`fusion` falls back to zero-shot → rules automatically if a model is missing,
so it always runs.

## 3. What to verify (the demo path)
1. Upload `data/samples/demo_msa.pdf` → register + key dates + ranked risks appear.
2. **Risks tab:** "Liability/direct damages uncapped" (sev ~85), "Auto-renewal
   90-day notice" — finding shows **"≈75th percentile of 41 real contracts"**.
3. Click any finding → the exact clause **highlights in the PDF** (traceability).
4. **Key dates:** non-renewal notice deadline computed (e.g. 2029-04-01) →
   "Add to calendar" downloads an .ics; "Excel register" downloads .xlsx.
5. Upload the **scanned** copy (`demo_msa_scanned.pdf`) → identical findings via OCR.
6. **Chat:** ask "what is the liability cap?" → grounded answer with clause cite.
7. **Compare / Heatmap / Portfolio / Defined terms** tabs render.

## 4. Measure accuracy (the slide number)
```bash
cd backend
../.venv/bin/python -m eval.run_eval --classifier legalbert --limit 50   # ~0.70 (LegalBERT)
../.venv/bin/python -m eval.run_eval --classifier zeroshot --limit 50    # 0.61 (zero-shot)
../.venv/bin/python -m eval.run_eval --classifier rules --limit 50       # 0.50 (rules)
```

## 5. Safety check (no regression)
```bash
cd backend && ../.venv/bin/python tests/test_risk_golden.py   # must print "golden test OK"
```

## Notes
- Deploy on **DeBERTa 0.67 (MIT)**, not LegalBERT (CC-BY-SA). Same code, one env var.
- The dynamic baseline is the differentiator — it works on any classifier.