docs/TEAM_TEST.md

Team test runbook — dynamic-finetuned

The full product: dynamic baseline + fine-tuned classifier (fusion) + LLM extraction + traceability + features. Here's how to run and what to verify.

1. Get the branch + the model

git fetch && git checkout dynamic-finetuned
# the fine-tuned weights are gitignored — drop the folder in by hand:
#   backend/finetune/legalbert-sent-cuad/   (LegalBERT 0.70, demo)
#   backend/finetune/deberta-sent-cuad/     (DeBERTa 0.67, deploy — clean licence)

2. Run it

./run.sh        # backend :8000, frontend :5173

Pick the classifier with env vars (backend):

Goal	Command
Demo, most accurate	CLASSIFIER=fusion CORE_LLM_BACKEND=ollama uvicorn app.main:app --port 8000
Deploy, clean licence	CLASSIFIER=fusion LEGALBERT_MODEL_DIR=finetune/deberta-sent-cuad uvicorn app.main:app --port 8000
No models / safest	CLASSIFIER=rules CORE_LLM_BACKEND=rules uvicorn app.main:app --port 8000
LLM extraction	also: ollama serve + ollama pull qwen2.5:7b once

fusion falls back to zero-shot → rules automatically if a model is missing, so it always runs.

3. What to verify (the demo path)

1. Upload data/samples/demo_msa.pdf → register + key dates + ranked risks appear.
2. Risks tab: "Liability/direct damages uncapped" (sev ~85), "Auto-renewal 90-day notice" — finding shows "≈75th percentile of 41 real contracts".
3. Click any finding → the exact clause highlights in the PDF (traceability).
4. Key dates: non-renewal notice deadline computed (e.g. 2029-04-01) → "Add to calendar" downloads an .ics; "Excel register" downloads .xlsx.
5. Upload the scanned copy (demo_msa_scanned.pdf) → identical findings via OCR.
6. Chat: ask "what is the liability cap?" → grounded answer with clause cite.
7. Compare / Heatmap / Portfolio / Defined terms tabs render.

4. Measure accuracy (the slide number)

cd backend
../.venv/bin/python -m eval.run_eval --classifier legalbert --limit 50   # ~0.70 (LegalBERT)
../.venv/bin/python -m eval.run_eval --classifier zeroshot --limit 50    # 0.61 (zero-shot)
../.venv/bin/python -m eval.run_eval --classifier rules --limit 50       # 0.50 (rules)

5. Safety check (no regression)

cd backend && ../.venv/bin/python tests/test_risk_golden.py   # must print "golden test OK"

Notes

• Deploy on DeBERTa 0.67 (MIT), not LegalBERT (CC-BY-SA). Same code, one env var.
• The dynamic baseline is the differentiator — it works on any classifier.