contract-extractor / docs /ACTIVATE_FINETUNED.md
myke69's picture
Add files using upload-large-folder tool
3752e5d verified
|
Raw
History Blame Contribute Delete
1.91 kB
# Activating the fine-tuned classifier (DeBERTa 0.67 / LegalBERT 0.70)
This branch (`dynamic-finetuned`) = the **dynamic baseline** + all features +
the **fine-tuned-classifier slot wired in**. It runs end-to-end **today** on
zero-shot DeBERTa (0.61, MIT). Drop in trained weights to upgrade β€” no code change.
## Right now (no weights) β€” fully working
```bash
cd backend && CLASSIFIER=zeroshot CORE_LLM_BACKEND=ollama ../.venv/bin/uvicorn app.main:app --port 8000
```
`CLASSIFIER=fusion` also works and gracefully falls back to zero-shot until
weights are present.
## To activate the fine-tuned model
1. Get the trained model folder (gitignored β€” shipped separately):
- **`deberta-sent-cuad/`** β†’ DeBERTa-v3, **0.67 macro-F1, MIT licence** ← **use this for deployment**
- `legalbert-sent-cuad/` β†’ LegalBERT, 0.70, but **CC-BY-SA** (avoid shipping)
2. Put it at `backend/finetune/<name>/`.
3. Run with the fusion classifier pointed at it:
```bash
cd backend
CLASSIFIER=fusion \
LEGALBERT_MODEL_DIR=finetune/deberta-sent-cuad \
CORE_LLM_BACKEND=ollama \
../.venv/bin/uvicorn app.main:app --port 8000
```
The fine-tuned model now owns its 12 CUAD categories; zero-shot DeBERTa fills
the rest; the dynamic baseline runs downstream. Verify with:
```bash
cd backend && CLASSIFIER=fusion LEGALBERT_MODEL_DIR=finetune/deberta-sent-cuad \
../.venv/bin/python -m eval.run_eval --classifier legalbert --limit 50
```
## To train the weights (if you don't have them)
See `docs/COLAB_FINETUNE.md` (GPU, ~15 min) β€” train with
`--model microsoft/deberta-v3-base` for the MIT/0.67 model.
## Why DeBERTa for deployment
The product ships on-prem (weights go to the client). LegalBERT is CC-BY-SA
(share-alike); DeBERTa is MIT β€” clean to sell. The ~3-point F1 gap (0.70β†’0.67)
is the price of a license you don't have to explain. See `docs/DECISIONS.md`.