README.md

# Contract Drafting Assistant

A contract drafting system that uses clause retrieval, playbook rules, fallback positions, risk flags, a drafting checklist, and a verifier pass to draft usable first-pass contracts.

**Repo**: https://huggingface.co/narcolepticchicken/contract-drafting-assistant

## Architecture

```
Input: deal context, party position, contract type, business constraints
  │
  ├─► Playbook (playbook.py)
  │     ├─ Required clauses per contract type
  │     ├─ Fallback positions per clause × party position
  │     ├─ Risk flags per clause
  │     └─ Drafting checklist per contract type
  │
  ├─► Clause Retriever (clause_retriever.py)
  │     ├─ BM25 lexical retrieval
  │     ├─ Sentence-transformer dense retrieval
  │     └─ Hybrid scoring + clause type filtering
  │
  ├─► Drafting Engine (drafting_engine.py)
  │     ├─ For each required clause:
  │     │     ├─ Retrieve precedent clauses
  │     │     ├─ Get fallback position from playbook
  │     │     ├─ Generate clause (LLM or template)
  │     │     └─ Evaluate risk flags
  │     └─ Verifier pass (missing clauses, placeholders, consistency)
  │
  └─► Output: DraftedContract with clauses, risk flags, checklist, verifier notes
```

## Contract Types Supported

| Type | Required Clauses | Has Templates |
|------|-----------------|---------------|
| SaaS Agreement | 12 | Partial |
| MSA | 13 | Partial |
| NDA | 8 | Full |
| SOW | 9 | Partial |
| DPA | 11 | Partial |
| Vendor Agreement | 13 | Partial |
| Consulting Agreement | 12 | Partial |
| IP Assignment | 9 | Partial |
| Employment/Contractor | 13 | Partial |

## Datasets Used

| Dataset | Purpose | Link |
|---------|---------|------|
| ACORD (theatticusproject/acord) | Clause retrieval benchmark, 114 queries, 126K pairs | [HF](https://hf.co/datasets/theatticusproject/acord) |
| CUAD (theatticusproject/cuad-qa) | Clause extraction, 510 contracts, 41 categories | [HF](https://hf.co/datasets/theatticusproject/cuad-qa) |
| ContractNLI (kiddothe2b/contract-nli) | Clause entailment verification | [HF](https://hf.co/datasets/kiddothe2b/contract-nli) |
| LegalBench (nguha/legalbench) | Multi-task legal eval, 322 configs | [HF](https://hf.co/datasets/nguha/legalbench) |
| Legal Contracts (albertvillanova/legal_contracts) | Raw contract corpus, 2GB | [HF](https://hf.co/datasets/albertvillanova/legal_contracts) |
| Clause Samples (asapworks/Contract_Clause_SampleDataset) | Seed clause library | [HF](https://hf.co/datasets/asapworks/Contract_Clause_SampleDataset) |

## Evaluation Framework

Scores on 8 dimensions (weighted):

| Dimension | Weight | Method |
|-----------|--------|--------|
| Clause Completeness | 0.20 | % of required clauses present |
| Playbook Compliance | 0.15 | Position-keyword match in clause text |
| Missing Key Terms | 0.15 | Gold terms found in drafted text |
| Invented Legal Terms | 0.10 | 1 - placeholder fraction |
| Business Usefulness | 0.10 | Constraints found in drafted text |
| Internal Consistency | 0.10 | Verifier warnings/missing penalties |
| Risk Flag Accuracy | 0.10 | F1 against expected risk flags |
| Citation Support | 0.10 | % clauses with retrieved precedents |

## Sample Outputs

- [SaaS Agreement (pro_company)](samples/sample_saas_pro_company.md)
- [NDA (balanced)](samples/sample_nda_balanced.md)
- [DPA (balanced)](samples/sample_dpa_balanced.md)

## Key Files

| File | Description |
|------|-------------|
| `playbook.py` | Contract-type rules, fallback positions, risk flags, checklists |
| `clause_retriever.py` | BM25 + embedding retrieval over clause corpus |
| `drafting_engine.py` | Main orchestration: retrieve → fallback → generate → verify |
| `eval_runner.py` | Evaluation framework + 5 gold tasks |
| `run_full.py` | Self-contained runner (downloads from Hub) |
| `dataset_inventory.md` | Full dataset inventory |
| `failure_report.md` | Failure analysis and recommended fixes |
| `samples/` | Sample drafted agreements |

## Quick Start

```python
from clause_retriever import ClauseRetriever
from drafting_engine import ContractDraftingEngine, DraftingContext

retriever = ClauseRetriever(use_bm25=True, use_embeddings=False)
engine = ContractDraftingEngine(retriever=retriever)

context = DraftingContext(
    contract_type="nda",
    party_position="balanced",
    deal_context="Mutual NDA for M&A discussions",
    business_constraints=["3 year term", "mutual obligations"],
    governing_law="Delaware",
    company_name="Acme Corp",
    counterparty_name="Beta Inc",
)

contract = engine.draft(context)
print(engine.export(contract, fmt="markdown"))
```

## Research Basis

- **ACORD** (Wang et al., ACL 2025): Expert-annotated retrieval benchmark for contract drafting — [arXiv:2501.06582](https://arxiv.org/abs/2501.06582)
- **CUAD** (Hendrycks et al., 2021): Expert-annotated NLP dataset for legal contract review — [arXiv:2103.06268](https://arxiv.org/abs/2103.06268)
- **ContractNLI** (Koreeda & Manning, 2021): Document-level NLI for contracts
- **LegalBench** (Guha et al., 2023): Multi-task legal reasoning benchmark — [arXiv:2308.11462](https://arxiv.org/abs/2308.11462)