cuad-cross-encoder-v10

A cross-encoder reranker fine-tuned for legal clause retrieval in contract review workflows. Built on top of cross-encoder/ms-marco-MiniLM-L-6-v2 and fine-tuned on a combination of CUAD, ACCORD, LEDGAR, ContractNLI, and EDGAR-sourced contract pairs.

Deployed as ONNX INT8 for in-browser inference via WebAssembly / ONNX Runtime Web.

Intended Use

Primary: Reranking retrieved contract chunks against natural-language clause queries (e.g. "What are the governing law provisions?", "What IP does each party retain?")
Domains covered: Joint Venture, Intellectual Property, Non-Compete / Non-Solicit, Non-Disclosure Agreement (NDA)
Not intended for: General-purpose document retrieval, non-legal domains, or as a standalone legal advisor

Training Data

Source	Description	Pairs
CUAD v1	510 contracts, 41 clause categories (Atticus Project)	~30,000
ACCORD	3,931 annotated legal passages	~6,000
LEDGAR	SEC EDGAR provisions, 14 labels filtered for JV/NC/IP	~4,000
ContractNLI / LegalBench	14 NLI tasks over contract text	~3,000
EDGAR scraped (default)	SC 13D + 8-K NC/IP exhibits, live EDGAR data	~2,500
EDGAR JV	8-K joint venture exhibit filings	~1,500
EDGAR Sino-JV	20-F chapter-format Sino-JV agreements	~4,272
Pipeline hard negatives	Clause queries where v9 failed — reranked negatives	254
Eval positives	Full-chunk positives extracted from passing eval cases	~200

Total: ~48,268 training pairs · 5,612 validation pairs

Pairs are (query, positive_chunk, negative_chunk) triplets. Negatives are a mix of hard negatives (wrong clause from same contract) and random negatives (chunks from other contracts).

Training Details

Hyperparameter	Value
Base model	`cross-encoder/ms-marco-MiniLM-L-6-v2`
Epochs	3
Batch size	32
Learning rate	2e-5
Max sequence length	512 tokens
Warmup steps	10% of total steps
Loss	Cross-entropy (sentence-transformers `CrossEncoderTrainer`)
Hardware	NVIDIA RTX 3090 / A10 (RunPod)
Training time	~45–60 min

Evaluation

Evaluated on a held-out set of 16 contracts across 4 clause domains. Each contract is queried with 3–8 clause-type questions; the top-ranked chunk is scored as pass (correct clause returned), partial (correct section but wrong chunk boundary), or fail.

Suite	Contracts	Queries	Pass	Partial	Fail
Joint Venture	9	51	9 (18%)	26 (51%)	16 (31%)
Intellectual Property	4	49	17 (35%)	20 (41%)	12 (24%)
Non-Compete / Non-Solicit	3	13	5 (38%)	8 (62%)	0 (0%)
NDA	3	19	8 (42%)	9 (47%)	2 (11%)

Test contracts (JV): MightyCell Batteries, BorrowMoney.com, Galera Therapeutics, MINDA IMPCO Technologies, Kiromic Biopharma, Novo Integrated Sciences, Transphorm / Aizu Fujitsu, Valence Technology / Baoding Fengfan, Veoneer

Test contracts (IP): Armstrong Flooring, Cerence Inc, Garrett Motion, Rare Element Resources

Test contracts (NDA): Kite Pharma / Gilead Sciences, Fortune Brands / Norcraft Companies, Aspect Medical Systems / Tyco Healthcare

Usage

ONNX Runtime (recommended for browser / edge)

import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

tokenizer = AutoTokenizer.from_pretrained("datgacon/cuad-cross-encoder-v10")
session = ort.InferenceSession("model_quantized.onnx")

query = "What governing law applies to this agreement?"
passage = "This Agreement shall be governed by and construed in accordance with the laws of the State of Delaware."

inputs = tokenizer(query, passage, return_tensors="np", max_length=512, truncation=True, padding=True)
outputs = session.run(None, {k: v for k, v in inputs.items() if k in ["input_ids", "attention_mask", "token_type_ids"]})
score = outputs[0][0][0]
print(f"Relevance score: {score:.4f}")

sentence-transformers (PyTorch)

from sentence_transformers.cross_encoder import CrossEncoder

model = CrossEncoder("datgacon/cuad-cross-encoder-v10")

query = "What governing law applies to this agreement?"
passages = [
    "This Agreement shall be governed by the laws of the State of Delaware.",
    "Each party shall maintain the confidentiality of the other party's information.",
    "The term of this Agreement shall commence on the Effective Date.",
]

scores = model.predict([(query, p) for p in passages])
ranked = sorted(zip(scores, passages), reverse=True)
for score, passage in ranked:
    print(f"{score:.4f}  {passage[:80]}")

Limitations

Trained on US commercial contracts (CUAD corpus); may underperform on EU, UK, or public-sector agreements
Partial matches are common at clause-boundary edges — chunk size and overlap in the retrieval pipeline significantly affect results
Not a legal advisor — scores indicate retrieval relevance, not legal interpretation
Performance on clause types outside the four trained domains (JV, IP, NC, NDA) is untested

Citation

If you use this model, please cite the underlying datasets:

@article{hendrycks2021cuad,
  title={CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review},
  author={Hendrycks, Dan and others},
  journal={arXiv preprint arXiv:2103.06268},
  year={2021}
}