legal-colbert-extractor

A small, open late-interaction (ColBERT) retriever optimized for extracting clauses from real, filed commercial contracts — assignment, termination (and its subtypes: for cause, for convenience, upon change of control…), notices, cure periods, indemnification, confidentiality, limitation of liability, governing law, and ~50 other provision subtypes. 149M parameters (ModernBERT-base backbone), 128-d token vectors, MaxSim scoring. Runs comfortably on CPU.

This is the real-contract sibling of legal-colbert-clause-retriever. The two share a lineage but are selected on different gates: the clause-retriever maximizes the public MLEB template-clause benchmark; this model maximizes retrieval quality on SEC-filed contract text with human-adjudicated relevance labels. If you are pulling clauses out of actual contracts, use this one; if you are matching against idealized clause-library text, use the sibling.

Results

Primary gate — a blind evaluation over 78 fresh SEC EDGAR Exhibit-10 contracts (830 passages, 15 clause-type queries, 171 relevance labels adjudicated independently three ways; zero overlap with any training data):

Model	Adjudicated blind NDCG@10	58-subtype discrimination
legal-colbert-extractor	0.7781	0.6968
legal-colbert-clause-retriever (sibling)	0.7683	0.6911
earlier best (P6b)	0.7642	0.6510

On MLEB Contractual Clause Retrieval it scores 0.8298 NDCG@10 (the sibling scores 0.8374 — that benchmark is the sibling's selection gate, not this model's).

Query phrasing matters

Trained on rigid clause-type queries and evaluated with descriptive definitions. Phrase queries as full descriptions of the provision you want:

• ✅ "Find contractual provisions governing termination rights, termination for cause, or the consequences of ending the agreement."
• ✅ "Find provisions related to 'Termination for Convenience'."
• ⚠️ "termination clause" — bare keywords underperform.

Queries are truncated at 48 tokens; documents at 300 tokens (chunk accordingly, ~180–220 words).

Extraction pattern (many contracts)

Index all contract chunks once, run one definition query per clause type, then group hits by document and take top-k within each document — for extraction you want the best candidate in every contract, not a global leaderboard.

from pylate import models
model = models.ColBERT("kmad00/legal-colbert-extractor")
q = model.encode(["Find provisions related to 'Termination for Cause'."], is_query=True)
d = model.encode(chunks, is_query=False)  # your contract chunks

Training

Continuation fine-tune lineage from lightonai/GTE-ModernColBERT-v1 through the clause-retriever line, then two extractor-specific rounds: (1) 62 clause subtypes mined from the full LEDGAR corpus (SEC EDGAR provisions with section-heading labels) plus FAR Part 52 public-domain clauses, (2) within-family and model-mined hard negatives (sibling-subtype confusables the incumbent model actually mis-ranked), contrastive training in very light doses (≤300 steps, lr ≤1e-7) with a CUAD anchor. Selection required improvement on the adjudicated blind gate with no MLEB regression below 0.82.

Limitations

• English-language, US-style commercial contracts.
• Multi-vector storage is heavier per document than single-vector embedders.
• Slightly weaker than the sibling on idealized template-clause text (that's the trade).
• Clause types far outside the training taxonomy get lower confidence.

License

CC BY 4.0. Training data: CUAD (CC BY 4.0), LEDGAR (SEC EDGAR public filings), ACORD (CC BY 4.0), FAR Part 52 (US public domain).