README.md

---
license: apache-2.0
library_name: sentence-transformers
pipeline_tag: sentence-similarity
base_model: sentence-transformers/LaBSE
language:
  - en
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - legal
  - legal-nlp
  - labse
  - law
---

# LeBSE — Legal-domain Sentence Embeddings (a legal-adapted LaBSE)

**LeBSE-v2** is [LaBSE](https://huggingface.co/sentence-transformers/LaBSE) fine-tuned with
**citation supervision** on U.S. case law, so legally-related opinions land closer together. It is a
drop-in `sentence-transformers` model (768-dim, same tokenizer, keeps LaBSE's multilingual base).

```python
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("ahbond/lebse")
emb = m.encode(["The district court lacked subject-matter jurisdiction over the claim."],
               normalize_embeddings=True)
```

## How it was trained
SPECTER-style contrastive fine-tuning: positive pairs are (citing opinion body, cited opinion body)
from the [CourtListener](https://www.courtlistener.com) citation graph (100,344 pairs); the rest of
the batch are negatives (`MultipleNegativesRankingLoss`). One NVIDIA A10, batch 96, `max_seq_length`
128, 2 epochs. **No case outcomes are used** — the signal is citation relatedness only.

## Evaluation (held out, opinion-disjoint)

| eval | base LaBSE | **LeBSE-v2** | Δ AUROC (95% CI) |
|------|-----------|--------------|------------------|
| citation retrieval (trained relation) | 0.765 | **0.971** | **+0.206 [+0.190, +0.223]** |
| docket-lineage (independent relation, unseen) | 0.545 | **0.562** | **+0.018 [+0.004, +0.031]** |

LeBSE-v2 dramatically improves the relatedness it was trained on and transfers a small-but-significant
amount to an **independent** legal relation (a district opinion ↔ its appellate reviewer, matched by
docket number, never trained on). It also improves embedding isotropy (anisotropy 0.570 → 0.259).

> An earlier **v1** used unsupervised SimCSE and did *not* beat base LaBSE — see the repo for that
> honest negative result. This model is v2.

## Intended use & limits
Legal opinion/paragraph retrieval, citation recommendation, clustering, and as a frozen legal-domain
feature extractor. **Not** for legal advice or case outcome decisions. U.S. federal law only;
specialized to citation-type relatedness; encodes a ~128-token paragraph, not a whole opinion.

Code, training, and evaluation: **https://github.com/ahb-sjsu/lebse** · License: Apache-2.0
(training data is U.S. federal case law, public domain).