metadata
title: Agreemind
emoji: ⚖️
colorFrom: blue
colorTo: purple
sdk: static
pinned: false
Agreemind
AI-powered legal risk detection for Terms of Service.
We build models that automatically classify unfair clauses in Terms of Service documents, helping legal teams and consumers identify potentially harmful terms.
Models
All models are fine-tuned on the LexGLUE UNFAIR-ToS benchmark and evaluated on the official test set (1,607 samples) using the paper's methodology (Chalkidis et al., 2022).
| Model | μ-F1 | m-F1 | Speed | Best for |
|---|---|---|---|---|
| lexglue-roberta-unfair-tos | 96.1 | 84.4 | Normal | 🥇 Best accuracy |
| lexglue-legalbert-unfair-tos | 96.0 | 84.1 | Normal | 🥈 Legal domain |
| lexglue-deberta-unfair-tos | 95.6 | 82.2 | Normal | General purpose |
| lexglue-legalbert-small-unfair-tos | 95.0 | 78.5 | ⚡ ~3x faster | Fast inference |
LexGLUE Leaderboard comparison: Legal-BERT (paper) = 96.0 μ-F1 / 83.0 m-F1. Our top models match or exceed this.
Quick Start
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_id = "Agreemind/lexglue-roberta-unfair-tos" # Best model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
labels = [
"Limitation of liability", "Unilateral termination",
"Unilateral change", "Content removal",
"Contract by using", "Choice of law",
"Jurisdiction", "Arbitration",
]
text = "We may terminate your account at any time without notice."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
probs = torch.sigmoid(model(**inputs).logits).squeeze()
for label, prob in sorted(zip(labels, probs), key=lambda x: x[1], reverse=True):
if prob > 0.5:
print(f" {label}: {prob:.3f}")
Risk Categories
| Category | Description |
|---|---|
| Limitation of liability | Limits the provider's legal responsibility |
| Unilateral termination | Provider may terminate without clear cause |
| Unilateral change | Terms can change with minimal notice |
| Content removal | Provider may remove user content |
| Contract by using | Agreement implied by using the service |
| Choice of law | Specifies governing jurisdiction's law |
| Jurisdiction | Specifies where disputes are handled |
| Arbitration | Requires arbitration instead of court |
Training Methodology
- Dataset: LexGLUE UNFAIR-ToS (standard split, no augmentation)
- Loss: Standard BCEWithLogitsLoss
- LR: 3e-5 with linear schedule
- Batch size: 8
- Epochs: Up to 20 with early stopping (patience=5)
- Evaluation: Official LexGLUE test set with paper's metric computation
Links
- 🌐 Website: https://agreemind.vercel.app
- 💻 GitHub: https://github.com/agree-mind
📄 License
Models and code are released under the MIT License, unless otherwise stated in individual repositories/models.