Agreemind

AI-powered legal risk detection for Terms of Service.

We build models that automatically classify unfair clauses in Terms of Service documents, helping legal teams and consumers identify potentially harmful terms.

Models

All models are fine-tuned on the LexGLUE UNFAIR-ToS benchmark and evaluated on the official test set (1,607 samples) using the paper's methodology (Chalkidis et al., 2022).

LexGLUE Leaderboard comparison: Legal-BERT (paper) = 96.0 μ-F1 / 83.0 m-F1. Our top models match or exceed this.

Quick Start

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Agreemind/lexglue-roberta-unfair-tos"  # Best model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

labels = [
    "Limitation of liability", "Unilateral termination",
    "Unilateral change", "Content removal",
    "Contract by using", "Choice of law",
    "Jurisdiction", "Arbitration",
]

text = "We may terminate your account at any time without notice."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    probs = torch.sigmoid(model(**inputs).logits).squeeze()

for label, prob in sorted(zip(labels, probs), key=lambda x: x[1], reverse=True):
    if prob > 0.5:
        print(f"  {label}: {prob:.3f}")

Risk Categories

Training Methodology

• Dataset: LexGLUE UNFAIR-ToS (standard split, no augmentation)
• Loss: Standard BCEWithLogitsLoss
• LR: 3e-5 with linear schedule
• Batch size: 8
• Epochs: Up to 20 with early stopping (patience=5)
• Evaluation: Official LexGLUE test set with paper's metric computation

Links

• LexGLUE Paper
• LexGLUE Dataset

• 🌐 Website: https://agreemind.vercel.app
• 💻 GitHub: https://github.com/agree-mind

📄 License

Models and code are released under the MIT License, unless otherwise stated in individual repositories/models.