README.md

---
title: Agreemind
emoji: ⚖️
colorFrom: blue
colorTo: purple
sdk: static
pinned: false
---
# Agreemind

**AI-powered legal risk detection for Terms of Service.**

We build models that automatically classify unfair clauses in Terms of Service documents, helping legal teams and consumers identify potentially harmful terms.

## Models

All models are fine-tuned on the [LexGLUE UNFAIR-ToS](https://huggingface.co/datasets/coastalcph/lex_glue) benchmark and evaluated on the official test set (1,607 samples) using the paper's methodology ([Chalkidis et al., 2022](https://arxiv.org/abs/2110.00976)).

| Model | μ-F1 | m-F1 | Speed | Best for |
|-------|------|------|-------|----------|
| [lexglue-roberta-unfair-tos](https://huggingface.co/Agreemind/lexglue-roberta-unfair-tos) | **96.1** | **84.4** | Normal | 🥇 Best accuracy |
| [lexglue-legalbert-unfair-tos](https://huggingface.co/Agreemind/lexglue-legalbert-unfair-tos) | **96.0** | **84.1** | Normal | 🥈 Legal domain |
| [lexglue-deberta-unfair-tos](https://huggingface.co/Agreemind/lexglue-deberta-unfair-tos) | 95.6 | 82.2 | Normal | General purpose |
| [lexglue-legalbert-small-unfair-tos](https://huggingface.co/Agreemind/lexglue-legalbert-small-unfair-tos) | 95.0 | 78.5 | ⚡ ~3x faster | Fast inference |

**LexGLUE Leaderboard comparison:** Legal-BERT (paper) = 96.0 μ-F1 / 83.0 m-F1. Our top models match or exceed this.

## Quick Start

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Agreemind/lexglue-roberta-unfair-tos"  # Best model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

labels = [
    "Limitation of liability", "Unilateral termination",
    "Unilateral change", "Content removal",
    "Contract by using", "Choice of law",
    "Jurisdiction", "Arbitration",
]

text = "We may terminate your account at any time without notice."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    probs = torch.sigmoid(model(**inputs).logits).squeeze()

for label, prob in sorted(zip(labels, probs), key=lambda x: x[1], reverse=True):
    if prob > 0.5:
        print(f"  {label}: {prob:.3f}")
```

## Risk Categories

| Category | Description |
|----------|-------------|
| Limitation of liability | Limits the provider's legal responsibility |
| Unilateral termination | Provider may terminate without clear cause |
| Unilateral change | Terms can change with minimal notice |
| Content removal | Provider may remove user content |
| Contract by using | Agreement implied by using the service |
| Choice of law | Specifies governing jurisdiction's law |
| Jurisdiction | Specifies where disputes are handled |
| Arbitration | Requires arbitration instead of court |

## Training Methodology

- **Dataset:** LexGLUE UNFAIR-ToS (standard split, no augmentation)
- **Loss:** Standard BCEWithLogitsLoss
- **LR:** 3e-5 with linear schedule
- **Batch size:** 8
- **Epochs:** Up to 20 with early stopping (patience=5)
- **Evaluation:** Official LexGLUE test set with paper's metric computation

## Links

- [LexGLUE Paper](https://arxiv.org/abs/2110.00976)
- [LexGLUE Dataset](https://huggingface.co/datasets/coastalcph/lex_glue)

* 🌐 Website: [https://agreemind.vercel.app](https://agreemind.vercel.app)
* 💻 GitHub: [https://github.com/agree-mind](https://github.com/agree-mind)

---

## 📄 License

Models and code are released under the **MIT License**, unless otherwise stated in individual repositories/models.