|
|
--- |
|
|
base_model: Equall/Saul-7B-Base |
|
|
library_name: peft |
|
|
license: mit |
|
|
datasets: |
|
|
- LawInformedAI/claudette_tos |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: text-classification |
|
|
--- |
|
|
|
|
|
|
|
|
# SaulLM-7B-AnomalyDetector |
|
|
|
|
|
LoRA fine-tuned adapter for [Equall/Saul-7B-Base](https://huggingface.co/Equall/Saul-7B-Base) on **anomaly detection in Terms of Service clauses**. |
|
|
The model predicts whether a clause is **anomalous/unfair** or **standard**, with Yes/No style answers and explanations. |
|
|
|
|
|
--- |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
- **Developed by:** Noshitha Juttu (University of Massachusetts Amherst, MS CS) |
|
|
- **Model type:** Causal Language Model + LoRA adapter |
|
|
- **Language(s):** English |
|
|
- **License:** MIT (Same as base model) |
|
|
- **Finetuned from:** `Equall/Saul-7B-Base` |
|
|
|
|
|
### Model Sources |
|
|
- **Repository:** [Hugging Face Repo](https://huggingface.co/Noshitha98/SaulLM-7B-AnomalyDetector) |
|
|
--- |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
|
- Detecting **unfair or anomalous clauses** in consumer Terms of Service. |
|
|
- Outputs Yes/No + brief justification. |
|
|
|
|
|
### Downstream Use |
|
|
- Integrating into **compliance tools** for legal transparency. |
|
|
- Supporting consumer advocacy research. |
|
|
|
|
|
### Out-of-Scope Use |
|
|
- Not a substitute for legal advice. |
|
|
- Not tested on contracts outside consumer ToS. |
|
|
- Should not be used as the **sole basis** for regulatory or compliance decisions. |
|
|
|
|
|
--- |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
- Trained only on Claudette ToS dataset; may not generalize. |
|
|
- Can overpredict “standard” clauses as fair (false negatives). |
|
|
- Explanations are generated text, **not guaranteed legally rigorous**. |
|
|
|
|
|
### Recommendations |
|
|
Use alongside human review and legal expertise. |
|
|
|
|
|
--- |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
from peft import PeftModel |
|
|
|
|
|
base = "Equall/Saul-7B-Base" |
|
|
adapter = "Noshitha98/SaulLM-7B-AnomalyDetector" |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(base) |
|
|
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto") |
|
|
model = PeftModel.from_pretrained(model, adapter) |
|
|
|
|
|
text = "The company reserves the right to terminate your account at any time without notice." |
|
|
inputs = tokenizer(text, return_tensors="pt").to("cuda") |
|
|
outputs = model.generate(**inputs, max_new_tokens=100) |
|
|
print(tokenizer.decode(outputs[0])) |
|
|
```` |
|
|
--- |
|
|
- **Repository:** https://github.com/Stimils02/UnfairTOSAgreementsDetection/tree/main/SaulLM7B |
|
|
|
|
|
--- |
|
|
## Training Details |
|
|
### Training Data |
|
|
- Dataset: Claudette ToS |
|
|
- Balanced: 1000 anomalous, 1000 normal clauses |
|
|
- Splits: 70% train (1400), 20% validation (400), 10% test (200) |
|
|
|
|
|
### Training Procedure |
|
|
- Quantization: 4-bit (NF4, bitsandbytes) |
|
|
- Fine-tuning: LoRA adapters applied to q_proj, k_proj, v_proj, o_proj |
|
|
- Max sequence length: 128 tokens |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
|
|
- Epochs: 3 |
|
|
- Batch size: 1 |
|
|
- Learning rate: 3e-5 |
|
|
- Optimizer: paged_adamw_32bit |
|
|
- Gradient checkpointing: enabled |
|
|
|
|
|
#### Speeds, Sizes, Times |
|
|
|
|
|
- Train runtime: ~4.3 hours (3 epochs) |
|
|
- GPU: NVIDIA Titan X (12 GB, Gypsum cluster) |
|
|
- Checkpoints: saved per epoch |
|
|
- Final adapter size: ~55 MB |
|
|
|
|
|
|
|
|
## Environmental Impact |
|
|
|
|
|
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> |
|
|
|
|
|
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
|
|
|
|
|
- **Hardware Type:** NVIDIA Titan X (12 GB) |
|
|
- **Hours used:** ~4 |
|
|
- **Provider:** UMass Gypsum HPC |
|
|
- **Carbon Estimate:** <1 kg CO₂ (low academic footprint) |
|
|
|
|
|
## Technical Specifications: |
|
|
|
|
|
### Model Architecture and Objective |
|
|
|
|
|
- Base: Saul-7B (LLaMA-style causal LM) |
|
|
- LoRA params: around 13M trainable (approx. 0.18% of total) |
|
|
|
|
|
### Compute Infrastructure |
|
|
|
|
|
- Hardware: 1x NVIDIA Titan X |
|
|
- Software: PyTorch 2.2, Transformers 4.51, PEFT 0.15.2, bitsandbytes |
|
|
|
|
|
## Glossary |
|
|
|
|
|
- **LoRA (Low-Rank Adaptation):** A parameter-efficient fine-tuning method where only small adapter matrices are trained, while the large base model remains frozen. This drastically reduces compute and storage costs. |
|
|
- **4-bit Quantization:** A compression technique that reduces model weights from 16/32-bit floating-point numbers to 4-bit representations. This allows large models (like Saul-7B) to fit and run on smaller GPUs with minimal accuracy loss. |
|
|
- **ToS (Terms of Service) Anomaly Detection:** The task of identifying clauses in service agreements that are potentially unfair, unusual, or restrictive for consumers (e.g., sudden account termination, hidden fees). |
|
|
- **PEFT (Parameter-Efficient Fine-Tuning):** A family of methods (like LoRA) that fine-tune large models by updating only a small subset of parameters instead of the entire model. |
|
|
- **Epoch:** One full pass through the training dataset during model fine-tuning. |
|
|
- **Checkpoint:** A saved state of the model during training, used for resuming training or restoring the best-performing version. |
|
|
|
|
|
## Model Card Authors |
|
|
|
|
|
- **Noshitha Juttu** – M.S. in Computer Science, University of Massachusetts Amherst |
|
|
- Research focus: NLP, model compression, On device NLP and Parameter-Efficient Fine-Tuning (PEFT). |
|
|
|
|
|
## 📚 Citation |
|
|
|
|
|
If you use this model in your research or work, please cite the following paper: |
|
|
|
|
|
> Juttu, Noshitha Padma Pratyusha. *Text to Trust: Evaluating Fine-Tuning and LoRA Trade-Offs in Language Models for Unfair Terms of Service Detection*. arXiv preprint arXiv:2510.22531, 2025. |
|
|
https://arxiv.org/abs/2510.22531 |
|
|
|
|
|
## Model Card Contact |
|
|
For questions, feedback, or collaborations, please reach out: |
|
|
- Email: noshithajuttu@gmail.com |
|
|
- LinkedIn: [Noshitha Juttu](https://www.linkedin.com/in/noshitha-juttu-0264a0167/) |
|
|
- GitHub: [Noshitha](https://github.com/Noshitha) |
|
|
- Hugging Face: [Noshitha98](https://huggingface.co/Noshitha98) |
|
|
|
|
|
|
|
|
### Framework Versions |
|
|
- **Transformers:** 4.51.3 |
|
|
- **PEFT:** 0.15.2 |
|
|
- **PyTorch:** 2.2.2 |
|
|
- **Datasets:** 2.21.0 |
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- PEFT 0.15.2 |
