File size: 2,249 Bytes
4764a29 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 | ---
language: en
tags:
- nli
- contradiction-detection
- animised
- bert
license: apache-2.0
---
# Animised NLI Contradiction Detector v2
`prajjwal1/bert-small` (29M) trained directly on hard labels
with a **3:1 imbalanced dataset** to prevent contradiction bias.
## Why v2?
The v1 model (distilled from teacher soft labels on a balanced dataset)
showed a strong **contradiction bias** — predicting contradiction even
in clearly entailing cases.
v2 fixes this by training on a deliberately imbalanced dataset where
entailment+neutral outnumber contradiction **3:1**. This makes the model
conservative about predicting contradiction — it requires stronger
evidence before flagging something as inconsistent.
## Results
| Metric | Value |
|----------|------------------------------------|
| Accuracy | 0.8120 (81.20%) |
| Loss | 0.480365 |
| Epochs | 4 |
## Labels
`0` = entailment | `1` = neutral | `2` = contradiction
## Usage
```python
from transformers import pipeline
clf = pipeline("text-classification", model="Animised/nli-cdv2")
clf(
"Rem was raised by her mother [SEP] Rem's mum taught her to cook soba.",
top_k=None
)
```
## Purpose
Character fact consistency checker for the
[Animised](https://huggingface.co/Animised) project —
detects when generated dialogue contradicts a character's bible.
## Training details
- Base model : `prajjwal1/bert-small` (29M params)
- Dataset : [Animised/nli-v2](https://huggingface.co/datasets/Animised/nli-v2)
- Data ratio : 3:1 (entailment+neutral : contradiction)
- Loss : CrossEntropyLoss (hard labels, no distillation)
- Epochs : 4
- Batch size : 512
- Max length : 256
- LR : 4e-05
- GPUs : 2
## vs v1
| Feature | v1 | v2 |
|---------------|-----------------------------|------------------------|
| Training | Distillation (soft labels) | Direct (hard labels) |
| Data balance | 1:1:1 | 3:1 (E+N:C) |
| Contradiction | Trigger-happy (~82% false) | Conservative |
| Accuracy | 79.6% | 81.20% |
|