README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: text-classification
language:
  - en
tags:
  - grounding
  - hallucination-detection
  - fact-verification
  - nli
  - zero-shot-classification
  - document-ai
  - cross-encoder
datasets:
  - nutrientdocs/grounding-benchmark
metrics:
  - roc_auc
---

# grounding-en

**Does the document actually support this claim?** `grounding-en` is a cross-encoder that scores whether
a hypothesis (a number, date, or fact) is **entailed by** a premise drawn from a real document — a
financial table, a filing, prose evidence.

It is the open, English member of Nutrient's grounding model family.

- 🎯 **Try it:** [grounding-demo](https://huggingface.co/spaces/nutrientdocs/grounding-demo?model=en)
- 🏆 **Leaderboard:** [grounding-leaderboard](https://huggingface.co/spaces/nutrientdocs/grounding-leaderboard)
- 📊 **Benchmark:** [grounding-benchmark](https://huggingface.co/datasets/nutrientdocs/grounding-benchmark)

## Results

On the held-out English [grounding-benchmark](https://huggingface.co/datasets/nutrientdocs/grounding-benchmark)
(ROC-AUC), against the strongest open English NLI models:

| Facet | `grounding-en` | `grounding-multilingual` | DeBERTa-v3-large zero-shot | DeBERTa-v3-large MNLI/FEVER/ANLI | BART-large MNLI |
| --- | ---: | ---: | ---: | ---: | ---: |
| **Overall** | **.882** | .925 | .786 | .769 | .636 |
| Number | **.923** | .969 | .658 | .642 | .478 |
| Date | **.998** | .999 | .995 | .995 | .924 |
| String | **.955** | .949 | .941 | .913 | .757 |
| Table premises | **.863** | .915 | .766 | .747 | .611 |
| Prose premises | **.964** | .970 | .929 | .945 | .892 |

`grounding-en` leads the field on the hard axis — **number grounding .92** vs .48–.66 for general-purpose
NLI models — while matching or beating them everywhere else. Full ranking on the
[leaderboard](https://huggingface.co/spaces/nutrientdocs/grounding-leaderboard). The commercial sibling
[`grounding-multilingual`](https://huggingface.co/nutrientdocs/grounding-multilingual) scores a touch
higher again and covers 15+ languages.

## Usage

```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

m = "nutrientdocs/grounding-en"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForSequenceClassification.from_pretrained(m).eval()

premise = "Revenue | 2023 | $4,213M\nRevenue | 2022 | $3,905M"
hypothesis = "2023 revenue was $4.2 billion."

enc = tok(premise, hypothesis, truncation=True, max_length=1024, return_tensors="pt")
with torch.no_grad():
    probs = torch.softmax(model(**enc).logits, dim=-1)[0]
p_support = probs[0].item()   # entailment is class index 0 (id2label = {0: entailment, 1: not_entailment})
print(f"grounded support = {p_support:.3f}")
```

An **ONNX** export is provided under [`onnx/`](./onnx) for on-device / ONNX Runtime deployment.

## Calibrating the score

Fine-tuning maximizes _ranking_ (AUC), which tends to make the raw probability overconfident. For a
score you can gate on ("0.9 means ~90% right"), apply **temperature scaling** — divide the logits by a
fitted `T` before softmax. It's monotonic, so it leaves AUC/ranking untouched and only repairs the
confidence values. On the serving distribution we fit **T = 1.29** (ECE 0.028 → 0.009). Re-fit `T` on
_your_ distribution whenever your input pipeline changes.

## Intended use & limits

- **Use it for:** verifying extracted values against source documents, hallucination/citation checking,
  routing low-confidence extractions for review.
- **Limits:** English only. The remaining ceiling is _reasoning_ table-claim negatives and
  multi-step arithmetic. As a dedicated grounding model it trades a little general-NLI accuracy for
  grounding.

## License & training data

Weights are **Apache-2.0**. Trained on a multi-corpus grounding set. The public,
redistributable slice of the _evaluation_ data is
[grounding-benchmark](https://huggingface.co/datasets/nutrientdocs/grounding-benchmark) (CC-BY-SA-4.0);
the full training set is not redistributed.

## About the author

<a href="https://nutrient.io/">
  <img src="https://avatars2.githubusercontent.com/u/1527679?v=3&s=200" height="80" />
</a>

This project is maintained and funded by [Nutrient](https://nutrient.io/) - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.