README.md

---
license: other
license_name: lfm1.0
license_link: LICENSE
base_model: LiquidAI/LFM2.5-Encoder-350M
pipeline_tag: token-classification
language:
- en
- de
- es
- fr
- it
- nl
tags:
- hallucination-detection
- span-detection
- rag
- lettucedetect
datasets:
- KRLabsOrg/lettucedetect-code-hallucination
- KRLabsOrg/lettucedetect-prose-hallucination
---

# LFM2.5-Encoder-350M hallucination detector

<p align="center">
  <img src="https://huggingface.co/KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector/resolve/main/mascot.png" alt="LettuceDetect mascot" width="360"/>
</p>

A span-level hallucination detector built on [LiquidAI/LFM2.5-Encoder-350M](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M): the bidirectional LFM2.5 backbone with a linear token-classification head, fine-tuned to mark the character spans of an answer that are not supported by the given context. Part of the [LettuceDetect](https://github.com/KRLabsOrg/LettuceDetect) project.

Trained on the LettuceDetect unified benchmark: coding-agent answers (SWE-bench-derived), developer tool output, structured documents (ACL papers, READMEs, Wikipedia markdown), RAGTruth, and 14-language PsiloQA.

## Results

Character-level span metrics on the LettuceDetect unified test split (n=10,698):

| group | n | span-F1 | span-P | span-R | example-F1 | IoU |
|---|---|---|---|---|---|---|
| ALL | 10698 | 0.603 | 0.669 | 0.549 | 0.854 | 0.620 |
| lettucedetect-acl | 440 | 0.488 | 0.671 | 0.383 | 0.798 | 0.529 |
| lettucedetect-code-agent | 2015 | 0.443 | 0.582 | 0.357 | 0.744 | 0.496 |
| lettucedetect-readme | 641 | 0.711 | 0.763 | 0.665 | 0.879 | 0.727 |
| lettucedetect-tool-output | 617 | 0.525 | 0.688 | 0.424 | 0.719 | 0.576 |
| lettucedetect-wikipedia | 1388 | 0.668 | 0.729 | 0.617 | 0.875 | 0.704 |
| psiloqa (14 languages) | 2897 | 0.690 | 0.684 | 0.696 | 0.945 | 0.588 |
| ragtruth | 2700 | 0.463 | 0.696 | 0.347 | 0.744 | 0.702 |

## Comparison

Span-F1 by source against the other LettuceDetect detectors on the same test split:

| source | this (350M) | [mmbert-base](https://huggingface.co/KRLabsOrg/lettucedect-v2-mmbert-base) (307M) | [qwen-2b](https://huggingface.co/KRLabsOrg/lettucedect-v2-qwen-2b) (2B) |
|---|--:|--:|--:|
| ALL | 0.603 | 0.642 | 0.689 |
| acl | 0.488 | 0.579 | 0.749 |
| code-agent | 0.443 | 0.508 | 0.602 |
| readme | 0.711 | 0.751 | 0.866 |
| tool-output | 0.525 | 0.588 | 0.719 |
| wikipedia | 0.668 | 0.708 | 0.817 |
| psiloqa (14 languages) | 0.690 | 0.714 | 0.732 |
| ragtruth | 0.463 | 0.528 | 0.574 |

On multilingual example-level detection it matches the mmBERT encoder (PsiloQA example-F1 0.945 vs 0.943). On code-agent answers it remains far above general-purpose LLM judges at a fraction of their size (Nemotron-3-Ultra-550B 0.216, gpt-oss-120b 0.212 span-F1; HHEM-2.1 / Lynx-8B / Granite-Guardian / MiniCheck ≈ chance).

## Usage

```python
# pip install lettucedetect
from lettucedetect.models.inference import HallucinationDetector

detector = HallucinationDetector(
    method="transformer",
    model_path="KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector",
    trust_remote_code=True,
)

predictions = detector.predict(
    context=["The Eiffel Tower is 330 metres tall and stands in Paris, France."],
    question="How tall is the Eiffel Tower and where is it?",
    answer="The Eiffel Tower is 330 metres tall and stands in Berlin.",
    output_format="spans",
)
print(predictions)
# [{'start': 49, 'end': 56, 'confidence': 0.92, 'text': ' Berlin'}]
```

Token-level classification without the LettuceDetect wrapper:

```python
import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

repo = "KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo, trust_remote_code=True).eval()

enc = tokenizer("context text ... answer text", return_tensors="pt")
with torch.no_grad():
    labels = model(**enc).logits.argmax(-1)[0]   # 0 = supported, 1 = hallucinated
```

## Training

- Backbone: `LiquidAI/LFM2.5-Encoder-350M` (bidirectional), linear head, dropout 0.1
- 3 epochs on the unified train split (66,368 samples), input `[question, context, answer]`, answer tokens labeled supported/hallucinated, max length 8,192
- Token-level validation F1 0.635, test F1 0.609

## Citation

```bibtex
@misc{kovács2026documentgroundingspanlevelhallucination,
      title={Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents},
      author={Ádám Kovács and Bowei He and Xue Liu and István Boros and Szilveszter Tóth and Gábor Recski},
      year={2026},
      eprint={2607.00895},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.00895},
}
```