katsubakirill/basiccontext-modernbert-base-toolace-hallucination

BasicContext_ModernBERT_base_finetuned

Fine-tuned ModernBERT token classifier for span-level hallucination detection in tool-calling dialogues.

Base Model

KRLabsOrg/lettucedect-base-modernbert-en-v1

Dataset

katsubakirill/toolace-hallucination-span-detection

Input Format

basic context input: question, tool output, answer

Only answer tokens are trained/evaluated for hallucination labels. Non-answer tokens are ignored with label -100.

Labels

• 0: non_hallucinated
• 1: hallucinated

Test Metrics

dataset_type	n_examples	span_precision	span_recall	span_f1	example_f1	exact_span_match_rate	threshold
overall	330	0.812371	0.750551	0.780238	0.787879	0.487879	0.98
contradiction_with_tool_output	130	0.290323	0.308767	0.299261	0.588235	0.461538	0.98
missing_tool	62	0.970234	0.779613	0.864541	0.9	0.596774	0.98
overgeneration	138	0.86538	0.820122	0.842143	0.888889	0.463768	0.98