chicham/artefactual-wepr-phi4

Model description

A calibrated WEPR hallucination detector for responses generated by microsoft/phi-4.

WEPR (Weighted EPR) keeps the ranks separate, giving the calibration one coefficient per rank (mean and max over the token axis, so 2k features). It reads strictly more of the distribution than EPR at the same calibration cost.

The artifact is the fitted LogisticRegression alone. The feature extraction that feeds it -- parsing top-15 log-probabilities out of a completion response and reducing them to entropy features -- lives in the artefactual library, so this file contains no custom classes and loads with an empty trusted list.

Introduced in Beyond Sampling: Efficient Hallucination Detection (ECIR 2026).

Intended uses & limitations

Scores a response on [0, 1], where 1 is the hallucination class.

• Tied to microsoft/phi-4. The coefficients are fit against that model's output distribution. Scoring another model's responses with them is not meaningful, even though nothing in the file prevents it.
• Fixed at k=15. Responses must be generated with logprobs=True and top_logprobs=15. Fewer ranks are rejected rather than zero-filled, because the missing ranks are unfetched rather than absent and padding them would score the response as more confident than it was.
• No published operating point. The paper reports ROC-AUC and PR-AUC, both threshold-free, so no decision threshold is published. Choose one on your own labelled data.

Evaluation Results

See the paper. It reports ROC-AUC and PR-AUC across the evaluated models; no figures are restated here so that this card cannot drift from the published results.

How to Get Started with the Model

from artefactual.scoring import wepr

detector = wepr("chicham/artefactual-wepr-phi4")
scores = detector.predict_proba(response)[:, 1]

response is an OpenAI-compatible chat completion or responses payload carrying top_logprobs=15.

Model Card Authors

Artefact Research Center

Model Card Contact

https://github.com/artefactory/artefactual/issues

Citation

@inproceedings{artefactual2026,
  title     = {Beyond Sampling: Efficient Hallucination Detection},
  booktitle = {ECIR},
  year      = {2026},
  eprint    = {2509.04492}
}