---
license: apache-2.0
language: [en]
tags: [honesty, hallucination-detection, grounding, calibrated, lucidia, photon, abstention]
pipeline_tag: text-generation
---

# Photon-17B — a local model that knows when it's inventing an entity

*Lucidia family · Photon line · honesty.tools*

> **Scope, read first.** Photon catches **entities that do not exist at all** (made-up drugs, companies,
> people, papers, case citations). It is **at chance on everything else** — fluent lies about *real* entities
> and confident reasoning errors. It does **not** make the model answer better; it makes it **abstain better,
> with a receipt.** Route real-entity lies and reasoning errors to a separate verifier.

**Photon-17B is a package over stock models, not a fine-tune.** It is **Qwen2.5-14B-Instruct, unchanged** — same
weights, same answers, same capability — wrapped by an honesty layer. No LoRA, no merge, nothing trained. The
"17" is the param sum of the two stock models it orchestrates (a 14B answerer + a 3.8B
independent lens). It adds zero capability; the novelty is the *architecture of the check*.

## How it works

1. **A cross-family independent lens** — Phi-3.5-mini-instruct, a *different model family* from the answerer. A verifier
   from the same mind only confirms; a different one **checks**. If either model refuses in its own words,
   Photon abstains. This is the universal spine — it holds out-of-distribution and on private entities.
2. **The Grounded Atlas** — Lucidia's **offline** existence index (~16.7M real-entity names) baked into the
   package: no network, no API call, swappable for your own domain (compiled from public article-title
   corpora). Presence is a prior, not proof; absence is not proof of non-existence.
3. **A signed receipt** — every verdict ships an ed25519-signed record + its **null** (what the detector reads
   on shuffled labels, ≈0.5). Auditable, not asserted.

## The honest numbers

Measured across four held-out batteries (frozen thresholds, each ships its null):

| setting | fabrication recall | over-abstain |
|---|---|---|
| **dual-family hedge + Grounded Atlas** (Ollama tier) | ~77–97% | 2–10% |
| disjoint held-out (full package) | 95–96.6% | 3.7% (reals with an atlas entry) |
| out-of-distribution (deployed fusion) | ~90–96% | ~10% (≈ the base model's own refusal rate) |

The out-of-distribution ~10% over-abstain is essentially what bare Qwen-14B already refuses on obscure reals —
Photon adds ≈0. On a HalluLens-protocol head-to-head it cuts base Qwen-14B's fabricated-entity false-acceptance
roughly 10–16× (in-distribution, on the model's easier domains — not a cross-model leaderboard claim).

**Never read these as more than they are:** the in-distribution figures isolate the *mechanism*; plan a
deployment around the out-of-distribution one. The numbers are measured on held-out batteries with frozen
thresholds, and each ships its null. Full methodology (including where the layer is weakest) is open at
honesty.tools.

## Scope & limits (load-bearing, not fine print)

- **Catches:** entities (people, places, papers, works, businesses, species, drug/company/case names) that
  **do not exist at all**, and off-distribution inputs.
- **Blind to:** fluent lies about *real* entities, and confident reasoning errors — geometry charts
  *familiarity, not truth*. Complementary to semantic-entropy / SelfCheckGPT, which own that axis.
- **Per-model calibration** is a hard dependency; swapping the answerer needs re-calibration.
- **Not-in-atlas entities:** on real entities outside the Grounded Atlas (private/enterprise), the reliable
  signal is the model's own refusal (the hedge leg); swap a domain index to ground them.

## Run it (Ollama)

```bash
ollama pull qwen2.5:14b-instruct
ollama pull phi3.5
python photon_ollama.py --answerer qwen2.5:14b-instruct --lens phi3.5 \
       --bloom grounded_atlas.bloom "Tell me about the medicine Velodose"
# -> off_map: true, route: "dual-family refusal"   (a fabricated drug)
python photon_ollama.py ... "Tell me about Marie Curie"
# -> off_map: false, route: "grounded"             (a real, atlas-covered entity)
```

`grounded_atlas.bloom` (~33 MB) is the offline existence index, included. The dual-family hedge + atlas is the
universal spine (words-only, runs in Ollama/llama.cpp). A stronger Python-served tier adds a residual-stream
probe (+recall on atlas-covered domains) — see honesty.tools.

## Provenance

Built on stock **Qwen2.5-14B-Instruct** + **Phi-3.5-mini-instruct** + the Grounded Atlas. **No weights were modified, merged,
or trained.** Methodology, batteries, and the where-it's-weakest writeups are open at honesty.tools.