validation/semantic_vs_nes/note.md

# Validation — does BioLORD cosine capture Hallmark biology?

**Setup.** A labeled benchmark over 23 of the 50 shipped Hallmark sets, grouped into
5 biological programs (cell_cycle, interferon_immune, hormone, lipid_metabolism, energy_stress). Within-program pairs are positives
(n=50); cross-program pairs among the *same labeled sets* are hard negatives (n=203).
We score every pair by the shipped BioLORD cosine and ask whether positives rank above negatives.

**Result.** **ROC-AUC = 0.890** — median same-program cosine **0.560** vs
cross-program **0.372**. BioLORD assigns systematically higher similarity to pathways
in the same biological program, even against hard negatives drawn from other coherent programs. This
is the property the semantic layer depends on: signatures built from these vectors group by biology,
not by surface lexical overlap (plan §13's "lexical false positives" concern).

**Caveat / v2.** A true external oracle (R `GOSemSim` GO:BP semantic similarity, scatter + Spearman)
is deferred to the v2 R-tools Docker Space; this is a free-tier, no-R self-consistency check on a
curated benchmark, not a GO-semantic ground truth.

![BioLORD same- vs cross-program](semantic_vs_nes.png)