validation/semantic_vs_nes/note.md

Validation — does BioLORD cosine capture Hallmark biology?

Setup. A labeled benchmark over 23 of the 50 shipped Hallmark sets, grouped into 5 biological programs (cell_cycle, interferon_immune, hormone, lipid_metabolism, energy_stress). Within-program pairs are positives (n=50); cross-program pairs among the same labeled sets are hard negatives (n=203). We score every pair by the shipped BioLORD cosine and ask whether positives rank above negatives.

Result. ROC-AUC = 0.890 — median same-program cosine 0.560 vs cross-program 0.372. BioLORD assigns systematically higher similarity to pathways in the same biological program, even against hard negatives drawn from other coherent programs. This is the property the semantic layer depends on: signatures built from these vectors group by biology, not by surface lexical overlap (plan §13's "lexical false positives" concern).

Caveat / v2. A true external oracle (R GOSemSim GO:BP semantic similarity, scatter + Spearman) is deferred to the v2 R-tools Docker Space; this is a free-tier, no-R self-consistency check on a curated benchmark, not a GO-semantic ground truth.