| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <title>Verified Knowledge Extraction from Language Models</title> |
| <meta name="description" content="Relations recovered as linear operators over language-model hidden states, gated by a 66M-edge knowledge graph, with three self-retracted results."> |
| <meta name="author" content="R. J. Thompson"> |
| <meta property="og:title" content="Verified Knowledge Extraction from Language Models"> |
| <meta property="og:description" content="Relations as operators, a knowledge graph as the gate, and three retracted results."> |
| <style> |
| html, body { margin:0; padding:0; } |
| :root { |
| --ground:#eef1f0; --surface:#f8faf9; --ink:#12171a; --muted:#5a6763; |
| --rule:#c6d0cd; --rule-soft:#dde4e2; |
| --signal:#0e6a58; --signal-bg:#dceee9; |
| --retract:#8f3120; --retract-bg:#f3e0dc; |
| --partial:#7a5a12; --partial-bg:#f1e8d3; |
| --serif: ui-serif, "Iowan Old Style", "Palatino Linotype", Palatino, Georgia, "Times New Roman", serif; |
| --sans: ui-sans-serif, system-ui, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; |
| --mono: ui-monospace, "Cascadia Mono", "SF Mono", "Roboto Mono", Menlo, Consolas, monospace; |
| --measure: 66ch; |
| } |
| @media (prefers-color-scheme: dark) { |
| :root { |
| --ground:#0d1114; --surface:#141a1d; --ink:#e3eae7; --muted:#8b9a96; |
| --rule:#2b3438; --rule-soft:#1e262a; |
| --signal:#4dbda3; --signal-bg:#10312b; |
| --retract:#d98070; --retract-bg:#331914; |
| --partial:#c9a44e; --partial-bg:#2c2415; |
| } |
| } |
| * { box-sizing:border-box; } |
| body { background:var(--ground); } |
| .page { background:var(--ground); color:var(--ink); font-family:var(--serif); font-size:17px; |
| line-height:1.62; padding:clamp(1.5rem,4vw,4rem) clamp(1rem,5vw,2rem) 6rem; |
| display:flex; flex-direction:column; align-items:center; min-height:100%; } |
| .col { width:100%; max-width:var(--measure); display:flex; flex-direction:column; gap:1.15rem; } |
| .wide { width:100%; max-width:78rem; display:flex; flex-direction:column; gap:1.15rem; } |
| .masthead { width:100%; max-width:var(--measure); display:flex; flex-direction:column; gap:1.1rem; margin-bottom:2.4rem; } |
| .kicker { font-family:var(--mono); font-size:.7rem; letter-spacing:.14em; text-transform:uppercase; |
| color:var(--muted); display:flex; flex-wrap:wrap; gap:.5rem 1.1rem; align-items:center; } |
| h1 { font-family:var(--serif); font-size:clamp(1.9rem,4.6vw,2.9rem); line-height:1.14; font-weight:600; |
| margin:0; text-wrap:balance; letter-spacing:-.015em; } |
| h1 .sub { display:block; font-size:.52em; font-weight:400; color:var(--muted); margin-top:.7rem; line-height:1.35; letter-spacing:0; } |
| .byline { font-family:var(--mono); font-size:.78rem; color:var(--muted); } |
| h2 { font-family:var(--serif); font-size:1.42rem; font-weight:600; margin:2.9rem 0 0; line-height:1.25; |
| text-wrap:balance; padding-top:1.5rem; border-top:1px solid var(--rule); } |
| h3 { font-family:var(--mono); font-size:.78rem; font-weight:600; letter-spacing:.1em; text-transform:uppercase; |
| color:var(--muted); margin:1.9rem 0 -.3rem; } |
| p { margin:0; } |
| a { color:var(--signal); text-underline-offset:.18em; } |
| a:focus-visible { outline:2px solid var(--signal); outline-offset:3px; } |
| strong { font-weight:600; } |
| .num { font-family:var(--mono); font-size:.93em; font-variant-numeric:tabular-nums; font-weight:500; } |
| .lede { background:var(--surface); border:1px solid var(--rule); padding:1.4rem 1.5rem; |
| display:flex; flex-direction:column; gap:.85rem; font-size:1.02rem; } |
| .lede .tag { font-family:var(--mono); font-size:.68rem; letter-spacing:.14em; text-transform:uppercase; color:var(--muted); } |
| .status { font-family:var(--sans); font-size:.87rem; line-height:1.55; color:var(--muted); |
| border-left:2px solid var(--rule); padding-left:1rem; } |
| .chip { display:inline-flex; align-items:center; gap:.4rem; font-family:var(--mono); font-size:.66rem; |
| font-weight:600; letter-spacing:.1em; text-transform:uppercase; padding:.2rem .55rem; |
| border:1px solid currentColor; white-space:nowrap; } |
| .chip.ok { color:var(--signal); background:var(--signal-bg); } |
| .chip.null { color:var(--retract); background:var(--retract-bg); } |
| .chip.part { color:var(--partial); background:var(--partial-bg); } |
| .tablewrap { overflow-x:auto; border:1px solid var(--rule); background:var(--surface); } |
| table { border-collapse:collapse; width:100%; font-family:var(--sans); font-size:.87rem; } |
| caption { caption-side:top; text-align:left; font-family:var(--mono); font-size:.68rem; letter-spacing:.1em; |
| text-transform:uppercase; color:var(--muted); padding:.8rem 1rem .5rem; } |
| th, td { padding:.6rem 1rem; text-align:left; border-bottom:1px solid var(--rule-soft); vertical-align:top; } |
| thead th { font-family:var(--mono); font-size:.68rem; letter-spacing:.07em; text-transform:uppercase; |
| color:var(--muted); font-weight:600; border-bottom:1px solid var(--rule); white-space:nowrap; } |
| tbody tr:last-child td { border-bottom:none; } |
| td.n { font-family:var(--mono); font-variant-numeric:tabular-nums; white-space:nowrap; } |
| td.n.hi { color:var(--signal); font-weight:600; } |
| td.n.lo { color:var(--muted); } |
| tr.dim td { color:var(--muted); } |
| .fnote { font-family:var(--sans); font-size:.79rem; color:var(--muted); line-height:1.5; } |
| .retraction { display:grid; grid-template-columns:1fr; border:1px solid var(--rule); background:var(--surface); } |
| @media (min-width:46rem) { .retraction { grid-template-columns:1fr 1fr; } } |
| .retraction > div { padding:1.05rem 1.2rem; display:flex; flex-direction:column; gap:.45rem; } |
| .retraction .was { border-bottom:1px solid var(--rule); } |
| @media (min-width:46rem) { .retraction .was { border-bottom:none; border-right:1px solid var(--rule); } } |
| .retraction .lbl { font-family:var(--mono); font-size:.65rem; letter-spacing:.12em; text-transform:uppercase; color:var(--muted); } |
| .retraction .claim { font-family:var(--sans); font-size:.92rem; } |
| .retraction .was .claim { text-decoration:line-through; text-decoration-color:var(--retract); |
| text-decoration-thickness:1.5px; color:var(--muted); } |
| .retraction .now .claim { font-weight:500; } |
| .pull { font-family:var(--serif); font-style:italic; font-size:1.15rem; line-height:1.45; |
| border-left:3px solid var(--signal); padding:.3rem 0 .3rem 1.2rem; text-wrap:balance; } |
| ul, ol { margin:0; padding-left:1.35rem; display:flex; flex-direction:column; gap:.6rem; } |
| li::marker { color:var(--muted); } |
| .foot { width:100%; max-width:var(--measure); margin-top:3.5rem; padding-top:1.4rem; |
| border-top:1px solid var(--rule); font-family:var(--sans); font-size:.82rem; color:var(--muted); |
| display:flex; flex-direction:column; gap:.6rem; } |
| .reflist { font-family:var(--sans); font-size:.82rem; color:var(--muted); line-height:1.7; } |
| </style> |
| </head> |
| <body> |
| <div class="page"> |
| <header class="masthead"> |
| <div class="kicker"><span>Preprint</span><span>·</span><span>Interpretability</span><span>·</span><span>July 2026</span></div> |
| <h1>Verified Knowledge Extraction from Language Models<span class="sub">Relations as operators, a knowledge graph as the gate, and three retracted results</span></h1> |
| <div class="byline">R. J. Thompson · independent</div> |
| </header> |
|
|
| <div class="col"> |
| <div class="lede"> |
| <div class="tag">The short version</div> |
| <p>We tried to read facts out of a language model and into a verified knowledge graph, under one rule: the model may only <em>propose</em>, and the graph must <em>independently confirm</em>.</p> |
| <p>The decoding works. Attribute relations - a person's language, a place's continent - recover at up to <span class="num">89.6%</span> top-10 against <span class="num">16,648</span> candidates, where a no-operator baseline gets <span class="num">0.0%</span>. On 12 invented entities the geometry proposed confidently and the gate committed <span class="num">0 of 72</span>.</p> |
| <p>Then we ran the check we had skipped - <em>what score does a constant guess get?</em> - and it demoted our own headline. Guessing "French" for every subject passes the same gate at <span class="num">88.9%</span>, so our <span class="num">55/55</span> was mostly base rate. Two further attempts, sparse-autoencoder features and single-neuron weight reading, each looked like clear wins and were each erased once the null searched as wide as we had.</p> |
| <p><strong>The transferable lesson: a bar that a random baseline also clears is not evidence, and the null must be as lucky as your search was.</strong></p> |
| </div> |
| <p class="status"><strong>Epistemic status:</strong> pilot scale, single model (Qwen2.5-1.5B), six relations, all bars pre-registered before scoring. The architecture claim is solid; the scale claims are not yet made. Every number here is transcribed from a pre-registered measurement report. Nothing was re-framed after seeing results.</p> |
|
|
| <h2>Three organs, strictly separated</h2> |
| <p>A knowledge graph is queryable, auditable, and silent about what it does not contain. A language model covers vastly more ground with no index, no provenance, and no way to tell recall from fabrication. The only conventional readout, generation, is exactly the operation that invents.</p> |
| <p>So the design problem is not extraction. Any method will emit confident triples. The problem is <strong>verification under adversarial self-interest, including our own.</strong> Geometry may only propose. Symbols must verify. Everything that survives carries how it is known, and everything that does not is reported as absent.</p> |
| <h3>The commit law</h3> |
| <p>One rule governs the system, derived by measurement rather than assumed: confidence is carried by <strong>edge-disjoint, provenance-distinct anchor count</strong>. In our graph one supporting path yields <span class="num">88.3%</span> precision; two disjoint paths yield <span class="num">100.0%</span>. That step is why the gate commits at two.</p> |
| <p><em>Independent</em> is the load-bearing word. Two chains through the same hub are one path wearing two hats - enforcing that cut fabrications from 19 to 2 in an earlier experiment. Two samples from the same model are likewise one source: asked to corroborate seven plausible fake entities, two passes of the same model family agreed on the same wrong answer <span class="num">7 times out of 7</span>, while the symbolic gate abstained on all seven.</p> |
| </div> |
|
|
| <div class="wide"><h2>Result I — the operator channel is real</h2></div> |
|
|
| <div class="col"> |
| <p>For six relations we sampled verified subject-object pairs from a 66.4M-edge graph (2,000 train / 250 blind held-out each, split by subject-name hash), encoded through Qwen2.5-1.5B. From the training pairs we estimated an additive offset and a ridge affine map, then decoded held-out subjects by nearest neighbour against a 16,648-entity bank.</p> |
| <p>The control that matters is the <strong>identity baseline</strong>: decode the subject's own state with no operator applied. If proximity alone explained the result, it would score well.</p> |
| </div> |
|
|
| <div class="wide"> |
| <div class="tablewrap"><table> |
| <caption>Blind held-out, n=250 per relation, 16,648 distractors</caption> |
| <thead><tr><th>Relation</th><th>Operator top-1</th><th>Operator top-10</th><th>No-operator baseline</th><th>Class</th></tr></thead> |
| <tbody> |
| <tr><td>native language</td><td class="n">65.2%</td><td class="n hi">89.6%</td><td class="n lo">0.0%</td><td>attribute</td></tr> |
| <tr><td>continent</td><td class="n">37.2%</td><td class="n hi">79.2%</td><td class="n lo">0.0%</td><td>attribute</td></tr> |
| <tr><td>country of citizenship</td><td class="n">39.6%</td><td class="n hi">73.2%</td><td class="n lo">0.0%</td><td>attribute</td></tr> |
| <tr><td>country</td><td class="n">23.2%</td><td class="n hi">54.8%</td><td class="n lo">0.4%</td><td>attribute</td></tr> |
| <tr class="dim"><td>capital</td><td class="n">16.8%</td><td class="n">24.4%</td><td class="n">22.4%</td><td>identity</td></tr> |
| <tr class="dim"><td>headquarters</td><td class="n">~15%</td><td class="n">~25%</td><td class="n">~23%</td><td>identity</td></tr> |
| </tbody></table></div> |
| <p class="fnote">Best cell shown per relation. 39 of 72 measured cells cleared the pre-declared bar of 30% top-10 with a 10-point margin over baseline.</p> |
| </div> |
|
|
| <div class="col"> |
| <h3>The wall that reproduced across architectures</h3> |
| <p>The bottom two rows are the interesting failure. Relations whose objects are <strong>attributes or types</strong> form a strong channel. Relations whose objects are <strong>specific individuals</strong> drawn from a large set do not - the operator adds roughly three points over raw proximity. We had already measured that same ceiling in a different embedding space with different machinery. Finding it again inside a language model's own contextual space suggests it is a property of the representation class:</p> |
| <p class="pull">Geometry carries kinds; it does not carry individuals. Symbols must own identity.</p> |
| <p>We treat that as a routing rule rather than an obstacle. Attribute questions go to the geometric channel, which is cheap and batched at 399 entities per second. Identity questions go to the graph. A related result supports the same split: decoding operator output through the model's own vocabulary was essentially dead at <span class="num">≤2%</span>, so extraction is inherently <strong>closed-world</strong> - the graph supplies the vocabulary of possible answers, the model supplies only the geometry that selects among them.</p> |
| <h3>The gate, tested where it can fail</h3> |
| <p>Corroboration ran with the target relation <strong>masked across the entire graph</strong>, including its inverse. The gate cannot verify a proposal by looking the answer up, only by assembling independent evidence from other relations.</p> |
| </div> |
|
|
| <div class="wide"> |
| <div class="tablewrap"><table> |
| <thead><tr><th>Test</th><th>Result</th><th>Verdict</th></tr></thead> |
| <tbody> |
| <tr><td>Strongest relation (native language)</td><td>55 / 55 commits correct</td><td><span class="chip null">Retracted below</span></td></tr> |
| <tr><td>Pooled across all six relations</td><td>93 / 108 (86.1%) at 7.2% coverage, below the 95% bar</td><td><span class="chip part">Partial</span></td></tr> |
| <tr><td>12 invented entities × 6 relations</td><td><strong>0 / 72 commits.</strong> The geometry proposed confidently for every fake.</td><td><span class="chip ok">Holds</span></td></tr> |
| </tbody></table></div> |
| </div> |
|
|
| <div class="col"> |
| <p>The pooled failures are not random. They are <em>type-confusions verified by association</em>: proposing "Thai" for a language slot attracts hundreds of converging anchors, because Thai and Thailand are genuinely related. The gate as built verifies <strong>relatedness</strong>. The missing constraint is that the committed object must be of the <strong>type the relation demands</strong>. That constraint was available in the graph and deliberately <em>not</em> applied after seeing the results, since fixing a bar post-hoc invalidates it.</p> |
| <h2>Correcting our own headline</h2> |
| <p>A follow-up applied the missing type constraint, and more importantly ran the null arm the original probe had not: what does a <strong>constant guess</strong> achieve through the same gate?</p> |
| <p>The type constraint worked as predicted - pooled precision rose from <span class="num">86.1%</span> to <span class="num">97.1%</span>, eliminating 13 of 15 wrong commits at a 35% coverage cost. But the null arms corrected us.</p> |
| </div> |
|
|
| <div class="wide"> |
| <div class="retraction"> |
| <div class="was"><div class="lbl">What we reported</div><div class="claim">Native language: 55 / 55 gate commits correct, 100% precision under masked corroboration.</div></div> |
| <div class="now"><div class="lbl">What the matched null showed</div><div class="claim">Proposing one fixed object for <em>every</em> subject - "French" - passes the same gate 63 times at <strong>88.9%</strong>. "United States" for citizenship commits 86 times at 66.3%. The gate cannot refuse a high-base-rate object, so the figure was <strong>substantially base rate, not earned</strong>. Under pre-declared bars, <strong>0 of 6</strong> relations passed once these nulls were in place.</div></div> |
| </div> |
| </div> |
|
|
| <div class="col"> |
| <p>A real channel survives underneath. Excluding high-base-rate objects and sweeping 62,000 subjects produced <strong>696 residual extractions</strong> with diverse objects, against a wrong-subject null of 32 and a random-direction null of 1 - beating its nulls by roughly <span class="num">22×</span>. Graded externally, 107 of 400 sampled rows were gradeable at all (73% are tail entities no external source covers) and those scored <span class="num">90.7%</span>. In the reverse direction, 48 disagreements with the graph were hand-classified: 26 graph-right, 18 entity-linking artifacts, and <strong>2 genuine graph errors the model caught</strong>. Against the pre-declared bar this is partial, so the artifact ships exploratory and is not promoted into the graph.</p> |
| <p class="pull">The same discipline that produced our best number is what took it back. A headline that cannot survive a constant-guess baseline was never a measurement of the method.</p> |
|
|
| <h2>Result II — a negative result, reported first-class</h2> |
| <p>Sparse autoencoders decompose activations into features. The standing weakness of that literature is that features get labeled by asking a language model what they seem to mean, which is unverified and circular. Our corpus aligns text to verified triples, so for any context we already know which entities and classes are present. That turns feature labeling into a measurement against external ground truth.</p> |
| <p>We trained a top-k sparse autoencoder (24,576 features, k=32) on 250,000 activation positions. It reconstructed well - <span class="num">96.5%</span> of variance explained against a 60% floor, 0.07% dead features - and under our first grounding bench it labeled <strong>41 features</strong> at ≥90% held-out precision. That number is wrong, and our own controls said so.</p> |
| </div> |
|
|
| <div class="wide"> |
| <div class="tablewrap"><table> |
| <thead><tr><th>Control arm</th><th>First bench</th><th>Corrected bench</th></tr></thead> |
| <tbody> |
| <tr><td>Sparse autoencoder</td><td class="n">41 grounded</td><td class="n">0 grounded / 121 candidate</td></tr> |
| <tr><td><strong>Random directions</strong></td><td class="n" style="color:var(--retract);font-weight:600">23 grounded</td><td class="n">0 / 0</td></tr> |
| <tr><td><strong>Shuffled labels</strong></td><td class="n" style="color:var(--retract);font-weight:600">5 grounded</td><td class="n">0 / 70 candidate</td></tr> |
| </tbody></table></div> |
| </div> |
|
|
| <div class="col"> |
| <p>The mechanism of the false positive generalises, which is why it is reported in detail. One class was 63% of the evaluation data and nearly linearly separable in raw activations; a threshold test on a dominant, geometrically loud class is easy to pass by accident. Cosine against an unnormalised class centroid mostly measures <em>whether this is an entity mention at all</em>, not which class. And running 24,576 candidates against a 5% false-pass threshold buys roughly 1,200 free passes. Two weak correlational tests conjoined still admit chance.</p> |
| <p>The corrected bench subtracts the shared mention direction, replaces the significance threshold with a <strong>selection-matched null</strong> (beat the best of 24,576 random directions), and requires a <strong>margin</strong> over every competing class. Random directions then ground nothing at all. The pre-declared retry at half the sparsity budget changed nothing: zero grounded, retry delta zero. Diagnosis: roughly eight scoreable activations per feature. Starvation, not refutation.</p> |
| <p class="pull">The measurable claim of this work is not that our extraction works. It is that when it does not work, our instruments say so before we do.</p> |
|
|
| <h2>A third path, and a third null</h2> |
| <p>Weights are the cheapest access of all, requiring no forward pass. Feed-forward layers behave as key-value memories, so a value vector can be decoded through the output embedding to ask what a neuron writes. We scanned every such neuron for a target vocabulary of programming idioms, on CPU, in about seven minutes.</p> |
| <p>Against the pre-declared bar it <strong>passed on all 13 targets</strong> - and it is still a null. The bar compared against 500 random draws, which scored zero, making every ratio infinite. But the search ranged over roughly 396,000 candidate directions. Running the null <em>as wide as the search</em> collapses the effect to <span class="num">0.7×</span>-<span class="num">1.8×</span>: real neurons hit idiom tokens at essentially the random rate, and random directions achieve rank 1 on 11 of 13 targets, matching the real ones.</p> |
| <p>One structural finding survives regardless. The idiom we searched for tokenises as a <strong>three-token sequence</strong>, so it cannot be stored in one neuron even in principle. The knowledge is a sequence of associations, not an address.</p> |
|
|
| <h2>The pattern: calibration is the whole game</h2> |
| <p>Three independent attempts to read knowledge out of a model each produced an apparently strong result, and each was retracted by a null matched to the actual selection pressure.</p> |
| </div> |
|
|
| <div class="wide"> |
| <div class="tablewrap"><table> |
| <thead><tr><th>Attempt</th><th>Apparent result</th><th>After a matched null</th></tr></thead> |
| <tbody> |
| <tr><td>SAE features</td><td>41 grounded features</td><td>0 - a random dictionary grounded 23 under the same bar</td></tr> |
| <tr><td>Single-neuron weight reading</td><td>13 of 13 targets pass</td><td>0.7×-1.8× - the null searched 500 draws against a 396,000-wide search</td></tr> |
| <tr><td>Two-channel retrieval union</td><td>regressions fixed, recall up</td><td>1.5× - having a second channel did the work, not the new one</td></tr> |
| </tbody></table></div> |
| </div> |
|
|
| <div class="col"> |
| <p class="pull">A bar that a random baseline can also pass is not evidence, however many candidates clear it. The null must search as wide as the claim.</p> |
| <p>This is the most transferable result in the work. Each of the three would have been publishable as a positive finding under a conventionally-specified control. What distinguishes the surviving positives is not that they were larger - it is that they were put through nulls built to match how hard we had searched. One of those nulls, applied late, correctly demoted our own best number.</p> |
|
|
| <h2>What this is for</h2> |
| <ol> |
| <li><strong>Verified knowledge-base growth and repair.</strong> Extraction is batched matrix arithmetic, not autoregressive generation - a 100,000-entity sweep costs about four minutes per template. The output is bidirectional: facts the model holds that the graph lacks, and facts the graph holds that the model contradicts. In an earlier hybrid measurement the graph overrode the model 11 times and was right 10.</li> |
| <li><strong>Abstention before generation.</strong> The fabrication test inverts into a guardrail. Confident geometric proposal with zero independent corroboration is the signature of a question that should be refused rather than answered, with a measured zero-fabrication property behind it rather than a heuristic.</li> |
| <li><strong>Knowledge inventory as a model audit.</strong> Per model, a map of which relations have a strong channel and which do not - the weak ones being where that model is most likely to confabulate. Model cards report benchmarks; this reports a verified factual inventory.</li> |
| <li><strong>Program synthesis with zero weights</strong>, which shows the same complementarity. A sibling system synthesises Python from a task description using no neural weights at all. Under MBPP+ deflation it reaches <span class="num">136 of 224</span> covered tasks (<span class="num">60.7%</span>). On the identical pool, identical spec, identical grader, Qwen2.5-Coder-7B scores <span class="num">161/224</span> - and that flatters it, since MBPP is in its pretraining. But the <strong>union is 186/224 (83.0%)</strong>, and <strong>25 tasks are solved only by the zero-weight system</strong>. Uncorrelated errors plus a verifier that can tell which channel to keep.</li> |
| </ol> |
|
|
| <h2>Honest limits</h2> |
| <ul> |
| <li><strong>This does not become a language model.</strong> It is a knowledge and verification engine: closed-world, auditable, silent where it cannot verify.</li> |
| <li><strong>Identity stays walled</strong>, reproduced across two embedding spaces and three mechanisms.</li> |
| <li><strong>Coverage has a principled floor.</strong> An entity with fewer than two independent facts cannot be verified at any depth. Measured: zero recovery on 69 such cases, 59 of which had at most one fact in total. Depth does not manufacture independence.</li> |
| <li><strong>Relation linearity is not our discovery.</strong> It is established in prior work. The contribution is the gated architecture around it.</li> |
| <li><strong>The feature and single-neuron rungs are null</strong> at the scales tested.</li> |
| <li><strong>Base rates can masquerade as verification.</strong> A gate cannot refuse a high-frequency object; constant-guess baselines must be run per relation. Our own strongest figure was demoted by exactly this.</li> |
| <li><strong>Scale.</strong> One 1.5B model, six relations, one pilot-scale autoencoder. The throughput figures make broader sweeps cheap; they have not yet been run.</li> |
| </ul> |
|
|
| <h2>Method notes</h2> |
| <p>The protocol is part of the contribution, since the failure mode of this area is not bad ideas but unfalsifiable ones. Bars were written before results, and the pooled 86.1% gate figure is reported as a partial failure rather than re-framed, with the identified fix withheld from the run. Adversarial controls were mandatory, and killed a 41-feature result that would otherwise have been reported as a success. Splits are blind by entity. Corroboration is masked, so the gate cannot verify by lookup. New code received an independent adversarial audit, which found a commit path that could emit code contradicting a user's own stated example; it was fixed before the result was reported. Negative results are ledgered so that dead levers stay dead.</p> |
|
|
| <h2>Selected references</h2> |
| <p class="reflist"> |
| Hernandez et al., <em>Linearity of Relation Decoding in Transformer Language Models</em>, ICLR 2024 (arXiv:2308.09124). · |
| Merullo, Eickhoff, Pavlick, <em>Language Models Implement Simple Word2Vec-style Vector Arithmetic</em>, NAACL 2024 (arXiv:2305.16130). · |
| Geva et al., <em>Transformer Feed-Forward Layers Are Key-Value Memories</em>, EMNLP 2021 (arXiv:2012.14913). · |
| Gao et al., <em>Scaling and Evaluating Sparse Autoencoders</em>, 2024 (arXiv:2406.04093). · |
| Meng et al., <em>Locating and Editing Factual Associations in GPT</em>, NeurIPS 2022 (arXiv:2202.05262). · |
| Cunningham et al., <em>Sparse Autoencoders Find Highly Interpretable Features</em>, ICLR 2024 (arXiv:2309.08600). · |
| Park, Choe, Veitch, <em>The Linear Representation Hypothesis</em>, ICML 2024 (arXiv:2311.03658). · |
| Petroni et al., <em>Language Models as Knowledge Bases?</em>, EMNLP 2019 (arXiv:1909.01066). |
| </p> |
| </div> |
|
|
| <div class="foot"> |
| <p>Full PDF with the complete result grids and reference list available. Pre-registered measurement reports and code available on request.</p> |
| <p>Correspondence: rj45thompson@gmail.com</p> |
| </div> |
| </div> |
| </body> |
| </html> |
|
|
