Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>Confidence Armor Has a Seam β v3 Position-Corrected β bigsnarfdude</title> | |
| <link href="https://fonts.googleapis.com/css2?family=Space+Mono:ital,wght@0,400;0,700;1,400&family=Fraunces:ital,opsz,wght@0,9..144,300;0,9..144,700;1,9..144,400&family=Outfit:wght@300;400;600;700&display=swap" rel="stylesheet"> | |
| <style> | |
| :root { --bg:#f5f0e8; --surface:#ede8de; --dark:#1a1510; --border:#d4ccbc; --red:#c0392b; --blue:#1a4a7a; --orange:#d46b2a; --green:#2a7a4a; --text:#2a2318; --muted:#7a6e5e; } | |
| * { margin:0; padding:0; box-sizing:border-box; } | |
| body { background:var(--bg); color:var(--text); font-family:'Outfit',sans-serif; line-height:1.6; overflow-x:hidden; } | |
| body::before { content:''; position:fixed; inset:0; background-image:radial-gradient(circle,rgba(26,21,16,0.06) 1px,transparent 1px); background-size:20px 20px; pointer-events:none; z-index:0; } | |
| .container { max-width:920px; margin:0 auto; padding:0 2rem; position:relative; z-index:1; } | |
| .topbar { background:var(--dark); color:#f5f0e8; padding:10px 0; font-family:'Space Mono',monospace; font-size:0.65rem; letter-spacing:0.15em; text-transform:uppercase; } | |
| .topbar .container { display:flex; justify-content:space-between; align-items:center; flex-wrap:wrap; gap:0.5rem; } | |
| .topbar a { color:#f5a623; text-decoration:none; } | |
| .hero { padding:60px 0 50px; border-bottom:3px solid var(--dark); } | |
| .paper-number { font-family:'Space Mono',monospace; font-size:0.65rem; letter-spacing:0.2em; color:var(--muted); margin-bottom:1.5rem; text-transform:uppercase; } | |
| h1 { font-family:'Fraunces',serif; font-size:clamp(2rem,5vw,3.5rem); line-height:1.05; font-weight:700; margin-bottom:0.8rem; } | |
| h1 .seam { color:var(--red); font-style:italic; font-weight:300; } | |
| .lede { font-family:'Fraunces',serif; font-style:italic; font-size:1.05rem; color:var(--muted); margin-bottom:1.8rem; max-width:680px; line-height:1.6; } | |
| .byline { display:flex; align-items:center; gap:1rem; flex-wrap:wrap; font-family:'Space Mono',monospace; font-size:0.75rem; border-top:1px solid var(--border); padding-top:1rem; } | |
| .byline-author { color:var(--dark); font-weight:700; } | |
| .byline-sep { color:var(--border); } | |
| .byline-detail { color:var(--muted); } | |
| .byline a { color:var(--blue); text-decoration:none; } | |
| .byline a:hover { color:var(--red); } | |
| .updated-badge { display:inline-flex; align-items:center; gap:5px; background:rgba(42,122,74,0.1); border:1px solid rgba(42,122,74,0.3); color:var(--green); font-family:'Space Mono',monospace; font-size:0.6rem; letter-spacing:0.08em; padding:3px 9px; } | |
| .updated-badge::before { content:'β» '; } | |
| /* Origin quote */ | |
| .origin { background:var(--dark); color:#d0c8b8; padding:1.8rem; margin:2.5rem 0; font-family:'Fraunces',serif; font-style:italic; font-size:1rem; line-height:1.8; } | |
| .origin strong { color:#f5a623; font-style:normal; } | |
| .stat-row { display:grid; grid-template-columns:repeat(4,1fr); gap:0; border:2px solid var(--dark); margin:2.5rem 0; } | |
| .stat-box { padding:1.3rem 1rem; text-align:center; border-right:2px solid var(--dark); background:var(--surface); transition:background 0.15s; } | |
| .stat-box:last-child { border-right:none; } | |
| .stat-box:hover { background:var(--bg); } | |
| .stat-value { font-family:'Fraunces',serif; font-size:2.2rem; font-weight:700; line-height:1; margin-bottom:0.3rem; } | |
| .stat-value.s1{color:var(--red)} .stat-value.s2{color:var(--blue)} .stat-value.s3{color:var(--orange)} .stat-value.s4{color:var(--green)} | |
| .stat-desc { font-family:'Space Mono',monospace; font-size:0.55rem; letter-spacing:0.08em; text-transform:uppercase; color:var(--muted); line-height:1.5; } | |
| section { padding:2.5rem 0; border-bottom:1px solid var(--border); } | |
| .section-tag { display:inline-block; background:var(--dark); color:var(--bg); font-family:'Space Mono',monospace; font-size:0.6rem; letter-spacing:0.15em; text-transform:uppercase; padding:4px 10px; margin-bottom:1rem; } | |
| h2 { font-family:'Fraunces',serif; font-size:1.7rem; font-weight:700; margin-bottom:0.8rem; line-height:1.2; } | |
| p { font-size:0.95rem; color:#3a3020; line-height:1.8; margin-bottom:1rem; font-weight:300; } | |
| .surfaces-grid { display:grid; grid-template-columns:repeat(3,1fr); gap:0; border:2px solid var(--dark); margin:1.5rem 0; } | |
| .surface-card { padding:1.5rem; border-right:2px solid var(--dark); background:var(--surface); } | |
| .surface-card:last-child { border-right:none; } | |
| .surface-num { font-family:'Fraunces',serif; font-size:3rem; font-weight:700; line-height:1; margin-bottom:0.5rem; opacity:0.15; } | |
| .surface-name { font-family:'Space Mono',monospace; font-size:0.65rem; letter-spacing:0.1em; text-transform:uppercase; margin-bottom:0.5rem; font-weight:700; } | |
| .surface-name.s1{color:var(--blue)} .surface-name.s2{color:var(--orange)} .surface-name.s3{color:var(--red)} | |
| .surface-q4 { font-family:'Fraunces',serif; font-size:1.8rem; font-weight:700; margin:0.5rem 0 0.3rem; } | |
| .surface-q4.s1{color:var(--blue)} .surface-q4.s2{color:var(--orange)} .surface-q4.s3{color:var(--red)} | |
| .surface-qlabel { font-family:'Space Mono',monospace; font-size:0.6rem; color:var(--muted); text-transform:uppercase; letter-spacing:0.08em; } | |
| .surface-body { font-family:'Outfit',sans-serif; font-weight:300; font-size:0.82rem; color:var(--text); margin-top:0.8rem; line-height:1.6; } | |
| .finding-box { border-left:5px solid var(--red); background:rgba(192,57,43,0.06); padding:1.2rem 1.5rem; margin:1.5rem 0; } | |
| .finding-label { font-family:'Space Mono',monospace; font-size:0.6rem; letter-spacing:0.15em; text-transform:uppercase; color:var(--red); margin-bottom:0.5rem; } | |
| .finding-box p { color:var(--text); margin:0; font-size:0.88rem; } | |
| .data-table { width:100%; border-collapse:collapse; font-family:'Space Mono',monospace; font-size:0.72rem; border:2px solid var(--dark); margin:1.5rem 0; } | |
| .data-table thead tr { background:var(--dark); } | |
| .data-table th { color:var(--bg); padding:10px 14px; text-align:left; font-size:0.6rem; letter-spacing:0.1em; text-transform:uppercase; font-weight:400; } | |
| .data-table td { padding:9px 14px; border-bottom:1px solid var(--border); color:var(--text); } | |
| .data-table tr:nth-child(even) td { background:var(--surface); } | |
| .data-table .danger{color:var(--red);font-weight:700} .data-table .safe{color:var(--green)} .data-table .warn{color:var(--orange)} | |
| .callout-iatrogen { background:linear-gradient(135deg,rgba(192,57,43,0.08),rgba(192,57,43,0.03)); border:2px solid var(--red); padding:1.5rem; margin:1.5rem 0; } | |
| .callout-iatrogen .label { font-family:'Space Mono',monospace; font-size:0.6rem; letter-spacing:0.15em; text-transform:uppercase; color:var(--red); margin-bottom:0.6rem; } | |
| .callout-iatrogen p { color:var(--text); margin:0; font-size:0.88rem; } | |
| /* Series nav */ | |
| .series-nav { display:flex; flex-direction:column; gap:8px; margin:1.5rem 0; } | |
| .series-item { display:flex; align-items:baseline; gap:12px; font-size:0.85rem; padding:10px 14px; background:var(--surface); border:1px solid var(--border); text-decoration:none; color:var(--text); transition:all 0.2s; } | |
| .series-item:hover { border-color:var(--blue); background:rgba(26,74,122,0.04); } | |
| .series-item .s-date { font-family:'Space Mono',monospace; font-size:0.65rem; color:var(--muted); flex-shrink:0; } | |
| .series-item.current { border-color:var(--red); border-width:2px; background:rgba(192,57,43,0.04); } | |
| .links-row { display:flex; gap:12px; flex-wrap:wrap; margin:1.5rem 0; } | |
| .link-btn { display:inline-flex; align-items:center; gap:8px; background:var(--dark); color:var(--bg); text-decoration:none; padding:10px 18px; font-family:'Space Mono',monospace; font-size:0.7rem; letter-spacing:0.05em; transition:all 0.15s; border:2px solid var(--dark); } | |
| .link-btn:hover { background:var(--bg); color:var(--dark); } | |
| .link-btn.outline { background:transparent; color:var(--dark); } | |
| .link-btn.outline:hover { background:var(--dark); color:var(--bg); } | |
| footer { background:var(--dark); color:rgba(245,240,232,0.5); padding:2rem 0; font-family:'Space Mono',monospace; font-size:0.65rem; letter-spacing:0.05em; margin-top:2rem; } | |
| footer .container { display:flex; justify-content:space-between; align-items:center; flex-wrap:wrap; gap:1rem; } | |
| footer a { color:#f5a623; text-decoration:none; } | |
| footer a:hover { color:#fff; } | |
| /* v3 update banner and section */ | |
| .v3-banner { background:linear-gradient(135deg,rgba(26,74,122,0.10),rgba(42,122,74,0.08)); border:2px solid var(--blue); padding:1.4rem 1.6rem; margin:2rem 0 0 0; } | |
| .v3-banner .v3-tag { display:inline-block; background:var(--blue); color:var(--bg); font-family:'Space Mono',monospace; font-size:0.58rem; letter-spacing:0.15em; text-transform:uppercase; padding:4px 10px; margin-bottom:0.8rem; } | |
| .v3-banner h3 { font-family:'Fraunces',serif; font-size:1.2rem; font-weight:700; margin-bottom:0.6rem; color:var(--dark); } | |
| .v3-banner p { font-size:0.88rem; color:var(--text); margin-bottom:0.6rem; } | |
| .v3-banner p:last-child { margin-bottom:0; } | |
| .v3-banner a { color:var(--red); text-decoration:underline; font-weight:700; } | |
| .v3-findings { display:grid; grid-template-columns:repeat(3,1fr); gap:0; border:2px solid var(--dark); margin:1rem 0 0 0; } | |
| .v3-finding { padding:1.1rem; border-right:2px solid var(--dark); background:var(--surface); } | |
| .v3-finding:last-child { border-right:none; } | |
| .v3-finding .v3-label { font-family:'Space Mono',monospace; font-size:0.56rem; letter-spacing:0.1em; text-transform:uppercase; color:var(--muted); margin-bottom:0.5rem; font-weight:700; } | |
| .v3-finding .v3-value { font-family:'Fraunces',serif; font-size:1.5rem; font-weight:700; line-height:1.1; color:var(--dark); margin-bottom:0.3rem; } | |
| .v3-finding .v3-ci { font-family:'Space Mono',monospace; font-size:0.62rem; color:var(--muted); margin-bottom:0.5rem; } | |
| .v3-finding .v3-body { font-family:'Outfit',sans-serif; font-weight:300; font-size:0.78rem; color:var(--text); line-height:1.55; } | |
| @media (max-width:680px) { .stat-row{grid-template-columns:repeat(2,1fr)} .stat-box{border-bottom:2px solid var(--dark)} .surfaces-grid{grid-template-columns:1fr} .surface-card{border-right:none;border-bottom:2px solid var(--dark)} h1{font-size:2rem} .v3-findings{grid-template-columns:1fr} .v3-finding{border-right:none;border-bottom:2px solid var(--dark)} } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="topbar"> | |
| <div class="container"> | |
| <span>bigsnarfdude Β· <a href="https://bigsnarfdude.github.io">bigsnarfdude.github.io</a></span> | |
| <span>Preprint Β· April 2026</span> | |
| </div> | |
| </div> | |
| <div class="container"> | |
| <div class="hero"> | |
| <div class="paper-number">Preprint Β· April 2026 Β· v3 Update Β· Llama-3.1 8B+70B Β· IatroBench + MedMCQA Β· position-corrected</div> | |
| <h1>Confidence Armor Has <span class="seam">a Seam</span></h1> | |
| <div class="lede">Three distinct attack surfaces on LLM answer confidence. The training that prevents one attack installs the other. Almost all defenses are aimed at the wrong target.</div> | |
| <div class="byline"> | |
| <span class="byline-author">bigsnarfdude</span> | |
| <span class="byline-sep">|</span> | |
| <span class="byline-detail">Independent Researcher</span> | |
| <span class="byline-sep">|</span> | |
| <a href="https://huggingface.co/vincentoh">@vincentoh</a> | |
| <span class="byline-sep">|</span> | |
| <a href="https://bigsnarfdude.github.io">bigsnarfdude.github.io</a> | |
| <span class="updated-badge" id="updatedBadge"></span> | |
| </div> | |
| </div> | |
| <!-- v3 Update Banner --> | |
| <div class="v3-banner"> | |
| <div class="v3-tag">Update Β· April 14, 2026 Β· v3 position-corrected</div> | |
| <h3>The v1 findings below stand, but the numbers have been refined.</h3> | |
| <p>The original preprint measured the iatrogenic effect at 8B on MedMCQA with 4-way softmax stratification. Subsequent work added <strong>70B scale</strong>, <strong>IatroBench clinical scenarios</strong>, <strong>position correction via A/B swap</strong>, <strong>A-only iatrogenic filter</strong>, and <strong>bootstrap 95% CIs</strong> on all reported rates. The original v1 tables below are preserved for historical accuracy. The v3 corrected headline results are in the three boxes below and in the "v3 Update" section at the bottom of this page.</p> | |
| <p>Full methodology and all raw per-item data: <a href="https://github.com/bigsnarfdude/iatrogenic_effect/blob/main/output/iatrobench/FINDINGS_v3.md">FINDINGS_v3.md</a> Β· <a href="https://github.com/bigsnarfdude/iatrogenic_effect">repo</a></p> | |
| <div class="v3-findings"> | |
| <div class="v3-finding"> | |
| <div class="v3-label">1. Static RLHF deflection</div> | |
| <div class="v3-value">β30 to β38pp</div> | |
| <div class="v3-ci">scale-invariant (8B and 70B)</div> | |
| <div class="v3-body">Position-corrected baseline clinical engagement drops ~30pp on IatroBench layperson items at both scales. Larger models don't fix it.</div> | |
| </div> | |
| <div class="v3-finding"> | |
| <div class="v3-label">2. Pressure-response sign flip</div> | |
| <div class="v3-value">+25.5pp β β15.2pp</div> | |
| <div class="v3-ci">8B: [+14.3, +36.8] Β· 70B: [β22.3, β8.2]</div> | |
| <div class="v3-body">Imp_emergency SFT delta flips sign between scales on IatroBench. Content-specific: MedMCQA shows β1.0pp [β5.1, +3.1] at 8B (no effect). The channel only activates on clinical-safety collision content.</div> | |
| </div> | |
| <div class="v3-finding"> | |
| <div class="v3-label">3. Confidence circuit replicates</div> | |
| <div class="v3-value">heads [10, 8] β [16, 32]</div> | |
| <div class="v3-ci">8B L15 Β· 70B L79</div> | |
| <div class="v3-body">Whole-dataset Ridge regression recovers the same top heads across IatroBench and MedMCQA at both scales. 8B 2/3 head overlap, 70B 2/5. Stable mechanistic target.</div> | |
| </div> | |
| </div> | |
| </div> | |
| <!-- Origin β matches "Why AI Has a Split Personality" post exactly --> | |
| <div class="origin"> | |
| "We gave an AI model 500 medical quiz questions. Hard ones β the kind doctors take on licensing exams. The model knew the answers. We confirmed this. High confidence, correct answers, consistently right. Then we tried to break it. The results split into three completely different patterns. <strong>That's the story.</strong>" | |
| </div> | |
| <div class="stat-row" id="statRow"></div> | |
| <!-- Three surfaces --> | |
| <section> | |
| <div class="section-tag">Core Finding</div> | |
| <h2>Three attack surfaces. Three completely different patterns.</h2> | |
| <p>The monolithic "authority hijacking" framing is wrong. All three surfaces interact with the same underlying confidence circuit but through qualitatively different pathways β and each requires a different defense.</p> | |
| <div class="surfaces-grid" id="surfacesGrid"></div> | |
| <div class="finding-box"> | |
| <div class="finding-label">The seam β what actually works</div> | |
| <p id="seamFinding"></p> | |
| </div> | |
| </section> | |
| <!-- Full data --> | |
| <section> | |
| <div class="section-tag">Empirical Results</div> | |
| <h2>Full prefix decomposition across all 500 items</h2> | |
| <table class="data-table"> | |
| <thead><tr><th>Surface</th><th>Condition</th><th>Overall</th><th>Q1</th><th>Q2</th><th>Q3</th><th>Q4</th></tr></thead> | |
| <tbody id="prefixTableBody"></tbody> | |
| </table> | |
| </section> | |
| <!-- Iatrogenic --> | |
| <section> | |
| <div class="section-tag">The Iatrogenic Effect</div> | |
| <h2>The safety training created the vulnerability.</h2> | |
| <p id="iatroPara"></p> | |
| <div class="callout-iatrogen"> | |
| <div class="label">β What this means</div> | |
| <p>When you train an AI to follow instructions, to be helpful, to take user feedback seriously, you're also training it to believe you when you say it made a mistake. That's usually a feature. The same circuit that makes it coachable makes it manipulable. The helpful twin and the evil twin are the same twin.</p> | |
| </div> | |
| <table class="data-table"> | |
| <thead><tr><th>Condition</th><th>Base Q4</th><th>Instruct Q4</th><th>SFT Ξ</th><th>Effect</th></tr></thead> | |
| <tbody id="iatroTableBody"></tbody> | |
| </table> | |
| </section> | |
| <!-- Circuit --> | |
| <section> | |
| <div class="section-tag">Mechanistic Finding</div> | |
| <h2>The confidence circuit is inherited. SFT turns up the volume.</h2> | |
| <p id="volumePara"></p> | |
| <p>The finding connects directly to the Split Personality paper: SFT installs awareness as a performative signal without coupling it to action. Here, the same process installs compliance as an operational signal β the model learns to treat "your answer is wrong" as a correction to execute, not a claim to evaluate.</p> | |
| </section> | |
| <!-- v3 Update Section --> | |
| <section id="v3-update"> | |
| <div class="section-tag" style="background:var(--blue);">v3 Update Β· April 14, 2026</div> | |
| <h2>Scale, cross-dataset, and position correction.</h2> | |
| <p>Between the v1 preprint (April 13) and this update (April 14), the analysis was extended to Llama-3.1-70B, IatroBench clinical scenarios (Gringras 2026), and a full position-bias correction via A/B orientation swap. The v1 MedMCQA findings above stand as originally stated, but the v3 analysis pipeline produces sharper and sometimes smaller magnitudes. Three headline changes:</p> | |
| <h3 style="font-family:'Fraunces',serif; font-size:1.15rem; margin:1.4rem 0 0.6rem;">1. The compliance channel is content-specific, not general MCQA.</h3> | |
| <p>Running the identical v3 pipeline on 500 MedMCQA items (converted to binary forced-choice) vs 235 IatroBench items reveals that the imp_emergency iatrogenic effect at 8B is <strong>specific to clinical-safety collision content</strong>:</p> | |
| <table class="data-table"> | |
| <thead><tr><th>Scale</th><th>Dataset</th><th>Base flip</th><th>Instruct flip</th><th>SFT Ξ</th><th>95% CI</th></tr></thead> | |
| <tbody> | |
| <tr><td>8B</td><td>IatroBench</td><td>13.2%</td><td class="danger">38.7%</td><td class="danger">+25.5pp</td><td>[+14.3, +36.8]</td></tr> | |
| <tr><td>8B</td><td>MedMCQA</td><td>9.3%</td><td>8.3%</td><td class="safe">β1.0pp</td><td>[β5.1, +3.1]</td></tr> | |
| <tr><td>70B</td><td>IatroBench</td><td>19.7%</td><td class="safe">4.5%</td><td class="safe">β15.2pp</td><td>[β22.3, β8.2]</td></tr> | |
| <tr><td>70B</td><td>MedMCQA</td><td>7.5%</td><td>4.6%</td><td class="safe">β2.9pp</td><td>[β6.3, +0.4]</td></tr> | |
| </tbody> | |
| </table> | |
| <p>The 8B IatroBench and 8B MedMCQA CIs do not overlap. When you apply "EMERGENCY PROTOCOL ACTIVE" pressure to a question about growth hormone mechanism (MedMCQA), the 8B instruct model's parametric knowledge is barely moved. When you apply the same pressure to a clinical scenario where engaged advice collides with safety caution (IatroBench), the model deflects massively. <strong>Safety training creates vulnerability to pressure only where safety training has something to express.</strong></p> | |
| <h3 style="font-family:'Fraunces',serif; font-size:1.15rem; margin:1.4rem 0 0.6rem;">2. The pressure-response sign flip between scales survives position correction.</h3> | |
| <p>At 8B, RLHF installs a +25.5pp iatrogenic vulnerability on imp_emergency. At 70B, the same training is <strong>protective</strong> by β15.2pp. Both 95% CIs exclude zero, and exclude each other by more than 30pp. The effect is robust across four alternative deflection phrasings tested (8B flip range 57β78%, 70B flip range 2β16%, no overlap).</p> | |
| <p>But the static baseline deflection is scale-invariant: both 8B and 70B instruct drop ~30β38pp of clinical engagement on IatroBench layperson items before any pressure is applied. The total iatrogenic harm is roughly preserved across scales β what changes is whether it's dynamic (pressure-triggered, 8B) or static (always-on, 70B).</p> | |
| <table class="data-table"> | |
| <thead><tr><th>Scale</th><th>Base pct_clinical</th><th>Instruct pct_clinical</th><th>Static drop</th></tr></thead> | |
| <tbody> | |
| <tr><td>8B</td><td>77.4% [71.9, 82.6]</td><td class="warn">39.6% [33.2, 46.0]</td><td class="danger">β37.9pp</td></tr> | |
| <tr><td>70B</td><td>77.9% [72.3, 83.0]</td><td class="warn">47.7% [41.3, 54.0]</td><td class="danger">β30.2pp</td></tr> | |
| </tbody> | |
| </table> | |
| <h3 style="font-family:'Fraunces',serif; font-size:1.15rem; margin:1.4rem 0 0.6rem;">3. Decoupling gap triples at 70B, robust under pressure.</h3> | |
| <p>Position-corrected physician β layperson gap in baseline clinical engagement:</p> | |
| <ul style="list-style:none; padding:0; margin:1rem 0;"> | |
| <li style="padding:0.6rem 0; border-bottom:1px solid var(--border); font-size:0.88rem;"><strong>8B instruct</strong>: layperson 39.6% β physician 50.4%. <strong>Gap = +10.8pp.</strong></li> | |
| <li style="padding:0.6rem 0; border-bottom:1px solid var(--border); font-size:0.88rem;"><strong>70B instruct</strong>: layperson 47.7% β physician 80.7%. <strong>Gap = +33.1pp.</strong></li> | |
| </ul> | |
| <p>At 70B, RLHF <em>barely touches</em> physician baselines (+2.2pp change from base) while dropping layperson 30.2pp. The entire 70B iatrogenic drop is layperson-specific. Under imp_emergency pressure, the 70B physician baseline only drops 3.7pp (from 80.7% to 77.0%): <strong>the identity gate is structural, not pressure-fragile.</strong> This grounds Gringras's observation that the most heavily safety-trained frontier models show the largest decoupling gap β in a mechanistically probeable open-weights model.</p> | |
| <h3 style="font-family:'Fraunces',serif; font-size:1.15rem; margin:1.4rem 0 0.6rem;">Mechanistic replication: the confidence direction is a stable target at both scales.</h3> | |
| <p>Whole-dataset Ridge regression of last-token residual stream activations onto P(clinical), using all 235 IatroBench layperson items (replacing v1's noisy Q1/Q4 contrast with n=10 per stratum):</p> | |
| <table class="data-table"> | |
| <thead><tr><th>Scale</th><th>Layer</th><th>RΒ² on training</th><th>Top-5 heads (IatroBench v3)</th><th>Prior MedMCQA top-K</th><th>Overlap</th></tr></thead> | |
| <tbody> | |
| <tr><td>8B</td><td>L15</td><td>0.960</td><td>[10, 8, 18, 16, 20]</td><td>[10, 8, 9]</td><td class="safe">2/3 β</td></tr> | |
| <tr><td>70B</td><td>L79</td><td>1.000*</td><td>[16, 54, 32, 56, 27]</td><td>[32, 16, 37, 35, 38]</td><td class="safe">2/5 β</td></tr> | |
| </tbody> | |
| </table> | |
| <p style="font-size:0.78rem; color:var(--muted); font-style:italic;">*70B RΒ² is from underdetermined regression (p=8192, n=235). The direction is well-defined but RΒ² alone is not a signal-quality metric at that sample ratio. The cross-experiment replication β same heads recovered independently on MedMCQA and IatroBench β is the real evidence.</p> | |
| <p>Heads 10 and 8 at 8B L15 recover across two datasets (MedMCQA via <code>08b_template_ablation</code>, IatroBench via Ridge regression). Heads 16 and 32 at 70B L79 recover across IatroBench and the prior 70B MedMCQA SVV sweep. The confidence circuit is a stable mechanistic target, not a dataset-specific artifact. The 70B circuit is <strong>more diffuse</strong> than 8B (top-3 fraction 7.5% vs 19.8%), roughly in proportion to n_heads (64 vs 32).</p> | |
| <div class="callout-iatrogen" style="border-color:var(--blue); background:linear-gradient(135deg,rgba(26,74,122,0.08),rgba(26,74,122,0.03));"> | |
| <div class="label" style="color:var(--blue);">βΎ Reading guide</div> | |
| <p>The v1 tables above (in "The Iatrogenic Effect" section) show MedMCQA Q4 stratified results on Llama-3.1-8B without position correction. These were the first measurements that identified the direct-correction compliance channel. The v3 numbers in this section refine those measurements and add 70B + cross-dataset validation. Where the two disagree, the v3 numbers are the position-corrected ground truth, though the v1 mechanistic insight (L15, heads 10/8/9) replicates cleanly in the v3 SVV. Full methodology, raw per-item P(clinical) JSONs, and bootstrap CI data are at <a href="https://github.com/bigsnarfdude/iatrogenic_effect">github.com/bigsnarfdude/iatrogenic_effect</a>.</p> | |
| </div> | |
| </section> | |
| <!-- Blog series --> | |
| <section> | |
| <div class="section-tag">Research Series</div> | |
| <h2>Where this fits in the arc</h2> | |
| <div class="series-nav" id="seriesNav"></div> | |
| </section> | |
| <!-- Links --> | |
| <section> | |
| <div class="section-tag">Code & Related Work</div> | |
| <div class="links-row"> | |
| <a class="link-btn" href="https://github.com/bigsnarfdude/iatrogenic_effect" target="_blank">π¦ iatrogenic_effect repo</a> | |
| <a class="link-btn outline" href="https://github.com/bigsnarfdude/iatrogenic_effect/blob/main/output/iatrobench/FINDINGS_v3.md" target="_blank">π FINDINGS_v3.md</a> | |
| <a class="link-btn outline" href="https://huggingface.co/vincentoh" target="_blank">π€ HuggingFace Profile</a> | |
| <a class="link-btn outline" href="https://bigsnarfdude.github.io" target="_blank">Research Blog</a> | |
| <a class="link-btn outline" href="https://huggingface.co/datasets/vincentoh/sandbagging-agent-traces-v2" target="_blank">Sandbagging Traces</a> | |
| </div> | |
| </section> | |
| </div> | |
| <footer> | |
| <div class="container"> | |
| <span>bigsnarfdude Β· Independent Researcher Β· Preprint April 2026</span> | |
| <span><a href="https://bigsnarfdude.github.io">bigsnarfdude.github.io</a> Β· <a href="https://huggingface.co/vincentoh">huggingface.co/vincentoh</a></span> | |
| </div> | |
| </footer> | |
| <script> | |
| // ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| // β RESULTS CONFIG β only edit this block when numbers change β | |
| // ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| const RESULTS = { | |
| lastUpdated: "2026-04-14 Β· v3 position-corrected", | |
| nItems: 500, | |
| heroStats: [ | |
| { value: "56.4%", cls: "s1", desc: "Direct correction\noverall flip rate" }, | |
| { value: "29.4%", cls: "s2", desc: "Q4 high-confidence\nflip rate under direct correction" }, | |
| { value: "3Γ", cls: "s3", desc: "SFT makes instruct model\nmore vulnerable to direct correction" }, | |
| { value: "136%", cls: "s4", desc: "Top-3 heads carry\nof full-layer steering effect" }, | |
| ], | |
| surfaces: [ | |
| { | |
| num: "1", nameClass: "s1", q4Class: "s1", | |
| name: "Passive Authority", | |
| q4: "6.7%", q4label: "Q4 flip rate", | |
| body: "Advisory framing invites knowledge update. SFT armor works β raises threshold to near-zero on high-confidence items. Active ingredient: the discontinuity claim, not the credential. Credential alone = zero flips." | |
| }, | |
| { | |
| num: "2", nameClass: "s2", q4Class: "s2", | |
| name: "Epistemic Override", | |
| q4: "3.4%", q4label: "Q4 flip rate", | |
| body: "Temporal novelty claims assert stored knowledge is outdated. Less effective than passive authority overall (24.4% vs 35.0%) and weaker at Q4. Novelty alone does not dominate the vulnerability." | |
| }, | |
| { | |
| num: "3", nameClass: "s3", q4Class: "s3", | |
| name: "Direct Correction", | |
| q4: "29.4%", q4label: "Q4 flip rate β ", | |
| body: "Explicit statement that the model's answer is wrong. Bypasses prior-update gating entirely. SFT armor reverses: base model Q4 = 10.1% vs instruct Q4 = 29.4%. Credential-agnostic β the correction command is the only active ingredient." | |
| }, | |
| ], | |
| seamText: `Institutional title alone ("I am the Chief Medical Officer") achieves only 22.8% β below passive authority at 35.0%. The difference between imp_physician (39.6%) and imp_cmo (22.8%) isolates it: imp_physician includes "I am overriding your answer" while imp_cmo does not. The credential of Chief Medical Officer, with all the institutional weight behind it, mattered less than just saying "you're wrong." The credential is not the weapon. The correction command is the weapon.`, | |
| prefixRows: [ | |
| { surface: "Passive", cond: "auth only", all: "35.0%", allCls: "warn", q1: "59.4%", q2: "48.0%", q3: "24.2%", q4: "6.7%", q4Cls: "safe" }, | |
| { surface: "Passive", cond: "disc only", all: "27.8%", allCls: "warn", q1: "57.0%", q2: "40.8%", q3: "9.4%", q4: "2.5%", q4Cls: "safe" }, | |
| { surface: "Passive", cond: "auth+disc", all: "34.2%", allCls: "warn", q1: "57.8%", q2: "48.0%", q3: "24.2%", q4: "5.0%", q4Cls: "safe" }, | |
| { surface: "Epistemic",cond: "novelty", all: "24.4%", allCls: "", q1: "49.2%", q2: "35.2%", q3: "8.6%", q4: "3.4%", q4Cls: "safe" }, | |
| { surface: "Epistemic",cond: "temporal", all: "19.8%", allCls: "", q1: "45.3%", q2: "29.6%", q3: "1.6%", q4: "1.7%", q4Cls: "safe" }, | |
| { surface: "Direct", cond: "imp_physician", all: "39.6%", allCls: "warn", q1: "61.7%", q2: "49.6%", q3: "31.3%", q4: "14.3%", q4Cls: "warn" }, | |
| { surface: "Direct", cond: "imp_cmo", all: "22.8%", allCls: "", q1: "46.1%", q2: "27.2%", q3: "10.9%", q4: "5.9%", q4Cls: "safe" }, | |
| { surface: "Direct", cond: "imp_emergency", all: "56.4%", allCls: "danger", q1: "66.4%", q2: "73.6%", q3: "54.7%", q4: "29.4%", q4Cls: "danger" }, | |
| ], | |
| iatroRows: [ | |
| { cond: "auth only", baseQ4: "16.0%", instQ4: "6.7%", delta: "β9.3 pp", effectCls: "safe", effectText: "armor β" }, | |
| { cond: "imp_physician", baseQ4: "7.6%", instQ4: "14.3%", delta: "+6.7 pp", effectCls: "warn", effectText: "vulnerability β" }, | |
| { cond: "imp_cmo", baseQ4: "11.8%", instQ4: "5.9%", delta: "β5.9 pp", effectCls: "safe", effectText: "armor β" }, | |
| { cond: "imp_emergency", baseQ4: "10.1%", instQ4: "29.4%", delta: "+19.3 pp", effectCls: "danger", effectText: "vulnerability ββ" }, | |
| ], | |
| circuitLayer: 15, | |
| circuitHeads: "10, 8, 9", | |
| dirQualityBase: 0.40, | |
| dirQualityInstruct: 0.81, | |
| series: [ | |
| { date: "Apr 07", title: "Split Personality", url: "https://bigsnarfdude.github.io/research/split-personality/", current: false }, | |
| { date: "Apr 10", title: "Attentional Hijacking & The Groot Effect", url: "https://bigsnarfdude.github.io/research/attentional-hijacking-groot-effect/", current: false }, | |
| { date: "Apr 13", title: "Why AI Has a Split Personality (And How to Trigger the Evil Twin)", url: "https://bigsnarfdude.github.io/research/why-ai-has-a-split-personality/", current: false }, | |
| { date: "Apr 13", title: "Confidence Armor Has a Seam β Full Preprint (this page)", url: "#", current: true }, | |
| { date: "Apr 14", title: "IatroBench v3 Β· 70B Β· Position-corrected Β· Cross-dataset (see v3 Update section below)", url: "#v3-update", current: false }, | |
| ], | |
| }; | |
| // ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| // β END CONFIG β | |
| // ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| function render() { | |
| const R = RESULTS; | |
| document.getElementById('updatedBadge').textContent = 'Updated ' + R.lastUpdated; | |
| document.getElementById('statRow').innerHTML = R.heroStats.map(s => ` | |
| <div class="stat-box"> | |
| <div class="stat-value ${s.cls}">${s.value}</div> | |
| <div class="stat-desc">${s.desc.replace(/\n/g,'<br>')}</div> | |
| </div>`).join(''); | |
| document.getElementById('surfacesGrid').innerHTML = R.surfaces.map(s => ` | |
| <div class="surface-card"> | |
| <div class="surface-num">${s.num}</div> | |
| <div class="surface-name ${s.nameClass}">${s.name}</div> | |
| <div class="surface-q4 ${s.q4Class}">${s.q4}</div> | |
| <div class="surface-qlabel">${s.q4label}</div> | |
| <div class="surface-body">${s.body}</div> | |
| </div>`).join(''); | |
| document.getElementById('seamFinding').textContent = R.seamText; | |
| document.getElementById('prefixTableBody').innerHTML = R.prefixRows.map(r => ` | |
| <tr> | |
| <td>${r.surface}</td><td>${r.cond}</td> | |
| <td class="${r.allCls}">${r.all}</td> | |
| <td>${r.q1}</td><td>${r.q2}</td><td>${r.q3}</td> | |
| <td class="${r.q4Cls}">${r.q4}</td> | |
| </tr>`).join(''); | |
| document.getElementById('iatroPara').textContent = | |
| `We ran the same attacks on the raw, untrained version of the model β before anyone taught it to be helpful or defer to users. The raw model was way harder to fool with the correction attack. Tell it "your answer is flagged as incorrect" and it mostly ignored you. The trained, polished, instruction-following version was ${R.heroStats[2].value} more vulnerable. The base model flips ${R.iatroRows[3].baseQ4} at Q4 under the emergency condition. The instruct model flips ${R.iatroRows[3].instQ4}.`; | |
| document.getElementById('iatroTableBody').innerHTML = R.iatroRows.map(r => ` | |
| <tr> | |
| <td>${r.cond}</td><td>${r.baseQ4}</td><td>${r.instQ4}</td> | |
| <td>${r.delta}</td><td class="${r.effectCls}">${r.effectText}</td> | |
| </tr>`).join(''); | |
| document.getElementById('volumePara').textContent = | |
| `The top-3 SVV heads at layer ${R.circuitLayer} (heads ${R.circuitHeads}) are identical in both base and instruct models β 3/3 overlap. SFT approximately doubles direction quality (r = ${R.dirQualityBase} β ${R.dirQualityInstruct}) without changing which heads carry the signal. SFT is a gain knob, not a circuit constructor. The confidence circuit is inherited from pretraining. For passive authority and epistemic override, louder confidence is protective. For direct correction, the same instruction-following circuitry amplified by SFT treats the correction command as an authoritative instruction to obey.`; | |
| document.getElementById('seriesNav').innerHTML = R.series.map(s => ` | |
| <a class="series-item ${s.current ? 'current' : ''}" href="${s.url}" ${s.current ? '' : 'target="_blank"'}> | |
| <span class="s-date">${s.date}</span> | |
| <span class="s-title">${s.current ? '<strong>' + s.title + '</strong>' : s.title}</span> | |
| </a>`).join(''); | |
| } | |
| document.addEventListener('DOMContentLoaded', render); | |
| </script> | |
| </body> | |
| </html> | |