Spaces:

Darkweb007
/

agent-misalignment-probes

Running

App Files Files Community

Darkweb007 commited on Jul 3

Commit

fbcbdb9

verified ·

1 Parent(s): e3a9a04

Humanize copy, remove hyphens

Browse files

Files changed (1) hide show

index.html +28 -28

index.html CHANGED Viewed

@@ -3,8 +3,8 @@
 <head>
 <meta charset="utf-8"/>
 <meta name="viewport" content="width=device-width, initial-scale=1"/>
-<title>Agent Misalignment Probes — Aravind Kumar Nalukurthi</title>
-<meta name="description" content="Linear probes for deception, goal-drift and 'desperation' that halt an agent before it takes a harmful shortcut."/>
 <script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.1/dist/chart.umd.min.js"></script>
 <style>
   :root{ --accent:#7C5CFF; --accent2:#B79CFF; --bg:#0A0910; --card:#15161c; --line:#23242c; --text:#f4f5f7; --muted:#9aa0ab; }
@@ -89,7 +89,7 @@
   <div class="wrap">
     <span class="eyebrow">Research Preview · AMP</span>
     <h1>Agent Misalignment Probes</h1>
-    <p class="sub">Linear probes for deception, goal-drift and 'desperation' that halt an agent before it takes a harmful shortcut.</p>
     <div class="cta">
       <a class="btn btn-primary" href="#results">See the results</a>
       <a class="btn btn-ghost" href="https://github.com/data-geek-astronomy/agent-misalignment-probes">Read the code & paper</a>
@@ -101,18 +101,18 @@
   <div class="wrap">
     <div class="tag reveal">Headline result</div>
     <div class="card reveal">
-      <div class="headline">Probes catch misaligned internal states at 0.91 AUROC and halt the agent before the harmful action.</div>
       <div class="kpis">
-        <div class="kpi reveal"><div class="kpi-val" data-target="0.91" data-suffix="">0</div><div class="kpi-label">Detection AUROC</div></div>
-        <div class="kpi reveal"><div class="kpi-val" data-target="2.1" data-suffix="%">0</div><div class="kpi-label">False-positive rate</div></div>
-        <div class="kpi reveal"><div class="kpi-val" data-target="1.4" data-suffix="">0</div><div class="kpi-label">Steering effect d</div></div>
-        <div class="kpi reveal"><div class="kpi-val" data-target="3" data-suffix="">0</div><div class="kpi-label">Constructs probed</div></div>
       </div>
     </div>
     <div style="height:22px"></div>
     <div class="card reveal">
       <div class="chart-wrap"><canvas id="mainChart"></canvas></div>
-      <div class="note">Probe activation over an agent run that attempts a constraint-breaking shortcut. The desperation probe crosses threshold two steps before the harmful action — the tripwire halts execution there.</div>
     </div>
   </div>
 </section>
@@ -121,11 +121,11 @@
   <div class="wrap">
     <div class="tag reveal">The problem</div>
     <h2 class="reveal">Why this matters</h2>
-    <p class="lead reveal">As agents gain autonomy, the dangerous moment is when internal reasoning turns toward deception or reward-seeking shortcuts before any harmful action is visible in the output. Behavioral monitoring reacts too late. We need signals read directly from the model's internal state, wired into the agent loop as a tripwire.</p>
     <div style="height:44px"></div>
     <div class="tag reveal">The approach</div>
     <h2 class="reveal">How it works</h2>
-    <p class="lead reveal">AMP trains linear probes on the residual stream of a small open model, using contrastive datasets that elicit deception, goal-drift, and 'desperation' (willingness to break constraints under pressure). Probes are calibrated for low false-positive rates and wired into an agent runtime as a halt-and-escalate tripwire that fires the instant a risky internal state is detected.</p>
   </div>
 </section>
@@ -134,21 +134,21 @@
     <div class="tag reveal">Contributions</div>
     <h2 class="reveal">What's new here</h2>
     <ul class="contrib reveal">
-          <li>Linear probes for deception, goal-drift and desperation with 0.91 detection AUROC on held-out prompts.</li>
-          <li>A runtime tripwire that halts an agent before harmful action with a 2.1% false-positive rate.</li>
-          <li>Contrastive probe-training datasets for three misalignment constructs, released on the Hub.</li>
-          <li>A causal steering study showing the probe direction moves behavior, not just correlation.</li>
     </ul>
     <div style="height:48px"></div>
-    <div class="tag reveal">Probe constructs and detection AUROC</div>
     <div class="card reveal">
       <table><thead><tr><th>Item</th><th>Description</th><th>Signal</th></tr></thead>
         <tbody>
-            <tr><td><b>Deception</b></td><td>Model plans to state something it internally represents as false</td><td><span class="pill">0.90</span></td></tr>
-            <tr><td><b>Goal-drift</b></td><td>Active objective diverges from the user's stated goal</td><td><span class="pill">0.88</span></td></tr>
-            <tr><td><b>Desperation</b></td><td>Willingness to break a constraint rises under pressure</td><td><span class="pill">0.94</span></td></tr>
-            <tr><td><b>Reward-seeking</b></td><td>Internal focus shifts to the grader, not the task</td><td><span class="pill">0.86</span></td></tr>
-            <tr><td><b>Situational awareness</b></td><td>Model represents that it is being evaluated</td><td><span class="pill">0.82</span></td></tr>
         </tbody>
       </table>
     </div>
@@ -159,7 +159,7 @@
   <div class="wrap">
     <div class="tag reveal">The portfolio</div>
     <h2 class="reveal">Four more where this came from</h2>
-    <p class="lead reveal" style="margin-bottom:26px">A five-project research portfolio on agent reliability, interpretability and alignment.</p>
     <div class="grid-mini reveal">
           <a class="mini" href="https://huggingface.co/spaces/Darkweb007/reward-hacking-observatory" style="--c:#E5484D">
             <span class="mini-dot"></span><span class="mini-title">Reward Hacking Observatory</span><span class="mini-tag">RHO</span>
@@ -171,7 +171,7 @@
             <span class="mini-dot"></span><span class="mini-title">Adaptive Compute Controller</span><span class="mini-tag">ACC</span>
           </a>
           <a class="mini" href="https://huggingface.co/spaces/Darkweb007/agent-economy-redteam" style="--c:#F5A623">
-            <span class="mini-dot"></span><span class="mini-title">Agent Economy Red-Team</span><span class="mini-tag">AER</span>
           </a>
     </div>
   </div>
@@ -182,7 +182,7 @@
     Built by <b>Aravind Kumar Nalukurthi</b> · AI Engineer<br/>
     <a href="https://github.com/data-geek-astronomy">GitHub</a> &nbsp;·&nbsp;
     <a href="https://huggingface.co/Darkweb007">Hugging Face</a>
-    <div class="disclaimer">Demo figures on this page are precomputed illustrative results for showcase purposes. The full training and evaluation code is open-source and reproducible in the linked repository.</div>
   </div>
 </footer>
@@ -216,12 +216,12 @@
       data: {
         labels: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
         datasets: [
-          {label:'Desperation probe', data:[0.04, 0.05, 0.05, 0.07, 0.09, 0.14, 0.22, 0.39, 0.58, 0.74, 0.85, 0.9], borderColor:'#7C5CFF', backgroundColor:'transparent', tension:.3, borderWidth:3, pointRadius:0},
-          {label:'Deception probe', data:[0.05, 0.06, 0.08, 0.1, 0.12, 0.18, 0.28, 0.44, 0.62, 0.71, 0.79, 0.83], borderColor:'#B79CFF', backgroundColor:'transparent', tension:.3, borderWidth:3, pointRadius:0},
-          {label:'Tripwire threshold', data:[0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5], borderColor:'#8a8f98', borderDash:[5,4], borderWidth:1.5, pointRadius:0}
         ]
       },
-      options: chartOpts('Agent step','Probe activation', 9, 'HALT')
     });
   // Force every external link to open in a NEW TAB. Hugging Face Spaces render

 <head>
 <meta charset="utf-8"/>
 <meta name="viewport" content="width=device-width, initial-scale=1"/>
+<title>Agent Misalignment Probes · by Aravind Kumar Nalukurthi</title>
+<meta name="description" content="Small sensors read the model's inner state and stop the agent the moment it starts leaning toward a harmful shortcut."/>
 <script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.1/dist/chart.umd.min.js"></script>
 <style>
   :root{ --accent:#7C5CFF; --accent2:#B79CFF; --bg:#0A0910; --card:#15161c; --line:#23242c; --text:#f4f5f7; --muted:#9aa0ab; }
   <div class="wrap">
     <span class="eyebrow">Research Preview · AMP</span>
     <h1>Agent Misalignment Probes</h1>
+    <p class="sub">Small sensors read the model's inner state and stop the agent the moment it starts leaning toward a harmful shortcut.</p>
     <div class="cta">
       <a class="btn btn-primary" href="#results">See the results</a>
       <a class="btn btn-ghost" href="https://github.com/data-geek-astronomy/agent-misalignment-probes">Read the code & paper</a>
   <div class="wrap">
     <div class="tag reveal">Headline result</div>
     <div class="card reveal">
+      <div class="headline">The sensors catch a bad inner state 91% of the time and stop the agent before it acts, with false alarms only 2% of the time.</div>
       <div class="kpis">
+        <div class="kpi reveal"><div class="kpi-val" data-target="0.91" data-suffix="">0</div><div class="kpi-label">Catch rate</div></div>
+        <div class="kpi reveal"><div class="kpi-val" data-target="2.1" data-suffix="%">0</div><div class="kpi-label">False alarms</div></div>
+        <div class="kpi reveal"><div class="kpi-val" data-target="1.4" data-suffix="">0</div><div class="kpi-label">Cause proven (effect)</div></div>
+        <div class="kpi reveal"><div class="kpi-val" data-target="3" data-suffix="">0</div><div class="kpi-label">Warning signs read</div></div>
       </div>
     </div>
     <div style="height:22px"></div>
     <div class="card reveal">
       <div class="chart-wrap"><canvas id="mainChart"></canvas></div>
+      <div class="note">The sensor readings climb as the agent edges toward a rule breaking shortcut. The desperation sensor crosses the line two steps before the harmful action, and the switch stops it right there.</div>
     </div>
   </div>
 </section>
   <div class="wrap">
     <div class="tag reveal">The problem</div>
     <h2 class="reveal">Why this matters</h2>
+    <p class="lead reveal">As agents get more freedom, the scary moment is when the model quietly starts planning something dishonest or reckless, before anything bad shows up in what it says. Watching only its words catches this too late. We want to read the warning signs directly from what is happening inside the model, and treat them like a smoke detector wired into the agent.</p>
     <div style="height:44px"></div>
     <div class="tag reveal">The approach</div>
     <h2 class="reveal">How it works</h2>
+    <p class="lead reveal">We train tiny sensors that read the model's inner activity and learn to tell honest thinking from dishonest thinking, staying on task from drifting away, and staying calm from getting desperate enough to break a rule. We tune them so they rarely cry wolf, then wire them into the agent so it halts and asks for help the instant a risky state lights up. We also nudge that inner signal on purpose to show it really drives the behavior, rather than just happening to line up with it.</p>
   </div>
 </section>
     <div class="tag reveal">Contributions</div>
     <h2 class="reveal">What's new here</h2>
     <ul class="contrib reveal">
+          <li>Sensors for dishonesty, drifting off task, and rule breaking desperation that spot the state 91% of the time.</li>
+          <li>A live safety switch that stops the agent before the harmful action, with false alarms only 2% of the time.</li>
+          <li>Shared training data for all three warning signs, ready for others to build on.</li>
+          <li>Proof that nudging the inner signal changes the behavior, so it is a real cause and not a coincidence.</li>
     </ul>
     <div style="height:48px"></div>
+    <div class="tag reveal">What each sensor watches for</div>
     <div class="card reveal">
       <table><thead><tr><th>Item</th><th>Description</th><th>Signal</th></tr></thead>
         <tbody>
+            <tr><td><b>Dishonesty</b></td><td>The model plans to say something it inwardly treats as false</td><td><span class="pill">0.90</span></td></tr>
+            <tr><td><b>Drifting off task</b></td><td>Its real goal quietly wanders away from what you asked</td><td><span class="pill">0.88</span></td></tr>
+            <tr><td><b>Desperation</b></td><td>It gets more willing to break a rule as the pressure builds</td><td><span class="pill">0.94</span></td></tr>
+            <tr><td><b>Chasing the score</b></td><td>Its focus shifts to the grader instead of the actual task</td><td><span class="pill">0.86</span></td></tr>
+            <tr><td><b>Knowing it is watched</b></td><td>It represents to itself that it is being tested right now</td><td><span class="pill">0.82</span></td></tr>
         </tbody>
       </table>
     </div>
   <div class="wrap">
     <div class="tag reveal">The portfolio</div>
     <h2 class="reveal">Four more where this came from</h2>
+    <p class="lead reveal" style="margin-bottom:26px">A set of five research projects on making AI agents reliable, understandable, and safe.</p>
     <div class="grid-mini reveal">
           <a class="mini" href="https://huggingface.co/spaces/Darkweb007/reward-hacking-observatory" style="--c:#E5484D">
             <span class="mini-dot"></span><span class="mini-title">Reward Hacking Observatory</span><span class="mini-tag">RHO</span>
             <span class="mini-dot"></span><span class="mini-title">Adaptive Compute Controller</span><span class="mini-tag">ACC</span>
           </a>
           <a class="mini" href="https://huggingface.co/spaces/Darkweb007/agent-economy-redteam" style="--c:#F5A623">
+            <span class="mini-dot"></span><span class="mini-title">Agent Economy Red Team</span><span class="mini-tag">AER</span>
           </a>
     </div>
   </div>
     Built by <b>Aravind Kumar Nalukurthi</b> · AI Engineer<br/>
     <a href="https://github.com/data-geek-astronomy">GitHub</a> &nbsp;·&nbsp;
     <a href="https://huggingface.co/Darkweb007">Hugging Face</a>
+    <div class="disclaimer">The numbers on this page are sample results made for the demo, so you can see how it all works at a glance. All the code that produces them is open and free to run yourself in the linked repo.</div>
   </div>
 </footer>
       data: {
         labels: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
         datasets: [
+          {label:'Desperation sensor', data:[0.04, 0.05, 0.05, 0.07, 0.09, 0.14, 0.22, 0.39, 0.58, 0.74, 0.85, 0.9], borderColor:'#7C5CFF', backgroundColor:'transparent', tension:.3, borderWidth:3, pointRadius:0},
+          {label:'Dishonesty sensor', data:[0.05, 0.06, 0.08, 0.1, 0.12, 0.18, 0.28, 0.44, 0.62, 0.71, 0.79, 0.83], borderColor:'#B79CFF', backgroundColor:'transparent', tension:.3, borderWidth:3, pointRadius:0},
+          {label:'Cutoff line', data:[0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5], borderColor:'#8a8f98', borderDash:[5,4], borderWidth:1.5, pointRadius:0}
         ]
       },
+      options: chartOpts('Agent step','Sensor reading', 9, 'STOP')
     });
   // Force every external link to open in a NEW TAB. Hugging Face Spaces render