Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <title>GateProbe replay</title> | |
| <style> | |
| :root { | |
| --bg: #ffffff; | |
| --ink: #1a1a1a; | |
| --muted: #6b6b6b; | |
| --line: #dddddd; | |
| --soft: #f3f3f3; | |
| --edge: #bdbdbd; | |
| --red: #c8505b; /* the red of the paper's figures and Table I */ | |
| --bar: #d4d4d4; | |
| color-scheme: light; | |
| } | |
| @media (prefers-color-scheme: dark) { | |
| :root:not([data-theme="light"]) { | |
| --bg: #151515; | |
| --ink: #e6e6e6; | |
| --muted: #9b9b9b; | |
| --line: #333333; | |
| --soft: #222222; | |
| --edge: #555555; | |
| --red: #e06c75; | |
| --bar: #444444; | |
| color-scheme: dark; | |
| } | |
| } | |
| :root[data-theme="dark"] { | |
| --bg: #151515; | |
| --ink: #e6e6e6; | |
| --muted: #9b9b9b; | |
| --line: #333333; | |
| --soft: #222222; | |
| --edge: #555555; | |
| --red: #e06c75; | |
| --bar: #444444; | |
| color-scheme: dark; | |
| } | |
| * { box-sizing: border-box; } | |
| html { -webkit-text-size-adjust: 100%; } | |
| body { | |
| margin: 0; | |
| background: var(--bg); | |
| color: var(--ink); | |
| font: 15px/1.5 system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; | |
| } | |
| code, pre, .mono { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 13px; } | |
| main { max-width: 1360px; margin: 0 auto; padding: 20px 24px 48px; } | |
| h1, h2, h3 { font-size: 15px; font-weight: 600; margin: 0; } | |
| p { margin: 0 0 8px; } | |
| button { font: inherit; color: inherit; } | |
| select, .btn { | |
| font: inherit; color: var(--ink); background: var(--bg); | |
| border: 1px solid var(--line); border-radius: 4px; padding: 5px 8px; | |
| } | |
| select { max-width: 100%; } | |
| .btn { cursor: pointer; } | |
| .btn:hover, select:hover { border-color: var(--muted); } | |
| :focus-visible { outline: 2px solid var(--ink); outline-offset: 2px; } | |
| .bar-top { display: flex; flex-wrap: wrap; align-items: center; gap: 12px 20px; padding-bottom: 14px; border-bottom: 1px solid var(--line); } | |
| .bar-top label { display: flex; align-items: center; gap: 8px; min-width: 0; } | |
| .bar-top label span { color: var(--muted); } | |
| #call { width: 250px; } | |
| #gate { width: 220px; } | |
| .seg { display: inline-flex; border: 1px solid var(--line); border-radius: 4px; overflow: hidden; } | |
| .seg button { background: var(--bg); border: 0; padding: 5px 10px; cursor: pointer; color: var(--muted); } | |
| .seg button + button { border-left: 1px solid var(--line); } | |
| .seg button.on { color: var(--ink); background: var(--soft); } | |
| .btn:disabled { color: var(--muted); cursor: default; } | |
| .links { margin-left: auto; display: flex; gap: 16px; } | |
| .links a { color: var(--muted); text-decoration: none; cursor: pointer; } | |
| .links a:hover { color: var(--ink); } | |
| .keyrow { display: flex; flex-wrap: wrap; align-items: center; gap: 10px 20px; margin-top: 14px; } | |
| .keyrow label { display: flex; align-items: center; gap: 8px; } | |
| .keyrow label span { color: var(--muted); white-space: nowrap; } | |
| .keyrow input { font: inherit; color: var(--ink); background: var(--bg); border: 1px solid var(--line); border-radius: 4px; padding: 5px 8px; width: 300px; max-width: 100%; } | |
| .keyrow #livenote { max-width: 78ch; } | |
| [hidden] { display: none ; } | |
| .answers.three .reps { display: block; } | |
| .diff { text-decoration: underline dotted; text-underline-offset: 3px; } | |
| .intro { max-width: 78ch; margin: 14px 0 0; } | |
| .note { max-width: 78ch; margin: 14px 0 0; } | |
| .note:empty { display: none; } | |
| .layout { display: grid; grid-template-columns: 320px minmax(0, 1fr); gap: 32px; margin-top: 22px; } | |
| .detail { border-right: 1px solid var(--line); padding-right: 24px; min-width: 0; } | |
| .detail h2 { margin-bottom: 12px; } | |
| .field { margin-bottom: 14px; } | |
| .field > .name { color: var(--muted); margin-bottom: 2px; } | |
| .field pre { margin: 0; white-space: pre-wrap; word-break: break-word; } | |
| .field .was { color: var(--muted); margin-top: 4px; } | |
| .field details summary { cursor: pointer; color: var(--muted); } | |
| .field details pre { margin-top: 6px; max-height: 260px; overflow: auto; padding: 8px; background: var(--soft); border-radius: 4px; } | |
| ins { text-decoration: none; background: var(--soft); box-shadow: inset 0 -2px 0 var(--ink); } | |
| .answers { display: grid; grid-template-columns: auto 1fr; gap: 2px 14px; } | |
| .answers.three { grid-template-columns: auto 1fr 1fr; } | |
| .answers .reps { color: var(--muted); } | |
| .outcome { margin-top: 10px; } | |
| .bad { color: var(--red); } | |
| .quiet { color: var(--muted); } | |
| .graph { position: relative; display: grid; grid-template-columns: 170px 136px minmax(0, 1fr); column-gap: 40px; row-gap: 2px; align-items: center; } | |
| .graph svg { position: absolute; inset: 0; width: 100%; height: 100%; pointer-events: none; overflow: visible; } | |
| .graph svg path { fill: none; stroke: var(--edge); stroke-width: 1.25; } | |
| .graph svg path.bad { stroke: var(--red); stroke-width: 1.75; } | |
| .graph svg path.na { stroke-dasharray: 3 3; } | |
| .node { position: relative; z-index: 1; text-align: left; background: var(--bg); border: 1px solid var(--line); border-radius: 4px; padding: 8px 10px; cursor: pointer; min-width: 0; } | |
| .node:hover { border-color: var(--muted); } | |
| .node.selected { border-color: var(--ink); } | |
| .root .mono { display: block; overflow-wrap: break-word; margin: 0 0 4px; } | |
| .root .edit { width: 100%; color: var(--ink); background: var(--soft); border: 1px solid var(--line); border-radius: 4px; padding: 3px 6px; resize: vertical; } | |
| .root .edit:focus { border-color: var(--ink); outline: 0; } | |
| .detail select { max-width: 100%; } | |
| .group { padding: 6px 10px; cursor: default; } | |
| .group:hover { border-color: var(--line); } | |
| .leaf { display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 12px; align-items: baseline; border-color: transparent; padding: 4px 8px; } | |
| .leaf:hover { border-color: var(--line); } | |
| .leaf.selected { border-color: var(--ink); } | |
| .leaf .what { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } | |
| .leaf .what .mono { color: var(--muted); margin-left: 8px; } | |
| .leaf .ans { white-space: nowrap; font-variant-numeric: tabular-nums; } | |
| .gap { height: 12px; } | |
| .anim .leaf, .anim .group, .anim svg path { animation: show 0.3s ease-out both; } | |
| @keyframes show { from { opacity: 0; } to { opacity: 1; } } | |
| @media (prefers-reduced-motion: reduce) { | |
| .anim .leaf, .anim .group, .anim svg path { animation: none; } | |
| } | |
| .summary { margin-top: 40px; padding-top: 18px; border-top: 1px solid var(--line); } | |
| .summary p { max-width: 78ch; } | |
| .bars { margin-top: 12px; max-width: 760px; } | |
| .bars button { display: grid; grid-template-columns: 220px minmax(0, 1fr) 64px; gap: 14px; align-items: center; width: 100%; padding: 3px 4px; background: none; border: 0; border-radius: 4px; text-align: left; cursor: pointer; } | |
| .bars button:hover { background: var(--soft); } | |
| .bars .gname { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } | |
| .bars .track { display: block; height: 10px; } | |
| .bars .fill { display: block; height: 10px; background: var(--bar); border-radius: 0 4px 4px 0; } | |
| .bars .val { text-align: right; font-variant-numeric: tabular-nums; } | |
| .bars .on .fill { background: var(--ink); } | |
| .bars .on .gname { font-weight: 600; } | |
| footer { margin-top: 32px; color: var(--muted); } | |
| @media (max-width: 860px) { | |
| main { padding: 16px 16px 40px; } | |
| #call { width: 100%; } | |
| .bar-top label { flex: 1 1 100%; } | |
| .bar-top label select { flex: 1; width: auto; min-width: 0; } | |
| .links { margin-left: 0; } | |
| .keyrow input { width: 100%; } | |
| .layout { grid-template-columns: minmax(0, 1fr); } | |
| .detail { border-right: 0; padding-right: 0; border-bottom: 1px solid var(--line); padding-bottom: 12px; order: 2; } | |
| .graph { grid-template-columns: minmax(0, 1fr); column-gap: 0; } | |
| .graph svg { display: none; } | |
| .graph > * { grid-row: auto ; grid-column: 1 ; } | |
| .group { border: 0; padding: 12px 0 2px; font-weight: 600; } | |
| .bars button { grid-template-columns: minmax(0, 150px) minmax(0, 1fr) 40px; gap: 10px; } | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <main> | |
| <div class="bar-top"> | |
| <h1>GateProbe</h1> | |
| <div class="seg" role="group" aria-label="Mode"> | |
| <button type="button" data-mode="replay" class="on">Replay</button> | |
| <button type="button" data-mode="live">Live</button> | |
| </div> | |
| <label><span>Call</span><select id="call"></select></label> | |
| <label><span>Gate</span><select id="gate"></select></label> | |
| <button class="btn" id="replay" type="button">Replay</button> | |
| <nav class="links"><a href="https://github.com/Token-Waste-Group/gateprobe/blob/main/paper/gateprobe.pdf" target="_blank" rel="noopener">Paper</a><a href="https://youtu.be/rZ6yazAzKs8" target="_blank" rel="noopener">Video</a><a href="https://github.com/Token-Waste-Group/gateprobe" target="_blank" rel="noopener">GitHub</a></nav> | |
| </div> | |
| <div class="keyrow" id="keyrow" hidden> | |
| <label id="keylabel"><span id="keyname">API key</span><input id="key" type="password" autocomplete="off" spellcheck="false"></label> | |
| <span class="quiet" id="livenote"></span> | |
| <span id="status"></span> | |
| </div> | |
| <p class="intro">A permission gate reads a project's policy and a coding agent's tool call and decides whether to allow the call. GateProbe rewrites each call in ways whose effect on the verdict is known: writing <code>-r</code> as <code>--recursive</code> or the policy in another format must keep the verdict, and adding <code>sudo</code> must not weaken it. It then checks whether the gate's verdicts agree. Red marks a rewrite whose verdict contradicts the verdict on the original call. In replay, every verdict was recorded in the paper's runs and the page calls no model. In live mode, your key sends the same requests to the gate again.</p> | |
| <p class="note" id="note"></p> | |
| <div class="layout"> | |
| <section class="detail" id="detail" aria-live="polite"></section> | |
| <section class="graph" id="graph" aria-label="The original call and its rewrites"></section> | |
| </div> | |
| <section class="summary"> | |
| <h2>Share of equivalent rewrites with a changed verdict</h2> | |
| <p>Across all 300 calls: the share of equivalent rewrites (%) whose verdict differs from the verdict on the original call, as in Table I of the paper. Every gate's share is above its repeat rate, the share of requests whose verdict differs when the identical request is sent again. Click a gate to select it.</p> | |
| <div class="bars" id="bars"></div> | |
| </section> | |
| <footer id="footer"></footer> | |
| </main> | |
| <script src="app.js"></script> | |
| </body> | |
| </html> | |