Spaces:
Running on Zero
Running on Zero
Commit ·
42650ec
1
Parent(s): 463bcbe
Release FeatureLens v0.9.0
Browse files- CHANGELOG.md +17 -0
- README.md +18 -13
- app.py +272 -153
- docs/METHODOLOGY.md +22 -0
- docs/VALIDATION.md +90 -187
- featurelens/runtime.py +140 -0
- pyproject.toml +1 -1
- research_config.json +9 -0
- scripts/release_check.py +21 -6
- tests/test_live_runtime_helpers.py +19 -0
- tests/test_ui_helpers.py +29 -4
CHANGELOG.md
CHANGED
|
@@ -1,5 +1,22 @@
|
|
| 1 |
# Changelog
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
## v0.8.0
|
| 4 |
|
| 5 |
### UI readability and focus
|
|
|
|
| 1 |
# Changelog
|
| 2 |
|
| 3 |
+
## v0.9.0
|
| 4 |
+
|
| 5 |
+
### In-place focus and layout polish
|
| 6 |
+
- Replaced the HF-iframe-hostile overlay/fullscreen experiment with **in-place focus** for plots and tables. The original component expands exactly where it is located; plots are scaled from their existing rendering so aspect ratio is preserved and no cloned toolbar icon can be mistaken for the chart.
|
| 7 |
+
- Focus is capped by both screen width and height, and the same fullscreen toolbar icon toggles the component back without moving the page to the top.
|
| 8 |
+
- Reworked explicit result-table headings into compact HTML headings that occupy the Dataframe toolbar whitespace instead of leaving a large empty band above the first row.
|
| 9 |
+
- Removed the unnecessary “Standalone experiment” dose-response explanation while keeping independent feature and target fields.
|
| 10 |
+
|
| 11 |
+
### Candidate-to-causality workflow
|
| 12 |
+
- Added **Batched causal candidate triage**. Up to eight concept-discovery candidates are independently ablated in one batched scoring run at the current Workbench location.
|
| 13 |
+
- The screen reports native activation, perturbation norm, target mean/sequence log-probability deltas, and next-token JS, and ranks candidates by absolute target effect.
|
| 14 |
+
- The triage deliberately omits random controls; its purpose is to identify which candidate is worth promoting to the existing single-feature 8-direction random-control test.
|
| 15 |
+
- Concept discovery now directly populates the candidate-screen multiselect, defaulting to up to five returned candidates.
|
| 16 |
+
|
| 17 |
+
### GPU-budget-aware validation
|
| 18 |
+
- HF acceptance no longer reruns identity paraphrase, layer trajectory, or 1/3/5 set-size sweeps when those code paths are unchanged. Automated tests cover them; scarce ZeroGPU minutes are reserved for new/touched inference paths.
|
| 19 |
+
|
| 20 |
## v0.8.0
|
| 21 |
|
| 22 |
### UI readability and focus
|
README.md
CHANGED
|
@@ -13,7 +13,7 @@ license: mit
|
|
| 13 |
|
| 14 |
# FeatureLens — Causal Interpretability Workbench
|
| 15 |
|
| 16 |
-
> **v0.
|
| 17 |
|
| 18 |
**Research question:**
|
| 19 |
|
|
@@ -97,8 +97,8 @@ The main workflow supports:
|
|
| 97 |
- exact **full-continuation teacher-forced scoring**;
|
| 98 |
- per-target-token log-probability decomposition;
|
| 99 |
- 8-direction norm-matched random controls;
|
| 100 |
-
-
|
| 101 |
-
-
|
| 102 |
|
| 103 |
### Feature sets
|
| 104 |
|
|
@@ -127,12 +127,14 @@ The difference is reported as an **interaction excess**. A non-zero excess means
|
|
| 127 |
|
| 128 |
### Feature evidence
|
| 129 |
|
| 130 |
-
The **Feature evidence** tab
|
| 131 |
|
| 132 |
-
1. **Concept-guided candidate discovery** — choose one of the seven controlled concept groups and rank SAE features with
|
| 133 |
-
2. **
|
| 134 |
-
3. **
|
| 135 |
-
4. **
|
|
|
|
|
|
|
| 136 |
|
| 137 |
- code;
|
| 138 |
- mathematics;
|
|
@@ -346,12 +348,15 @@ FeatureLens/
|
|
| 346 |
└── research_config.json
|
| 347 |
```
|
| 348 |
|
| 349 |
-
## v0.
|
| 350 |
|
| 351 |
- **Causal-ready candidate ranking** requires concept contrast and current-token activity before suggesting a feature for immediate intervention.
|
| 352 |
-
- **
|
| 353 |
-
- **
|
| 354 |
-
- **
|
|
|
|
|
|
|
|
|
|
| 355 |
|
| 356 |
## Validation
|
| 357 |
|
|
@@ -363,7 +368,7 @@ python scripts/ui_smoke.py
|
|
| 363 |
python scripts/release_check.py
|
| 364 |
```
|
| 365 |
|
| 366 |
-
See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.
|
| 367 |
|
| 368 |
## Limitations
|
| 369 |
|
|
|
|
| 13 |
|
| 14 |
# FeatureLens — Causal Interpretability Workbench
|
| 15 |
|
| 16 |
+
> **v0.9:** a GPU-budget-aware discovery-to-causality workflow with in-place aspect-preserving focus, compact result headings, causal-ready candidate discovery, and a batched multi-candidate ablation screen before the full random-control causal test.
|
| 17 |
|
| 18 |
**Research question:**
|
| 19 |
|
|
|
|
| 97 |
- exact **full-continuation teacher-forced scoring**;
|
| 98 |
- per-target-token log-probability decomposition;
|
| 99 |
- 8-direction norm-matched random controls;
|
| 100 |
+
- batched scale dose-response with its own feature id and target continuation;
|
| 101 |
+
- in-place plot/table focus that preserves the original plot aspect ratio and page location.
|
| 102 |
|
| 103 |
### Feature sets
|
| 104 |
|
|
|
|
| 127 |
|
| 128 |
### Feature evidence
|
| 129 |
|
| 130 |
+
The **Feature evidence** tab supports a discovery-to-causality workflow:
|
| 131 |
|
| 132 |
+
1. **Concept-guided candidate discovery** — choose one of the seven controlled concept groups and rank SAE features with Balanced selectivity, Causal-ready-at-current-token, or raw mean-difference evidence. The result is a candidate list, **not a semantic label**.
|
| 133 |
+
2. **Batched causal candidate triage** — ablate up to eight discovered candidates in one scoring batch and rank their target effects before spending the full random-control ensemble on a selected feature.
|
| 134 |
+
3. **Feature-token activation trace** — show exactly where a selected feature enters the SAE TopK support across every token of the current Workbench prompt.
|
| 135 |
+
4. **Completion-cue sensitivity** — append controlled suffixes/cues to one prompt stem and measure the selected feature at the final token.
|
| 136 |
+
5. **Cue × context specificity** — cross cues with unrelated stems; a strong tested **cue-dominant** pattern is reported explicitly when one cue activates across all tested contexts and off-cue conditions stay inactive.
|
| 137 |
+
6. **Prompt-wide controlled concept contrast** — evaluate one selected feature on a small balanced batch from:
|
| 138 |
|
| 139 |
- code;
|
| 140 |
- mathematics;
|
|
|
|
| 348 |
└── research_config.json
|
| 349 |
```
|
| 350 |
|
| 351 |
+
## v0.9 live workflow improvements
|
| 352 |
|
| 353 |
- **Causal-ready candidate ranking** requires concept contrast and current-token activity before suggesting a feature for immediate intervention.
|
| 354 |
+
- **Batched causal candidate triage** screens up to eight discovered candidates in one target-scoring batch before the expensive random-control causal test.
|
| 355 |
+
- **Dose response** has independent feature and target inputs with concise scale semantics.
|
| 356 |
+
- **Cue × context interpretation** reports cue-dominant tested patterns from the measured activation matrix rather than generic boilerplate.
|
| 357 |
+
- **In-place focus** expands plots/tables at their existing page location and preserves plot aspect ratio instead of opening an HF-iframe overlay.
|
| 358 |
+
- **Compact table headings** use the Dataframe toolbar band rather than leaving a large blank gap above the table.
|
| 359 |
+
- HF validation is **GPU-budget-aware**: unchanged inference paths are not rerun on every release.
|
| 360 |
|
| 361 |
## Validation
|
| 362 |
|
|
|
|
| 368 |
python scripts/release_check.py
|
| 369 |
```
|
| 370 |
|
| 371 |
+
See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.9 validation guide uses the **exact labels shown in the UI** and deliberately avoids rerunning unchanged GPU-heavy regressions. HF acceptance is limited to code paths changed in this release.
|
| 372 |
|
| 373 |
## Limitations
|
| 374 |
|
app.py
CHANGED
|
@@ -121,12 +121,16 @@ CSS = r"""
|
|
| 121 |
}
|
| 122 |
.start-card h3 { margin-top: 0; }
|
| 123 |
.graph-note { font-size: .95rem; opacity: .72; margin-top: 2px; }
|
| 124 |
-
.table-heading {
|
| 125 |
-
|
| 126 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
font-weight: 700 !important;
|
| 128 |
-
line-height: 1.
|
| 129 |
-
margin: 0 !important;
|
| 130 |
}
|
| 131 |
.result-table .label-wrap,
|
| 132 |
.result-table .label-wrap span,
|
|
@@ -148,70 +152,22 @@ CSS = r"""
|
|
| 148 |
font-weight: 700 !important;
|
| 149 |
line-height: 1.25 !important;
|
| 150 |
}
|
| 151 |
-
/*
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
z-index:
|
| 157 |
-
background:
|
| 158 |
-
display: flex;
|
| 159 |
-
align-items: flex-start;
|
| 160 |
-
justify-content: center;
|
| 161 |
-
padding: 5vh 3vw;
|
| 162 |
-
box-sizing: border-box;
|
| 163 |
-
}
|
| 164 |
-
.featurelens-focus-panel {
|
| 165 |
-
width: min(92vw, 1120px);
|
| 166 |
-
max-height: 90vh;
|
| 167 |
-
overflow: auto;
|
| 168 |
-
background: var(--background-fill-primary);
|
| 169 |
-
border: 1px solid var(--border-color-primary);
|
| 170 |
-
border-radius: 4px;
|
| 171 |
-
box-shadow: 0 14px 46px rgba(0, 0, 0, .45);
|
| 172 |
-
padding: 14px 18px 18px;
|
| 173 |
-
}
|
| 174 |
-
.featurelens-focus-toolbar {
|
| 175 |
-
display: flex;
|
| 176 |
-
align-items: center;
|
| 177 |
-
justify-content: space-between;
|
| 178 |
-
margin-bottom: 8px;
|
| 179 |
-
}
|
| 180 |
-
.featurelens-focus-title { font-size: 1.18rem; font-weight: 700; }
|
| 181 |
-
.featurelens-focus-close {
|
| 182 |
-
min-width: 36px !important;
|
| 183 |
-
min-height: 32px !important;
|
| 184 |
-
background: var(--background-fill-secondary) !important;
|
| 185 |
-
color: var(--body-text-color) !important;
|
| 186 |
border: 1px solid var(--border-color-primary) !important;
|
|
|
|
|
|
|
| 187 |
}
|
| 188 |
-
.featurelens-
|
| 189 |
-
|
| 190 |
-
display: flex;
|
| 191 |
-
align-items: flex-start;
|
| 192 |
-
justify-content: center;
|
| 193 |
-
overflow: hidden;
|
| 194 |
-
}
|
| 195 |
-
.featurelens-focus-chart svg,
|
| 196 |
-
.featurelens-focus-chart canvas {
|
| 197 |
-
display: block;
|
| 198 |
-
width: 100% !important;
|
| 199 |
-
max-width: 1080px !important;
|
| 200 |
-
height: auto !important;
|
| 201 |
-
max-height: 78vh !important;
|
| 202 |
-
object-fit: contain;
|
| 203 |
}
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
top: 5vh !important;
|
| 207 |
-
left: 50% !important;
|
| 208 |
-
transform: translateX(-50%) !important;
|
| 209 |
-
width: min(94vw, 1420px) !important;
|
| 210 |
-
height: 90vh !important;
|
| 211 |
-
padding: 18px 22px 24px !important;
|
| 212 |
-
border-radius: 4px !important;
|
| 213 |
-
overflow: auto !important;
|
| 214 |
-
box-shadow: 0 0 0 100vmax rgba(0, 0, 0, .66), 0 14px 40px rgba(0, 0, 0, .35) !important;
|
| 215 |
}
|
| 216 |
.candidate-help { opacity: .78; font-size: .97rem; margin-top: -2px; }
|
| 217 |
@media (max-width: 900px) {
|
|
@@ -281,96 +237,92 @@ INSTALL_REFLOW_JS = r"""
|
|
| 281 |
mutation.observe(root, {subtree: true, childList: true});
|
| 282 |
window.__featurelens_mutation_observer = mutation;
|
| 283 |
|
| 284 |
-
const
|
| 285 |
-
|
| 286 |
-
|
| 287 |
-
|
| 288 |
-
|
| 289 |
-
|
| 290 |
-
|
| 291 |
-
window.
|
| 292 |
-
if (y != null) window.scrollTo(0, y);
|
| 293 |
};
|
| 294 |
|
| 295 |
-
const
|
| 296 |
-
|
| 297 |
-
|
| 298 |
-
|
| 299 |
};
|
| 300 |
|
| 301 |
-
const
|
| 302 |
-
|
| 303 |
-
|
| 304 |
-
|
| 305 |
-
|
| 306 |
-
|
| 307 |
-
const
|
| 308 |
-
|
| 309 |
-
const
|
| 310 |
-
|
| 311 |
-
|
| 312 |
-
|
| 313 |
-
|
| 314 |
-
title.className = "featurelens-focus-title";
|
| 315 |
-
title.textContent = readableTitle(block);
|
| 316 |
-
const close = document.createElement("button");
|
| 317 |
-
close.className = "featurelens-focus-close";
|
| 318 |
-
close.type = "button";
|
| 319 |
-
close.setAttribute("aria-label", "Close chart focus view");
|
| 320 |
-
close.textContent = "Close";
|
| 321 |
-
toolbar.append(title, close);
|
| 322 |
-
|
| 323 |
-
const chart = document.createElement("div");
|
| 324 |
-
chart.className = "featurelens-focus-chart";
|
| 325 |
-
const sourceSvg = block.querySelector("svg");
|
| 326 |
-
const sourceCanvas = block.querySelector("canvas");
|
| 327 |
-
if (sourceSvg) {
|
| 328 |
-
const clone = sourceSvg.cloneNode(true);
|
| 329 |
-
clone.removeAttribute("width");
|
| 330 |
-
clone.removeAttribute("height");
|
| 331 |
-
chart.appendChild(clone);
|
| 332 |
-
} else if (sourceCanvas) {
|
| 333 |
-
const canvas = document.createElement("canvas");
|
| 334 |
-
canvas.width = sourceCanvas.width;
|
| 335 |
-
canvas.height = sourceCanvas.height;
|
| 336 |
-
const context = canvas.getContext("2d");
|
| 337 |
-
if (context) context.drawImage(sourceCanvas, 0, 0);
|
| 338 |
-
chart.appendChild(canvas);
|
| 339 |
-
} else {
|
| 340 |
-
const fallback = block.cloneNode(true);
|
| 341 |
-
fallback.querySelectorAll("button").forEach((node) => node.remove());
|
| 342 |
-
fallback.removeAttribute("id");
|
| 343 |
-
chart.appendChild(fallback);
|
| 344 |
}
|
| 345 |
-
|
| 346 |
-
|
| 347 |
-
|
| 348 |
-
|
| 349 |
-
|
| 350 |
-
|
| 351 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 352 |
};
|
| 353 |
|
| 354 |
-
//
|
| 355 |
-
//
|
| 356 |
document.addEventListener("click", (event) => {
|
| 357 |
const button = event.target && event.target.closest ? event.target.closest("button") : null;
|
| 358 |
if (!button) return;
|
| 359 |
const label = `${button.getAttribute("aria-label") || ""} ${button.getAttribute("title") || ""} ${button.textContent || ""}`.toLowerCase();
|
| 360 |
if (!label.includes("fullscreen")) return;
|
| 361 |
-
const
|
| 362 |
-
if (!
|
| 363 |
event.preventDefault();
|
| 364 |
event.stopImmediatePropagation();
|
| 365 |
-
|
| 366 |
}, true);
|
| 367 |
|
| 368 |
document.addEventListener("keydown", (event) => {
|
| 369 |
-
if (event.key ==
|
|
|
|
|
|
|
| 370 |
});
|
| 371 |
|
| 372 |
-
// Gradio's
|
| 373 |
-
// the export, then rename the transient download anchor before the browser handles it.
|
| 374 |
document.addEventListener("click", (event) => {
|
| 375 |
const button = event.target && event.target.closest ? event.target.closest("button") : null;
|
| 376 |
if (button) {
|
|
@@ -411,8 +363,8 @@ def _copy_button(label: str = "Copy table with headers") -> gr.Button:
|
|
| 411 |
return gr.Button(label, size="sm", variant="primary", elem_classes=["copy-btn"])
|
| 412 |
|
| 413 |
|
| 414 |
-
def _table_heading(text: str) -> gr.
|
| 415 |
-
return gr.
|
| 416 |
|
| 417 |
|
| 418 |
def _copy_ack(_text: str) -> None:
|
|
@@ -658,6 +610,28 @@ def _discovery_metrics_markdown(result) -> str:
|
|
| 658 |
)
|
| 659 |
|
| 660 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 661 |
def _cue_context_metrics_markdown(result) -> str:
|
| 662 |
active = ", ".join(
|
| 663 |
f"{cue}: {count}/{len(result.stems)} contexts"
|
|
@@ -1199,12 +1173,72 @@ def run_concept_feature_discovery(
|
|
| 1199 |
choices = [str(fid) for fid in result.candidate_ids]
|
| 1200 |
default = str(result.default_candidate_id) if result.default_candidate_id is not None else (choices[0] if choices else None)
|
| 1201 |
candidate_update = gr.update(choices=choices, value=default)
|
| 1202 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1203 |
except Exception as exc:
|
| 1204 |
_raise_ui_error(exc)
|
| 1205 |
|
| 1206 |
|
| 1207 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1208 |
@gpu(duration=25)
|
| 1209 |
def run_feature_cue_scan(feature_id: str, layer: int, prompt_stem: str, cue_text: str):
|
| 1210 |
try:
|
|
@@ -1286,7 +1320,7 @@ def set_mode_help(mode: str):
|
|
| 1286 |
with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_width=True) as demo:
|
| 1287 |
gr.HTML(
|
| 1288 |
'<header class="hero">'
|
| 1289 |
-
'<h1>FeatureLens <span style="font-size:.48em;opacity:.58;font-weight:400">v0.
|
| 1290 |
'<div class="subtitle">Causal Interpretability Workbench</div>'
|
| 1291 |
'<div class="question">Discover sparse features, test robustness, and separate correlation from causal influence.</div>'
|
| 1292 |
'</header>'
|
|
@@ -1475,14 +1509,12 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
|
|
| 1475 |
dose_target_text = gr.Textbox(
|
| 1476 |
label="Dose-response target continuation",
|
| 1477 |
value="2x",
|
| 1478 |
-
info="
|
| 1479 |
scale=2,
|
| 1480 |
)
|
| 1481 |
gr.Markdown(
|
| 1482 |
-
"
|
| 1483 |
-
"
|
| 1484 |
-
"Always a **scale** experiment: 0× = ablation, 1× = numerical no-edit reference, 2× = double "
|
| 1485 |
-
"the native coefficient. The six conditions are evaluated together against the 1× row."
|
| 1486 |
)
|
| 1487 |
dose_btn = gr.Button("Run scale dose-response", variant="primary", elem_classes=["action-btn"])
|
| 1488 |
dose_metrics = gr.Markdown()
|
|
@@ -1814,7 +1846,69 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
|
|
| 1814 |
)
|
| 1815 |
candidate_use_status = gr.Markdown()
|
| 1816 |
|
| 1817 |
-
gr.HTML('<div class="section-rule">B.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1818 |
contrast_location = gr.Markdown(
|
| 1819 |
"**Activation trace uses the current Workbench prompt.** The controlled concept scan below uses its own balanced prompt set."
|
| 1820 |
)
|
|
@@ -1873,7 +1967,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
|
|
| 1873 |
height=320,
|
| 1874 |
)
|
| 1875 |
|
| 1876 |
-
gr.HTML('<div class="section-rule">
|
| 1877 |
gr.Markdown(
|
| 1878 |
"Feature 22632 in the current math example fires only on the final `is` token. This scan tests whether a "
|
| 1879 |
"selected feature responds to **completion cues themselves** rather than the underlying concept. Enter a stem "
|
|
@@ -1897,7 +1991,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
|
|
| 1897 |
buttons=["fullscreen", "export"], elem_classes=["fl-plot"], height=320
|
| 1898 |
)
|
| 1899 |
|
| 1900 |
-
gr.HTML('<div class="section-rule">
|
| 1901 |
gr.Markdown(
|
| 1902 |
"The single-stem cue scan can tell you that a feature prefers `is`, but not whether that preference is lexical "
|
| 1903 |
"or context-sensitive. This matrix crosses several prompt stems with the same completion cues in one batch."
|
|
@@ -1952,7 +2046,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
|
|
| 1952 |
height=340,
|
| 1953 |
)
|
| 1954 |
|
| 1955 |
-
gr.HTML('<div class="section-rule">
|
| 1956 |
gr.Markdown("### Prompt-wide controlled concept contrast")
|
| 1957 |
contrast_btn = gr.Button("Run controlled concept contrast", variant="primary", elem_classes=["action-btn"])
|
| 1958 |
contrast_metrics = gr.Markdown()
|
|
@@ -2129,8 +2223,8 @@ Batched experiments include an explicit **zero-edit row**. Causal effects are me
|
|
| 2129 |
|
| 2130 |
1. SAE reconstruction quality.
|
| 2131 |
2. Held-out feature/concept prediction.
|
| 2132 |
-
3. Concept-guided candidate discovery and
|
| 2133 |
-
4.
|
| 2134 |
5. Local and prompt-wide paraphrase robustness.
|
| 2135 |
6. Single-feature causal intervention and dose-response.
|
| 2136 |
7. Contrastive continuation preference under intervention.
|
|
@@ -2229,7 +2323,31 @@ Association, robustness, geometry, and intervention evidence remain separate cla
|
|
| 2229 |
discovery_concept, discovery_layer, discovery_n, discovery_top_n, discovery_ranking,
|
| 2230 |
prompt, token_index,
|
| 2231 |
],
|
| 2232 |
-
outputs=[
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2233 |
)
|
| 2234 |
discovery_table.select(
|
| 2235 |
select_candidate_row,
|
|
@@ -2278,6 +2396,7 @@ Association, robustness, geometry, and intervention evidence remain separate cla
|
|
| 2278 |
(trace_copy, trace_tsv),
|
| 2279 |
(contrast_copy, contrast_tsv),
|
| 2280 |
(discovery_copy, discovery_tsv),
|
|
|
|
| 2281 |
(cue_copy, cue_tsv),
|
| 2282 |
(cue_context_copy, cue_context_tsv),
|
| 2283 |
(para_copy, para_tsv),
|
|
|
|
| 121 |
}
|
| 122 |
.start-card h3 { margin-top: 0; }
|
| 123 |
.graph-note { font-size: .95rem; opacity: .72; margin-top: 2px; }
|
| 124 |
+
.table-heading {
|
| 125 |
+
margin: 0 0 -34px !important;
|
| 126 |
+
padding: 4px 58px 0 0 !important;
|
| 127 |
+
min-height: 34px;
|
| 128 |
+
position: relative;
|
| 129 |
+
z-index: 3;
|
| 130 |
+
pointer-events: none;
|
| 131 |
+
font-size: 1.20rem !important;
|
| 132 |
font-weight: 700 !important;
|
| 133 |
+
line-height: 1.22 !important;
|
|
|
|
| 134 |
}
|
| 135 |
.result-table .label-wrap,
|
| 136 |
.result-table .label-wrap span,
|
|
|
|
| 152 |
font-weight: 700 !important;
|
| 153 |
line-height: 1.25 !important;
|
| 154 |
}
|
| 155 |
+
/* Gradio native fullscreen is intercepted and converted to an in-place focus mode.
|
| 156 |
+
Nothing is moved to the top of the HF iframe: the original component expands where the user clicked it. */
|
| 157 |
+
.fl-plot.featurelens-inline-focus,
|
| 158 |
+
.result-table.featurelens-inline-focus {
|
| 159 |
+
position: relative !important;
|
| 160 |
+
z-index: 5000 !important;
|
| 161 |
+
background: var(--background-fill-primary) !important;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
border: 1px solid var(--border-color-primary) !important;
|
| 163 |
+
box-shadow: 0 12px 36px rgba(0, 0, 0, .42) !important;
|
| 164 |
+
border-radius: 4px !important;
|
| 165 |
}
|
| 166 |
+
.fl-plot.featurelens-inline-focus {
|
| 167 |
+
transform-origin: top left !important;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 168 |
}
|
| 169 |
+
.result-table.featurelens-inline-focus {
|
| 170 |
+
overflow: visible !important;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 171 |
}
|
| 172 |
.candidate-help { opacity: .78; font-size: .97rem; margin-top: -2px; }
|
| 173 |
@media (max-width: 900px) {
|
|
|
|
| 237 |
mutation.observe(root, {subtree: true, childList: true});
|
| 238 |
window.__featurelens_mutation_observer = mutation;
|
| 239 |
|
| 240 |
+
const restoreFocus = (block) => {
|
| 241 |
+
if (!block || !block.classList.contains("featurelens-inline-focus")) return;
|
| 242 |
+
const saved = block.__featurelens_saved_style;
|
| 243 |
+
if (saved == null || saved === "") block.removeAttribute("style");
|
| 244 |
+
else block.setAttribute("style", saved);
|
| 245 |
+
block.classList.remove("featurelens-inline-focus");
|
| 246 |
+
block.__featurelens_saved_style = null;
|
| 247 |
+
window.setTimeout(kick, 30);
|
|
|
|
| 248 |
};
|
| 249 |
|
| 250 |
+
const closeOtherFocus = (except) => {
|
| 251 |
+
document.querySelectorAll(".featurelens-inline-focus").forEach((node) => {
|
| 252 |
+
if (node !== except) restoreFocus(node);
|
| 253 |
+
});
|
| 254 |
};
|
| 255 |
|
| 256 |
+
const focusPlotInPlace = (block) => {
|
| 257 |
+
const rect = block.getBoundingClientRect();
|
| 258 |
+
if (rect.width <= 0 || rect.height <= 0) return;
|
| 259 |
+
const viewportWidth = Math.max(320, document.documentElement.clientWidth || window.innerWidth || rect.width);
|
| 260 |
+
const screenHeight = Math.max(600, (window.screen && window.screen.availHeight) || 900);
|
| 261 |
+
const maxWidth = Math.min(viewportWidth * 0.90, 1100);
|
| 262 |
+
const maxHeight = Math.min(screenHeight * 0.68, 700);
|
| 263 |
+
const scale = Math.max(1, Math.min(maxWidth / rect.width, maxHeight / rect.height, 1.8));
|
| 264 |
+
const focusedWidth = rect.width * scale;
|
| 265 |
+
let dx = (viewportWidth - focusedWidth) / 2 - rect.left;
|
| 266 |
+
if (rect.left + dx < 12) dx += 12 - (rect.left + dx);
|
| 267 |
+
if (rect.left + dx + focusedWidth > viewportWidth - 12) {
|
| 268 |
+
dx -= (rect.left + dx + focusedWidth) - (viewportWidth - 12);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 269 |
}
|
| 270 |
+
block.style.transformOrigin = "top left";
|
| 271 |
+
block.style.transform = `translate(${dx}px, 0px) scale(${scale})`;
|
| 272 |
+
block.style.marginBottom = `${Math.max(8, rect.height * (scale - 1) + 8)}px`;
|
| 273 |
+
block.style.zIndex = "5000";
|
| 274 |
+
};
|
| 275 |
+
|
| 276 |
+
const focusTableInPlace = (block) => {
|
| 277 |
+
const rect = block.getBoundingClientRect();
|
| 278 |
+
if (rect.width <= 0) return;
|
| 279 |
+
const viewportWidth = Math.max(320, document.documentElement.clientWidth || window.innerWidth || rect.width);
|
| 280 |
+
const targetWidth = Math.max(rect.width, Math.min(viewportWidth * 0.94, 1400));
|
| 281 |
+
let dx = (viewportWidth - targetWidth) / 2 - rect.left;
|
| 282 |
+
if (rect.left + dx < 12) dx += 12 - (rect.left + dx);
|
| 283 |
+
if (rect.left + dx + targetWidth > viewportWidth - 12) {
|
| 284 |
+
dx -= (rect.left + dx + targetWidth) - (viewportWidth - 12);
|
| 285 |
+
}
|
| 286 |
+
block.style.width = `${targetWidth}px`;
|
| 287 |
+
block.style.maxWidth = "none";
|
| 288 |
+
block.style.transform = `translateX(${dx}px)`;
|
| 289 |
+
block.style.zIndex = "5000";
|
| 290 |
+
};
|
| 291 |
+
|
| 292 |
+
const toggleInlineFocus = (block) => {
|
| 293 |
+
if (block.classList.contains("featurelens-inline-focus")) {
|
| 294 |
+
restoreFocus(block);
|
| 295 |
+
return;
|
| 296 |
+
}
|
| 297 |
+
closeOtherFocus(block);
|
| 298 |
+
block.__featurelens_saved_style = block.getAttribute("style") || "";
|
| 299 |
+
block.classList.add("featurelens-inline-focus");
|
| 300 |
+
if (block.classList.contains("fl-plot")) focusPlotInPlace(block);
|
| 301 |
+
else focusTableInPlace(block);
|
| 302 |
+
window.setTimeout(kick, 30);
|
| 303 |
};
|
| 304 |
|
| 305 |
+
// Keep the native toolbar icon, but replace Gradio fullscreen with an in-place expansion.
|
| 306 |
+
// This avoids HF iframe jumps and preserves the chart's exact rendered aspect ratio.
|
| 307 |
document.addEventListener("click", (event) => {
|
| 308 |
const button = event.target && event.target.closest ? event.target.closest("button") : null;
|
| 309 |
if (!button) return;
|
| 310 |
const label = `${button.getAttribute("aria-label") || ""} ${button.getAttribute("title") || ""} ${button.textContent || ""}`.toLowerCase();
|
| 311 |
if (!label.includes("fullscreen")) return;
|
| 312 |
+
const block = button.closest(".fl-plot, .result-table");
|
| 313 |
+
if (!block) return;
|
| 314 |
event.preventDefault();
|
| 315 |
event.stopImmediatePropagation();
|
| 316 |
+
toggleInlineFocus(block);
|
| 317 |
}, true);
|
| 318 |
|
| 319 |
document.addEventListener("keydown", (event) => {
|
| 320 |
+
if (event.key !== "Escape") return;
|
| 321 |
+
const active = document.querySelector(".featurelens-inline-focus");
|
| 322 |
+
if (active) restoreFocus(active);
|
| 323 |
});
|
| 324 |
|
| 325 |
+
// Rename Gradio's generic chart.png export without touching the export implementation.
|
|
|
|
| 326 |
document.addEventListener("click", (event) => {
|
| 327 |
const button = event.target && event.target.closest ? event.target.closest("button") : null;
|
| 328 |
if (button) {
|
|
|
|
| 363 |
return gr.Button(label, size="sm", variant="primary", elem_classes=["copy-btn"])
|
| 364 |
|
| 365 |
|
| 366 |
+
def _table_heading(text: str) -> gr.HTML:
|
| 367 |
+
return gr.HTML(f'<div class="table-heading">{text}</div>')
|
| 368 |
|
| 369 |
|
| 370 |
def _copy_ack(_text: str) -> None:
|
|
|
|
| 610 |
)
|
| 611 |
|
| 612 |
|
| 613 |
+
def _candidate_screen_metrics_markdown(result) -> str:
|
| 614 |
+
tokens = " ".join(repr(token) for token in result.target_tokens)
|
| 615 |
+
if result.rows:
|
| 616 |
+
top = result.rows[0]
|
| 617 |
+
strongest = (
|
| 618 |
+
f"Strongest screened target effect: feature **{int(top[1])}** with "
|
| 619 |
+
f"Δ mean log p/token **{float(top[5]):+.4f}** and next-token JS **{float(top[7]):.6f}**."
|
| 620 |
+
)
|
| 621 |
+
else:
|
| 622 |
+
strongest = "No candidate rows were produced."
|
| 623 |
+
return (
|
| 624 |
+
f"Screened **{result.candidate_count}** candidate feature(s); **{result.active_feature_count}** were active "
|
| 625 |
+
f"at the selected Workbench token. Target continuation: {len(result.target_tokens)} token(s): {tokens}. \n"
|
| 626 |
+
f"{strongest} \n"
|
| 627 |
+
f"Execution-context null drift: mean log p/token **{result.execution_drift_mean_logprob:+.2e}**, "
|
| 628 |
+
f"JS **{result.execution_drift_js:.2e}**. \n\n"
|
| 629 |
+
"This is a **triage screen**: all ablations share one batched zero-edit reference, but no random-control "
|
| 630 |
+
"ensemble is spent here. Take a promising feature into **Workbench → II. Single-feature causal test** for "
|
| 631 |
+
"the full 8-direction specificity comparison."
|
| 632 |
+
)
|
| 633 |
+
|
| 634 |
+
|
| 635 |
def _cue_context_metrics_markdown(result) -> str:
|
| 636 |
active = ", ".join(
|
| 637 |
f"{cue}: {count}/{len(result.stems)} contexts"
|
|
|
|
| 1173 |
choices = [str(fid) for fid in result.candidate_ids]
|
| 1174 |
default = str(result.default_candidate_id) if result.default_candidate_id is not None else (choices[0] if choices else None)
|
| 1175 |
candidate_update = gr.update(choices=choices, value=default)
|
| 1176 |
+
screen_update = gr.update(
|
| 1177 |
+
choices=choices,
|
| 1178 |
+
value=choices[: min(5, len(choices))],
|
| 1179 |
+
)
|
| 1180 |
+
return (
|
| 1181 |
+
_discovery_metrics_markdown(result),
|
| 1182 |
+
table,
|
| 1183 |
+
chart,
|
| 1184 |
+
candidate_update,
|
| 1185 |
+
screen_update,
|
| 1186 |
+
_tsv(table),
|
| 1187 |
+
)
|
| 1188 |
except Exception as exc:
|
| 1189 |
_raise_ui_error(exc)
|
| 1190 |
|
| 1191 |
|
| 1192 |
|
| 1193 |
+
@gpu(duration=30)
|
| 1194 |
+
def run_candidate_causal_screen(
|
| 1195 |
+
prompt: str,
|
| 1196 |
+
layer: int,
|
| 1197 |
+
token_index: int,
|
| 1198 |
+
feature_ids: list[str] | None,
|
| 1199 |
+
target_text: str,
|
| 1200 |
+
):
|
| 1201 |
+
try:
|
| 1202 |
+
selected = [int(float(value)) for value in (feature_ids or [])]
|
| 1203 |
+
result = RUNTIME.candidate_causal_screen(
|
| 1204 |
+
text=prompt,
|
| 1205 |
+
layer=int(layer),
|
| 1206 |
+
token_index=int(token_index),
|
| 1207 |
+
feature_ids=selected,
|
| 1208 |
+
target_text=target_text,
|
| 1209 |
+
)
|
| 1210 |
+
columns = [
|
| 1211 |
+
"Rank",
|
| 1212 |
+
"Feature id",
|
| 1213 |
+
"Native activation",
|
| 1214 |
+
"Active at current token",
|
| 1215 |
+
"Perturbation L2",
|
| 1216 |
+
"Δ mean log p/token",
|
| 1217 |
+
"Δ sequence log p",
|
| 1218 |
+
"Next-token JS",
|
| 1219 |
+
]
|
| 1220 |
+
table = pd.DataFrame(result.rows, columns=columns)
|
| 1221 |
+
chart = pd.DataFrame(
|
| 1222 |
+
result.chart_rows,
|
| 1223 |
+
columns=["Feature", "Δ mean log p/token"],
|
| 1224 |
+
)
|
| 1225 |
+
chart["Series"] = "Candidate ablation"
|
| 1226 |
+
choices = [str(feature_id) for feature_id in result.feature_ids]
|
| 1227 |
+
candidate_update = gr.update(
|
| 1228 |
+
choices=choices,
|
| 1229 |
+
value=choices[0] if choices else None,
|
| 1230 |
+
)
|
| 1231 |
+
return (
|
| 1232 |
+
_candidate_screen_metrics_markdown(result),
|
| 1233 |
+
table,
|
| 1234 |
+
chart,
|
| 1235 |
+
candidate_update,
|
| 1236 |
+
_tsv(table),
|
| 1237 |
+
)
|
| 1238 |
+
except Exception as exc:
|
| 1239 |
+
_raise_ui_error(exc)
|
| 1240 |
+
|
| 1241 |
+
|
| 1242 |
@gpu(duration=25)
|
| 1243 |
def run_feature_cue_scan(feature_id: str, layer: int, prompt_stem: str, cue_text: str):
|
| 1244 |
try:
|
|
|
|
| 1320 |
with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_width=True) as demo:
|
| 1321 |
gr.HTML(
|
| 1322 |
'<header class="hero">'
|
| 1323 |
+
'<h1>FeatureLens <span style="font-size:.48em;opacity:.58;font-weight:400">v0.9</span></h1>'
|
| 1324 |
'<div class="subtitle">Causal Interpretability Workbench</div>'
|
| 1325 |
'<div class="question">Discover sparse features, test robustness, and separate correlation from causal influence.</div>'
|
| 1326 |
'</header>'
|
|
|
|
| 1509 |
dose_target_text = gr.Textbox(
|
| 1510 |
label="Dose-response target continuation",
|
| 1511 |
value="2x",
|
| 1512 |
+
info="Exact continuation scored across all dose-response conditions.",
|
| 1513 |
scale=2,
|
| 1514 |
)
|
| 1515 |
gr.Markdown(
|
| 1516 |
+
"0× = ablation, 1× = numerical no-edit reference, and 2× = double the native coefficient. "
|
| 1517 |
+
"The six scale conditions are evaluated together against the 1× row."
|
|
|
|
|
|
|
| 1518 |
)
|
| 1519 |
dose_btn = gr.Button("Run scale dose-response", variant="primary", elem_classes=["action-btn"])
|
| 1520 |
dose_metrics = gr.Markdown()
|
|
|
|
| 1846 |
)
|
| 1847 |
candidate_use_status = gr.Markdown()
|
| 1848 |
|
| 1849 |
+
gr.HTML('<div class="section-rule">B. Batched causal candidate triage</div>')
|
| 1850 |
+
gr.Markdown(
|
| 1851 |
+
"Use discovery candidates as a short list, then ablate several of them in **one batched scoring run**. "
|
| 1852 |
+
"This ranks target effects cheaply before you spend a full 8-random-control causal test on a feature. "
|
| 1853 |
+
"Uses the current Workbench prompt/token and the discovery layer."
|
| 1854 |
+
)
|
| 1855 |
+
with gr.Row(equal_height=True):
|
| 1856 |
+
candidate_screen_ids = gr.Dropdown(
|
| 1857 |
+
choices=[],
|
| 1858 |
+
value=[],
|
| 1859 |
+
multiselect=True,
|
| 1860 |
+
allow_custom_value=True,
|
| 1861 |
+
max_choices=8,
|
| 1862 |
+
label="Candidate features to screen",
|
| 1863 |
+
info="Populated by concept-guided discovery; up to eight features per batch.",
|
| 1864 |
+
scale=3,
|
| 1865 |
+
)
|
| 1866 |
+
candidate_screen_target = gr.Textbox(
|
| 1867 |
+
label="Screen target continuation",
|
| 1868 |
+
value="2x",
|
| 1869 |
+
info="Exact continuation used only for this screening run.",
|
| 1870 |
+
scale=2,
|
| 1871 |
+
)
|
| 1872 |
+
candidate_screen_btn = gr.Button(
|
| 1873 |
+
"Screen candidate ablations", variant="primary", elem_classes=["action-btn"]
|
| 1874 |
+
)
|
| 1875 |
+
candidate_screen_metrics = gr.Markdown()
|
| 1876 |
+
with gr.Row(equal_height=False):
|
| 1877 |
+
with gr.Column(scale=3):
|
| 1878 |
+
_table_heading('Candidate ablation screen')
|
| 1879 |
+
candidate_screen_table = gr.Dataframe(
|
| 1880 |
+
interactive=False,
|
| 1881 |
+
label="Candidate ablation screen",
|
| 1882 |
+
show_label=False,
|
| 1883 |
+
buttons=["fullscreen"],
|
| 1884 |
+
elem_classes=["result-table"],
|
| 1885 |
+
wrap=False,
|
| 1886 |
+
max_height=380,
|
| 1887 |
+
)
|
| 1888 |
+
candidate_screen_tsv = gr.Textbox(visible="hidden")
|
| 1889 |
+
candidate_screen_copy = _copy_button()
|
| 1890 |
+
with gr.Column(scale=2):
|
| 1891 |
+
candidate_screen_plot = gr.BarPlot(
|
| 1892 |
+
x="Feature",
|
| 1893 |
+
y="Δ mean log p/token",
|
| 1894 |
+
color="Series",
|
| 1895 |
+
color_map={"Candidate ablation": INK_TEAL},
|
| 1896 |
+
title="Candidate ablation target effect",
|
| 1897 |
+
elem_id="plot-candidate-causal-screen",
|
| 1898 |
+
x_title="Feature id",
|
| 1899 |
+
y_title="Δ mean log p/token",
|
| 1900 |
+
x_label_angle=-35,
|
| 1901 |
+
buttons=["fullscreen", "export"],
|
| 1902 |
+
elem_classes=["fl-plot"],
|
| 1903 |
+
height=330,
|
| 1904 |
+
)
|
| 1905 |
+
gr.Markdown(
|
| 1906 |
+
"Click a row in **Candidate ablation screen** to move that feature into **Selected candidate feature id**, "
|
| 1907 |
+
"then use the existing handoff button above if you want it copied into the deeper feature tests.",
|
| 1908 |
+
elem_classes=["candidate-help"],
|
| 1909 |
+
)
|
| 1910 |
+
|
| 1911 |
+
gr.HTML('<div class="section-rule">C. Inspect one feature</div>')
|
| 1912 |
contrast_location = gr.Markdown(
|
| 1913 |
"**Activation trace uses the current Workbench prompt.** The controlled concept scan below uses its own balanced prompt set."
|
| 1914 |
)
|
|
|
|
| 1967 |
height=320,
|
| 1968 |
)
|
| 1969 |
|
| 1970 |
+
gr.HTML('<div class="section-rule">D. Completion-cue sensitivity</div>')
|
| 1971 |
gr.Markdown(
|
| 1972 |
"Feature 22632 in the current math example fires only on the final `is` token. This scan tests whether a "
|
| 1973 |
"selected feature responds to **completion cues themselves** rather than the underlying concept. Enter a stem "
|
|
|
|
| 1991 |
buttons=["fullscreen", "export"], elem_classes=["fl-plot"], height=320
|
| 1992 |
)
|
| 1993 |
|
| 1994 |
+
gr.HTML('<div class="section-rule">E. Cue × context specificity</div>')
|
| 1995 |
gr.Markdown(
|
| 1996 |
"The single-stem cue scan can tell you that a feature prefers `is`, but not whether that preference is lexical "
|
| 1997 |
"or context-sensitive. This matrix crosses several prompt stems with the same completion cues in one batch."
|
|
|
|
| 2046 |
height=340,
|
| 2047 |
)
|
| 2048 |
|
| 2049 |
+
gr.HTML('<div class="section-rule">F. Controlled concept contrast for selected feature</div>')
|
| 2050 |
gr.Markdown("### Prompt-wide controlled concept contrast")
|
| 2051 |
contrast_btn = gr.Button("Run controlled concept contrast", variant="primary", elem_classes=["action-btn"])
|
| 2052 |
contrast_metrics = gr.Markdown()
|
|
|
|
| 2223 |
|
| 2224 |
1. SAE reconstruction quality.
|
| 2225 |
2. Held-out feature/concept prediction.
|
| 2226 |
+
3. Concept-guided candidate discovery and batched causal candidate triage.
|
| 2227 |
+
4. Selected-feature concept contrast and token-local activation traces, completion-cue sensitivity, and cue × context specificity.
|
| 2228 |
5. Local and prompt-wide paraphrase robustness.
|
| 2229 |
6. Single-feature causal intervention and dose-response.
|
| 2230 |
7. Contrastive continuation preference under intervention.
|
|
|
|
| 2323 |
discovery_concept, discovery_layer, discovery_n, discovery_top_n, discovery_ranking,
|
| 2324 |
prompt, token_index,
|
| 2325 |
],
|
| 2326 |
+
outputs=[
|
| 2327 |
+
discovery_metrics,
|
| 2328 |
+
discovery_table,
|
| 2329 |
+
discovery_plot,
|
| 2330 |
+
discovery_candidate,
|
| 2331 |
+
candidate_screen_ids,
|
| 2332 |
+
discovery_tsv,
|
| 2333 |
+
],
|
| 2334 |
+
)
|
| 2335 |
+
candidate_screen_btn.click(
|
| 2336 |
+
run_candidate_causal_screen,
|
| 2337 |
+
inputs=[prompt, discovery_layer, token_index, candidate_screen_ids, candidate_screen_target],
|
| 2338 |
+
outputs=[
|
| 2339 |
+
candidate_screen_metrics,
|
| 2340 |
+
candidate_screen_table,
|
| 2341 |
+
candidate_screen_plot,
|
| 2342 |
+
discovery_candidate,
|
| 2343 |
+
candidate_screen_tsv,
|
| 2344 |
+
],
|
| 2345 |
+
)
|
| 2346 |
+
candidate_screen_table.select(
|
| 2347 |
+
select_candidate_row,
|
| 2348 |
+
inputs=[candidate_screen_table],
|
| 2349 |
+
outputs=[discovery_candidate],
|
| 2350 |
+
queue=False,
|
| 2351 |
)
|
| 2352 |
discovery_table.select(
|
| 2353 |
select_candidate_row,
|
|
|
|
| 2396 |
(trace_copy, trace_tsv),
|
| 2397 |
(contrast_copy, contrast_tsv),
|
| 2398 |
(discovery_copy, discovery_tsv),
|
| 2399 |
+
(candidate_screen_copy, candidate_screen_tsv),
|
| 2400 |
(cue_copy, cue_tsv),
|
| 2401 |
(cue_context_copy, cue_context_tsv),
|
| 2402 |
(para_copy, para_tsv),
|
docs/METHODOLOGY.md
CHANGED
|
@@ -417,3 +417,25 @@ The log factor makes current-token presence matter without allowing a single ver
|
|
| 417 |
The cue × context matrix now summarizes the measured activation pattern rather than always returning generic interpretation text. For each cue, FeatureLens counts the number of tested contexts in which the feature is active and computes mean activation across those contexts.
|
| 418 |
|
| 419 |
A particularly strong tested cue-specific pattern occurs when one cue activates in every tested context and all other tested cues remain inactive. FeatureLens describes that pattern as **cue-dominant under the tested matrix**. The wording is intentionally local to the controlled stems and cues; it does not assert a universal semantic label for the SAE feature.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 417 |
The cue × context matrix now summarizes the measured activation pattern rather than always returning generic interpretation text. For each cue, FeatureLens counts the number of tested contexts in which the feature is active and computes mean activation across those contexts.
|
| 418 |
|
| 419 |
A particularly strong tested cue-specific pattern occurs when one cue activates in every tested context and all other tested cues remain inactive. FeatureLens describes that pattern as **cue-dominant under the tested matrix**. The wording is intentionally local to the controlled stems and cues; it does not assert a universal semantic label for the SAE feature.
|
| 420 |
+
|
| 421 |
+
## v0.9 batched causal candidate triage
|
| 422 |
+
|
| 423 |
+
Concept-guided discovery and causal testing answer different questions. A feature can be selective for a controlled concept but have no activation at the selected Workbench token; conversely, a current-token-active feature may be causally irrelevant for the continuation under study.
|
| 424 |
+
|
| 425 |
+
v0.9 therefore inserts a low-cost triage stage between discovery and the full random-control causal test. For up to eight candidate features active or inactive at the current location, FeatureLens constructs the native ablation
|
| 426 |
+
|
| 427 |
+
$$
|
| 428 |
+
\Delta h_i = -z_i d_i
|
| 429 |
+
$$
|
| 430 |
+
|
| 431 |
+
for each candidate feature $i$, stacks a zero-edit reference plus all candidate deltas along the batch dimension, and teacher-forces the same target continuation for every condition in one model forward. The screen reports:
|
| 432 |
+
|
| 433 |
+
- native feature activation;
|
| 434 |
+
- perturbation L2 norm;
|
| 435 |
+
- target mean log-probability delta per token;
|
| 436 |
+
- target sequence log-probability delta;
|
| 437 |
+
- next-token Jensen-Shannon divergence.
|
| 438 |
+
|
| 439 |
+
Rows are ordered by absolute target mean-log-probability effect. This ordering is deliberately a **native-ablation effect screen**, not a significance or specificity statistic. No random-control ensemble is used at this stage. A promising candidate should be promoted to the existing single-feature causal test, which compares the SAE edit with the eight-direction norm-matched random ensemble.
|
| 440 |
+
|
| 441 |
+
This two-stage design reduces live GPU use while preserving the stronger causal standard for any result that is ultimately interpreted.
|
docs/VALIDATION.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
-
# FeatureLens v0.
|
| 2 |
|
| 3 |
-
This guide
|
| 4 |
|
| 5 |
-
##
|
| 6 |
|
| 7 |
From the repository root:
|
| 8 |
|
|
@@ -14,238 +14,141 @@ python3 scripts/ui_smoke.py && \
|
|
| 14 |
python3 scripts/release_check.py
|
| 15 |
```
|
| 16 |
|
| 17 |
-
Expected
|
| 18 |
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
layers: [4, 14, 26]
|
| 24 |
-
feature-set sizes: [1, 3, 5]
|
| 25 |
-
random controls: 8
|
| 26 |
-
release: v0.8.0
|
| 27 |
-
```
|
| 28 |
-
|
| 29 |
-
## 1. Establish the canonical Workbench context
|
| 30 |
-
|
| 31 |
-
Open **Workbench → I. Inspect a prompt location** and set:
|
| 32 |
-
|
| 33 |
-
```text
|
| 34 |
-
Prompt: The derivative of x squared is
|
| 35 |
-
Residual layer: 14
|
| 36 |
-
Prompt token index: -1
|
| 37 |
-
Displayed active features: 12
|
| 38 |
-
```
|
| 39 |
-
|
| 40 |
-
Click **Inspect sparse features**.
|
| 41 |
-
|
| 42 |
-
Pass conditions:
|
| 43 |
-
- **Current Workbench context** reports layer 14 and token 5 (`' is'`).
|
| 44 |
-
- **Strongest active SAE features** begins with feature `22632` at about `32.90625`.
|
| 45 |
-
- Result-table titles such as **Strongest active SAE features** are visually larger than table body text and are not rendered as tiny native component labels.
|
| 46 |
-
|
| 47 |
-
## 2. Plot focus-view regression
|
| 48 |
-
|
| 49 |
-
On **Activation profile**, click the top-right fullscreen/focus icon.
|
| 50 |
-
|
| 51 |
-
Pass conditions:
|
| 52 |
-
- FeatureLens opens a darkened overlay with a centered chart rather than Gradio's stretched native fullscreen view.
|
| 53 |
-
- The chart reading surface is bounded to roughly desktop-document width; it should not require multiple horizontal scrolls.
|
| 54 |
-
- The focused chart starts at its own top.
|
| 55 |
-
- Press **Close**, click the backdrop, or press **Esc**.
|
| 56 |
-
- The page returns to exactly the same Workbench scroll position.
|
| 57 |
-
|
| 58 |
-
Then click the export icon.
|
| 59 |
-
|
| 60 |
-
Pass condition:
|
| 61 |
-
- the filename is descriptive (for example `featurelens_activation-profile.png`), not `chart.png`.
|
| 62 |
-
|
| 63 |
-
Repeat the focus-view check on one long-label plot such as **Cue response across contexts**.
|
| 64 |
-
|
| 65 |
-
## 3. Standalone scale dose-response
|
| 66 |
|
| 67 |
-
|
| 68 |
|
| 69 |
-
|
| 70 |
|
| 71 |
-
|
| 72 |
-
Dose-response feature id: 22632
|
| 73 |
-
Dose-response target continuation: 2x
|
| 74 |
-
```
|
| 75 |
-
|
| 76 |
-
Click **Run scale dose-response**.
|
| 77 |
|
| 78 |
-
|
| 79 |
-
- The experiment runs without using the optional target field from Section II.
|
| 80 |
-
- No `Enter a target continuation...` error occurs.
|
| 81 |
-
- The 1× row is the exact numerical null:
|
| 82 |
-
- Δ feature coefficient = 0
|
| 83 |
-
- Perturbation L2 = 0
|
| 84 |
-
- Δ mean log p/token = 0
|
| 85 |
-
- Δ sequence log p = 0
|
| 86 |
-
- Next-token JS = 0
|
| 87 |
|
| 88 |
-
|
|
|
|
|
|
|
| 89 |
|
| 90 |
-
|
| 91 |
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
```text
|
| 95 |
-
Target concept: mathematics
|
| 96 |
-
Residual layer: 14
|
| 97 |
-
Prompts per concept: 4
|
| 98 |
-
Candidate features: 12
|
| 99 |
-
Candidate ranking: Balanced selectivity
|
| 100 |
-
```
|
| 101 |
|
| 102 |
-
|
| 103 |
|
| 104 |
-
|
| 105 |
-
- **Candidate feature evidence** has an explicit readable heading.
|
| 106 |
-
- The result summary states how many displayed candidates are active at the selected Workbench token.
|
| 107 |
-
- The ranking is selectivity-oriented rather than dominated by giant globally active features.
|
| 108 |
|
| 109 |
-
|
| 110 |
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
``
|
| 116 |
-
Candidate ranking: Causal-ready at current token
|
| 117 |
-
```
|
| 118 |
|
| 119 |
-
Click
|
| 120 |
|
| 121 |
-
|
| 122 |
-
- Every returned row has **Active at current token = True**.
|
| 123 |
-
- The result summary reports `displayed/current-active = N/N`.
|
| 124 |
-
- If no positively selective current-token-active feature exists, the UI reports that clearly instead of silently falling back to a prompt-wide candidate.
|
| 125 |
|
| 126 |
-
|
| 127 |
|
| 128 |
-
|
|
|
|
|
|
|
|
|
|
| 129 |
|
| 130 |
-
|
| 131 |
|
| 132 |
-
|
| 133 |
-
Candidate ranking: Raw mean difference
|
| 134 |
-
```
|
| 135 |
|
| 136 |
-
|
| 137 |
|
| 138 |
-
|
| 139 |
-
-
|
| 140 |
-
-
|
|
|
|
|
|
|
| 141 |
|
| 142 |
-
##
|
| 143 |
|
| 144 |
-
|
| 145 |
|
| 146 |
-
|
| 147 |
|
| 148 |
-
|
|
|
|
|
|
|
| 149 |
|
| 150 |
-
|
| 151 |
-
- the status message names the selected feature;
|
| 152 |
-
- the feature propagates to:
|
| 153 |
-
- **Single feature id**
|
| 154 |
-
- **Dose-response feature id**
|
| 155 |
-
- **Contrastive feature id**
|
| 156 |
-
- **Feature evidence → Feature id**
|
| 157 |
-
- no GPU job is launched by the handoff action itself.
|
| 158 |
|
| 159 |
-
|
| 160 |
|
| 161 |
-
|
| 162 |
|
| 163 |
-
|
| 164 |
-
Feature id: 22632
|
| 165 |
-
Residual layer: 14
|
| 166 |
-
Prompt stems:
|
| 167 |
-
The derivative of x squared
|
| 168 |
-
The capital of Germany
|
| 169 |
-
The weather today
|
| 170 |
-
My name
|
| 171 |
|
| 172 |
-
|
| 173 |
-
is
|
| 174 |
-
=
|
| 175 |
-
:
|
| 176 |
-
equals
|
| 177 |
-
therefore
|
| 178 |
-
```
|
| 179 |
|
| 180 |
-
|
| 181 |
|
| 182 |
-
|
| 183 |
-
- summary explicitly calls the tested matrix a **cue-dominant pattern**;
|
| 184 |
-
- `is: 4/4 contexts`;
|
| 185 |
-
- all off-cue conditions remain inactive;
|
| 186 |
-
- the plot uses the restrained FeatureLens cue palette rather than saturated default blue/orange/red/green colors.
|
| 187 |
|
| 188 |
-
|
| 189 |
|
| 190 |
-
|
| 191 |
|
| 192 |
-
|
| 193 |
|
| 194 |
-
|
| 195 |
|
| 196 |
-
|
| 197 |
-
The derivative of x squared is
|
| 198 |
-
```
|
| 199 |
|
| 200 |
-
|
| 201 |
|
| 202 |
-
|
| 203 |
-
-
|
| 204 |
-
-
|
| 205 |
-
-
|
| 206 |
-
-
|
|
|
|
|
|
|
|
|
|
| 207 |
|
| 208 |
-
|
| 209 |
|
| 210 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 211 |
|
| 212 |
-
|
| 213 |
-
Prompt: The derivative of x squared is
|
| 214 |
-
Prompt token index: -1
|
| 215 |
-
```
|
| 216 |
|
| 217 |
-
|
| 218 |
-
- layer 4 cosine ≈ 0.962
|
| 219 |
-
- layer 14 cosine ≈ 0.902
|
| 220 |
-
- layer 26 cosine ≈ 0.961
|
| 221 |
|
| 222 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 223 |
|
| 224 |
-
|
| 225 |
|
| 226 |
-
|
| 227 |
|
| 228 |
-
|
| 229 |
-
Target continuation for set-size sweep: 2x
|
| 230 |
-
```
|
| 231 |
|
| 232 |
-
|
| 233 |
|
| 234 |
-
|
| 235 |
-
-
|
| 236 |
-
-
|
| 237 |
-
- all conditions share the batched null reference;
|
| 238 |
-
- no regression to a single arbitrary random control.
|
| 239 |
|
| 240 |
-
|
| 241 |
|
| 242 |
-
|
| 243 |
-
1. whether the bounded plot focus view finally solves the ultrawide/stretch problem;
|
| 244 |
-
2. whether closing focus returns to the same page position;
|
| 245 |
-
3. whether table headings are finally readable;
|
| 246 |
-
4. the **Causal-ready at current token** candidate table (or the explicit no-candidate message);
|
| 247 |
-
5. the cue × context summary for `22632`;
|
| 248 |
-
6. whether standalone dose response works without Section II;
|
| 249 |
-
7. any unexpected regression from 9a–9c.
|
| 250 |
|
| 251 |
-
The comprehensive
|
|
|
|
| 1 |
+
# FeatureLens v0.9 validation
|
| 2 |
|
| 3 |
+
This guide intentionally **does not** ask you to rerun unchanged GPU-heavy regressions. Identity paraphrase, layer trajectory, and the 1/3/5 feature-set sweep were unchanged in v0.9 and remain covered by automated tests. Save the HF ZeroGPU quota for the two inference paths changed/added here.
|
| 4 |
|
| 5 |
+
## 1. Local release gate — no HF GPU
|
| 6 |
|
| 7 |
From the repository root:
|
| 8 |
|
|
|
|
| 14 |
python3 scripts/release_check.py
|
| 15 |
```
|
| 16 |
|
| 17 |
+
Expected high-level result:
|
| 18 |
|
| 19 |
+
- all tests pass;
|
| 20 |
+
- Ruff reports `All checks passed!`;
|
| 21 |
+
- `FeatureLens UI launch smoke: PASS`;
|
| 22 |
+
- release check ends with `release: v0.9.0`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
+
If any command fails, stop there rather than running HF tests.
|
| 25 |
|
| 26 |
+
---
|
| 27 |
|
| 28 |
+
# HF acceptance — only two GPU calls required
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
+
Keep the default Workbench prompt unless stated otherwise:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
+
- **Workbench → Prompt:** `The derivative of x squared is`
|
| 33 |
+
- **Workbench → Residual layer:** `14`
|
| 34 |
+
- **Workbench → Prompt token index:** `-1`
|
| 35 |
|
| 36 |
+
You do **not** need to run **Inspect sparse features** first for these v0.9 checks.
|
| 37 |
|
| 38 |
+
## 2. GPU call 1 — causal-ready discovery + layout/focus test
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
+
Go to:
|
| 41 |
|
| 42 |
+
**Feature evidence → A. Concept-guided candidate discovery**
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
+
Set exactly:
|
| 45 |
|
| 46 |
+
- **Target concept:** `mathematics`
|
| 47 |
+
- **Residual layer:** `14`
|
| 48 |
+
- **Prompts per concept:** `4`
|
| 49 |
+
- **Candidate features:** `12`
|
| 50 |
+
- **Candidate ranking:** `Causal-ready at current token`
|
|
|
|
|
|
|
| 51 |
|
| 52 |
+
Click:
|
| 53 |
|
| 54 |
+
**Discover concept-associated candidates**
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
+
### Pass conditions
|
| 57 |
|
| 58 |
+
1. **Candidate feature evidence** is visually close to the table itself. There should not be a large empty band between the heading and the first table row/header.
|
| 59 |
+
2. **Candidate features to screen** in section B is automatically populated with up to the first five returned candidate IDs.
|
| 60 |
+
3. The discovery summary says all displayed candidates are active at the selected Workbench token for this ranking.
|
| 61 |
+
4. No page jump occurs merely because results render.
|
| 62 |
|
| 63 |
+
### In-place plot focus
|
| 64 |
|
| 65 |
+
On the **Candidate evidence score** graph, click the existing fullscreen/focus toolbar icon.
|
|
|
|
|
|
|
| 66 |
|
| 67 |
+
Pass if:
|
| 68 |
|
| 69 |
+
- the chart expands **where it already sits on the page**;
|
| 70 |
+
- it does not jump to the top of the HF Space;
|
| 71 |
+
- it remains aspect-ratio preserving rather than becoming extremely tall/long;
|
| 72 |
+
- it does not require horizontal page traversal;
|
| 73 |
+
- clicking the same icon again (or pressing `Esc`) returns it to the normal size without changing your surrounding page location.
|
| 74 |
|
| 75 |
+
### In-place table focus
|
| 76 |
|
| 77 |
+
Click the fullscreen/focus icon on **Candidate feature evidence**.
|
| 78 |
|
| 79 |
+
Pass if:
|
| 80 |
|
| 81 |
+
- the table widens at its current location rather than opening at the top of the Space;
|
| 82 |
+
- the surrounding page position is unchanged after toggling back;
|
| 83 |
+
- horizontal scrolling stays inside the table when needed.
|
| 84 |
|
| 85 |
+
Do not rerun discovery merely to test focus. Use this already-populated output for both focus checks.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
|
| 87 |
+
---
|
| 88 |
|
| 89 |
+
## 3. GPU call 2 — batched causal candidate triage
|
| 90 |
|
| 91 |
+
Stay in:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
|
| 93 |
+
**Feature evidence → B. Batched causal candidate triage**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 94 |
|
| 95 |
+
The discovery run should already have populated **Candidate features to screen**. Leave the default first five candidates selected unless fewer than five were returned.
|
| 96 |
|
| 97 |
+
Set:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
+
- **Screen target continuation:** `2x`
|
| 100 |
|
| 101 |
+
Click:
|
| 102 |
|
| 103 |
+
**Screen candidate ablations**
|
| 104 |
|
| 105 |
+
### Expected output
|
| 106 |
|
| 107 |
+
The summary should explicitly call this a **triage screen** and explain that the full random-control ensemble is deferred to **Workbench → II. Single-feature causal test**.
|
|
|
|
|
|
|
| 108 |
|
| 109 |
+
The **Candidate ablation screen** table must contain these columns:
|
| 110 |
|
| 111 |
+
- `Rank`
|
| 112 |
+
- `Feature id`
|
| 113 |
+
- `Native activation`
|
| 114 |
+
- `Active at current token`
|
| 115 |
+
- `Perturbation L2`
|
| 116 |
+
- `Δ mean log p/token`
|
| 117 |
+
- `Δ sequence log p`
|
| 118 |
+
- `Next-token JS`
|
| 119 |
|
| 120 |
+
Pass if:
|
| 121 |
|
| 122 |
+
- all selected causal-ready candidates report `Active at current token = True`;
|
| 123 |
+
- rows are ordered from the largest absolute `Δ mean log p/token` to the smallest;
|
| 124 |
+
- the 1st-ranked screen result is automatically placed into **Selected candidate feature id**;
|
| 125 |
+
- clicking another row updates **Selected candidate feature id** without a GPU call;
|
| 126 |
+
- the plot and table focus controls behave in-place as in Test 2.
|
| 127 |
|
| 128 |
+
### What to send back
|
|
|
|
|
|
|
|
|
|
| 129 |
|
| 130 |
+
For v0.9 I only need:
|
|
|
|
|
|
|
|
|
|
| 131 |
|
| 132 |
+
1. the **Candidate ablation screen** table;
|
| 133 |
+
2. the triage summary;
|
| 134 |
+
3. whether plot focus passed;
|
| 135 |
+
4. whether table focus passed;
|
| 136 |
+
5. whether heading/table spacing now looks natural.
|
| 137 |
|
| 138 |
+
No paraphrase, layer-trajectory, feature-set-sweep, or dose-response reruns are required for v0.9 acceptance because those inference implementations were not changed.
|
| 139 |
|
| 140 |
+
---
|
| 141 |
|
| 142 |
+
## 4. Zero-GPU visual checks
|
|
|
|
|
|
|
| 143 |
|
| 144 |
+
These do not require running an experiment:
|
| 145 |
|
| 146 |
+
- **Workbench → III. Single-feature scale dose-response** should show its own **Dose-response feature id** and **Dose-response target continuation** fields without the old “Standalone experiment” explanation.
|
| 147 |
+
- The short explanatory line should only define the 0× / 1× / 2× scale semantics.
|
| 148 |
+
- Existing result-table headings should use the same compact spacing pattern throughout the app.
|
|
|
|
|
|
|
| 149 |
|
| 150 |
+
---
|
| 151 |
|
| 152 |
+
# Final hardening later
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
|
| 154 |
+
The comprehensive adversarial suite (invalid feature IDs, long prompts, inactive-feature edge cases, rapid double clicks, mixed-language prompts, etc.) remains deferred until the live feature set is frozen. That final pass should be done once, not on every iteration.
|
featurelens/runtime.py
CHANGED
|
@@ -243,6 +243,18 @@ class ConceptFeatureDiscoveryResult:
|
|
| 243 |
displayed_current_active_count: int
|
| 244 |
|
| 245 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 246 |
@dataclass
|
| 247 |
class FeatureCueScanResult:
|
| 248 |
feature_id: int
|
|
@@ -1991,6 +2003,134 @@ class FeatureLensRuntime:
|
|
| 1991 |
displayed_current_active_count=sum(bool(row[-1]) for row in table_rows),
|
| 1992 |
)
|
| 1993 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1994 |
@staticmethod
|
| 1995 |
def _cue_prompt(stem: str, cue: str) -> str:
|
| 1996 |
stem = stem.rstrip()
|
|
|
|
| 243 |
displayed_current_active_count: int
|
| 244 |
|
| 245 |
|
| 246 |
+
@dataclass
|
| 247 |
+
class CandidateCausalScreenResult:
|
| 248 |
+
feature_ids: list[int]
|
| 249 |
+
target_tokens: list[str]
|
| 250 |
+
rows: list[list[object]]
|
| 251 |
+
chart_rows: list[list[object]]
|
| 252 |
+
active_feature_count: int
|
| 253 |
+
candidate_count: int
|
| 254 |
+
execution_drift_mean_logprob: float
|
| 255 |
+
execution_drift_js: float
|
| 256 |
+
|
| 257 |
+
|
| 258 |
@dataclass
|
| 259 |
class FeatureCueScanResult:
|
| 260 |
feature_id: int
|
|
|
|
| 2003 |
displayed_current_active_count=sum(bool(row[-1]) for row in table_rows),
|
| 2004 |
)
|
| 2005 |
|
| 2006 |
+
@torch.inference_mode()
|
| 2007 |
+
def candidate_causal_screen(
|
| 2008 |
+
self,
|
| 2009 |
+
text: str,
|
| 2010 |
+
layer: int,
|
| 2011 |
+
token_index: int,
|
| 2012 |
+
feature_ids: Sequence[int],
|
| 2013 |
+
target_text: str,
|
| 2014 |
+
) -> CandidateCausalScreenResult:
|
| 2015 |
+
"""Cheaply triage several candidate features with one batched ablation screen.
|
| 2016 |
+
|
| 2017 |
+
This deliberately omits random controls. Its purpose is to rank candidates before
|
| 2018 |
+
spending a full live causal test (with the random-control ensemble) on one or two
|
| 2019 |
+
promising features. All feature ablations share the same batched zero-edit reference.
|
| 2020 |
+
"""
|
| 2021 |
+
if not text.strip():
|
| 2022 |
+
raise ValueError('Enter and inspect a Workbench prompt first.')
|
| 2023 |
+
if not target_text.strip():
|
| 2024 |
+
raise ValueError('Enter a target continuation for candidate causal screening.')
|
| 2025 |
+
ids = list(dict.fromkeys(int(x) for x in feature_ids))
|
| 2026 |
+
if not ids:
|
| 2027 |
+
raise ValueError('Select at least one candidate feature to screen.')
|
| 2028 |
+
if len(ids) > 8:
|
| 2029 |
+
raise ValueError('Candidate causal screening supports at most eight features per run.')
|
| 2030 |
+
if any(feature_id < 0 or feature_id >= self.settings.sae_width for feature_id in ids):
|
| 2031 |
+
raise ValueError(f'Feature ids must be in [0, {self.settings.sae_width - 1}].')
|
| 2032 |
+
|
| 2033 |
+
self.ensure_ready(preload_saes=False)
|
| 2034 |
+
assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
|
| 2035 |
+
prompt_inputs = self._inputs(text)
|
| 2036 |
+
prompt_len = int(prompt_inputs['input_ids'].shape[1])
|
| 2037 |
+
idx = self._resolve_index(int(token_index), prompt_len)
|
| 2038 |
+
target_ids = self._target_ids(target_text)
|
| 2039 |
+
full_inputs = self._append_target(prompt_inputs, target_ids)
|
| 2040 |
+
sae = self.sae_store.get(int(layer))
|
| 2041 |
+
|
| 2042 |
+
capture: dict = {}
|
| 2043 |
+
with self._capture_hook(int(layer), capture):
|
| 2044 |
+
single_baseline_out = self.model(**full_inputs, use_cache=False)
|
| 2045 |
+
single_logits = single_baseline_out.logits[0]
|
| 2046 |
+
_, single_mean, _ = sequence_logprob_summary(
|
| 2047 |
+
single_logits, prompt_length=prompt_len, target_ids=target_ids
|
| 2048 |
+
)
|
| 2049 |
+
|
| 2050 |
+
residual = capture['hidden'][0, idx]
|
| 2051 |
+
encoding = sae.encode(residual)
|
| 2052 |
+
activations = [float(encoding.activation_for(feature_id)) for feature_id in ids]
|
| 2053 |
+
deltas = [
|
| 2054 |
+
residual_delta(
|
| 2055 |
+
sae.decoder_direction(feature_id),
|
| 2056 |
+
activation,
|
| 2057 |
+
InterventionSpec('ablate', 0.0),
|
| 2058 |
+
)
|
| 2059 |
+
for feature_id, activation in zip(ids, activations, strict=True)
|
| 2060 |
+
]
|
| 2061 |
+
all_deltas = torch.stack([torch.zeros_like(deltas[0]), *deltas], dim=0)
|
| 2062 |
+
repeated = self._repeat_inputs(full_inputs, all_deltas.shape[0])
|
| 2063 |
+
with self._batch_delta_hook(int(layer), idx, all_deltas):
|
| 2064 |
+
outputs = self.model(**repeated, use_cache=False)
|
| 2065 |
+
|
| 2066 |
+
baseline_logits = outputs.logits[0]
|
| 2067 |
+
baseline_seq, baseline_mean, _ = sequence_logprob_summary(
|
| 2068 |
+
baseline_logits, prompt_length=prompt_len, target_ids=target_ids
|
| 2069 |
+
)
|
| 2070 |
+
baseline_next = baseline_logits[prompt_len - 1]
|
| 2071 |
+
execution_drift_mean = float(baseline_mean - single_mean)
|
| 2072 |
+
execution_drift_js = js_divergence_from_logits(
|
| 2073 |
+
single_logits[prompt_len - 1], baseline_next
|
| 2074 |
+
)
|
| 2075 |
+
|
| 2076 |
+
scored: list[dict[str, object]] = []
|
| 2077 |
+
for row_idx, (feature_id, activation, delta) in enumerate(
|
| 2078 |
+
zip(ids, activations, deltas, strict=True), start=1
|
| 2079 |
+
):
|
| 2080 |
+
logits = outputs.logits[row_idx]
|
| 2081 |
+
seq_logp, mean_logp, _ = sequence_logprob_summary(
|
| 2082 |
+
logits, prompt_length=prompt_len, target_ids=target_ids
|
| 2083 |
+
)
|
| 2084 |
+
mean_delta = float(mean_logp - baseline_mean)
|
| 2085 |
+
seq_delta = float(seq_logp - baseline_seq)
|
| 2086 |
+
js = float(js_divergence_from_logits(baseline_next, logits[prompt_len - 1]))
|
| 2087 |
+
norm = float(torch.linalg.vector_norm(delta.float()).item())
|
| 2088 |
+
scored.append(
|
| 2089 |
+
{
|
| 2090 |
+
'feature_id': int(feature_id),
|
| 2091 |
+
'activation': float(activation),
|
| 2092 |
+
'active': bool(activation > 0),
|
| 2093 |
+
'norm': norm,
|
| 2094 |
+
'mean_delta': mean_delta,
|
| 2095 |
+
'seq_delta': seq_delta,
|
| 2096 |
+
'js': js,
|
| 2097 |
+
}
|
| 2098 |
+
)
|
| 2099 |
+
|
| 2100 |
+
scored.sort(
|
| 2101 |
+
key=lambda item: (abs(float(item['mean_delta'])), float(item['js'])),
|
| 2102 |
+
reverse=True,
|
| 2103 |
+
)
|
| 2104 |
+
rows: list[list[object]] = []
|
| 2105 |
+
chart_rows: list[list[object]] = []
|
| 2106 |
+
for rank, item in enumerate(scored, start=1):
|
| 2107 |
+
feature_id = int(item['feature_id'])
|
| 2108 |
+
mean_delta = float(item['mean_delta'])
|
| 2109 |
+
rows.append(
|
| 2110 |
+
[
|
| 2111 |
+
rank,
|
| 2112 |
+
feature_id,
|
| 2113 |
+
float(item['activation']),
|
| 2114 |
+
bool(item['active']),
|
| 2115 |
+
float(item['norm']),
|
| 2116 |
+
mean_delta,
|
| 2117 |
+
float(item['seq_delta']),
|
| 2118 |
+
float(item['js']),
|
| 2119 |
+
]
|
| 2120 |
+
)
|
| 2121 |
+
chart_rows.append([str(feature_id), mean_delta])
|
| 2122 |
+
|
| 2123 |
+
return CandidateCausalScreenResult(
|
| 2124 |
+
feature_ids=[int(item['feature_id']) for item in scored],
|
| 2125 |
+
target_tokens=[self.tokenizer.decode([int(token_id)]) for token_id in target_ids],
|
| 2126 |
+
rows=rows,
|
| 2127 |
+
chart_rows=chart_rows,
|
| 2128 |
+
active_feature_count=sum(bool(item['active']) for item in scored),
|
| 2129 |
+
candidate_count=len(scored),
|
| 2130 |
+
execution_drift_mean_logprob=execution_drift_mean,
|
| 2131 |
+
execution_drift_js=float(execution_drift_js),
|
| 2132 |
+
)
|
| 2133 |
+
|
| 2134 |
@staticmethod
|
| 2135 |
def _cue_prompt(stem: str, cue: str) -> str:
|
| 2136 |
stem = stem.rstrip()
|
pyproject.toml
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
[project]
|
| 2 |
name = "featurelens"
|
| 3 |
-
version = "0.
|
| 4 |
description = "Causal sparse-feature interpretability workbench for Qwen3 and Qwen-Scope SAEs"
|
| 5 |
requires-python = ">=3.10"
|
| 6 |
|
|
|
|
| 1 |
[project]
|
| 2 |
name = "featurelens"
|
| 3 |
+
version = "0.9.0"
|
| 4 |
description = "Causal sparse-feature interpretability workbench for Qwen3 and Qwen-Scope SAEs"
|
| 5 |
requires-python = ">=3.10"
|
| 6 |
|
research_config.json
CHANGED
|
@@ -117,5 +117,14 @@
|
|
| 117 |
"causal_ready_current_token_candidate_ranking",
|
| 118 |
"cue_dominance_specificity_interpretation",
|
| 119 |
"muted_cue_context_plot_palette"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
]
|
| 121 |
}
|
|
|
|
| 117 |
"causal_ready_current_token_candidate_ranking",
|
| 118 |
"cue_dominance_specificity_interpretation",
|
| 119 |
"muted_cue_context_plot_palette"
|
| 120 |
+
],
|
| 121 |
+
"candidate_causal_screen_limit": 8,
|
| 122 |
+
"candidate_causal_screen_control": "batched zero-edit reference; no random controls in triage screen",
|
| 123 |
+
"live_features_v0_9": [
|
| 124 |
+
"in_place_aspect_preserving_plot_and_table_focus",
|
| 125 |
+
"compact_table_heading_alignment",
|
| 126 |
+
"concise_independent_dose_response_copy",
|
| 127 |
+
"batched_candidate_causal_triage",
|
| 128 |
+
"gpu_budget_aware_hf_validation_scope"
|
| 129 |
]
|
| 130 |
}
|
scripts/release_check.py
CHANGED
|
@@ -83,6 +83,7 @@ def check_config(config: dict) -> None:
|
|
| 83 |
'interaction_feature_limit': 5,
|
| 84 |
'live_geometry_feature_limit': 8,
|
| 85 |
'concept_contrast_pooling': 'max activation across non-padding prompt tokens',
|
|
|
|
| 86 |
}
|
| 87 |
for key, value in expected.items():
|
| 88 |
if config.get(key) != value:
|
|
@@ -165,6 +166,19 @@ def check_config(config: dict) -> None:
|
|
| 165 |
'research_config.json live_features_v0_8 mismatch: ' f'{sorted(actual_live_v08)}'
|
| 166 |
)
|
| 167 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 168 |
if 'german_language' not in config.get('concepts', []) or 'french_language' in config.get('concepts', []):
|
| 169 |
raise SystemExit('research_config.json must use german_language and must not contain french_language.')
|
| 170 |
|
|
@@ -238,19 +252,20 @@ def check_readme() -> None:
|
|
| 238 |
'german',
|
| 239 |
'start here',
|
| 240 |
'causal-ready',
|
| 241 |
-
'standalone',
|
| 242 |
-
'plot focus',
|
| 243 |
'cue-dominant',
|
|
|
|
|
|
|
|
|
|
| 244 |
]
|
| 245 |
missing = [value for value in required_strings if value.lower() not in readme.lower()]
|
| 246 |
if missing:
|
| 247 |
-
raise SystemExit(f'README.md is missing required v0.
|
| 248 |
|
| 249 |
|
| 250 |
def check_pyproject() -> None:
|
| 251 |
text = (ROOT / 'pyproject.toml').read_text(encoding='utf-8')
|
| 252 |
-
if 'version = "0.
|
| 253 |
-
raise SystemExit('pyproject.toml must declare version 0.
|
| 254 |
|
| 255 |
|
| 256 |
def main() -> None:
|
|
@@ -268,7 +283,7 @@ def main() -> None:
|
|
| 268 |
print(f' layers: {config["layers"]}')
|
| 269 |
print(f' feature-set sizes: {config["feature_set_sizes"]}')
|
| 270 |
print(f' random controls: {config["live_random_controls"]}')
|
| 271 |
-
print(' release: v0.
|
| 272 |
|
| 273 |
|
| 274 |
if __name__ == '__main__':
|
|
|
|
| 83 |
'interaction_feature_limit': 5,
|
| 84 |
'live_geometry_feature_limit': 8,
|
| 85 |
'concept_contrast_pooling': 'max activation across non-padding prompt tokens',
|
| 86 |
+
'candidate_causal_screen_limit': 8,
|
| 87 |
}
|
| 88 |
for key, value in expected.items():
|
| 89 |
if config.get(key) != value:
|
|
|
|
| 166 |
'research_config.json live_features_v0_8 mismatch: ' f'{sorted(actual_live_v08)}'
|
| 167 |
)
|
| 168 |
|
| 169 |
+
required_live_v09 = {
|
| 170 |
+
'in_place_aspect_preserving_plot_and_table_focus',
|
| 171 |
+
'compact_table_heading_alignment',
|
| 172 |
+
'concise_independent_dose_response_copy',
|
| 173 |
+
'batched_candidate_causal_triage',
|
| 174 |
+
'gpu_budget_aware_hf_validation_scope',
|
| 175 |
+
}
|
| 176 |
+
actual_live_v09 = set(config.get('live_features_v0_9', []))
|
| 177 |
+
if actual_live_v09 != required_live_v09:
|
| 178 |
+
raise SystemExit(
|
| 179 |
+
'research_config.json live_features_v0_9 mismatch: ' f'{sorted(actual_live_v09)}'
|
| 180 |
+
)
|
| 181 |
+
|
| 182 |
if 'german_language' not in config.get('concepts', []) or 'french_language' in config.get('concepts', []):
|
| 183 |
raise SystemExit('research_config.json must use german_language and must not contain french_language.')
|
| 184 |
|
|
|
|
| 252 |
'german',
|
| 253 |
'start here',
|
| 254 |
'causal-ready',
|
|
|
|
|
|
|
| 255 |
'cue-dominant',
|
| 256 |
+
'batched causal candidate triage',
|
| 257 |
+
'in-place',
|
| 258 |
+
'gpu',
|
| 259 |
]
|
| 260 |
missing = [value for value in required_strings if value.lower() not in readme.lower()]
|
| 261 |
if missing:
|
| 262 |
+
raise SystemExit(f'README.md is missing required v0.9 content: {missing}')
|
| 263 |
|
| 264 |
|
| 265 |
def check_pyproject() -> None:
|
| 266 |
text = (ROOT / 'pyproject.toml').read_text(encoding='utf-8')
|
| 267 |
+
if 'version = "0.9.0"' not in text:
|
| 268 |
+
raise SystemExit('pyproject.toml must declare version 0.9.0.')
|
| 269 |
|
| 270 |
|
| 271 |
def main() -> None:
|
|
|
|
| 283 |
print(f' layers: {config["layers"]}')
|
| 284 |
print(f' feature-set sizes: {config["feature_set_sizes"]}')
|
| 285 |
print(f' random controls: {config["live_random_controls"]}')
|
| 286 |
+
print(' release: v0.9.0')
|
| 287 |
|
| 288 |
|
| 289 |
if __name__ == '__main__':
|
tests/test_live_runtime_helpers.py
CHANGED
|
@@ -323,3 +323,22 @@ def test_feature_cue_context_scan_runs_on_toy_runtime() -> None:
|
|
| 323 |
assert 0 <= result.dominant_cue_context_count <= 2
|
| 324 |
assert 0 <= result.off_dominant_active_count <= result.active_condition_count
|
| 325 |
assert len(result.chart_rows) == 6
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 323 |
assert 0 <= result.dominant_cue_context_count <= 2
|
| 324 |
assert 0 <= result.off_dominant_active_count <= result.active_condition_count
|
| 325 |
assert len(result.chart_rows) == 6
|
| 326 |
+
|
| 327 |
+
|
| 328 |
+
def test_candidate_causal_screen_batches_multiple_ablation_candidates() -> None:
|
| 329 |
+
runtime = make_runtime()
|
| 330 |
+
result = runtime.candidate_causal_screen(
|
| 331 |
+
text='abc',
|
| 332 |
+
layer=0,
|
| 333 |
+
token_index=-1,
|
| 334 |
+
feature_ids=[0, 1, 2],
|
| 335 |
+
target_text='d',
|
| 336 |
+
)
|
| 337 |
+
assert result.candidate_count == 3
|
| 338 |
+
assert len(result.rows) == 3
|
| 339 |
+
assert len(result.chart_rows) == 3
|
| 340 |
+
assert all(len(row) == 8 for row in result.rows)
|
| 341 |
+
assert all(row[0] == rank for rank, row in enumerate(result.rows, start=1))
|
| 342 |
+
assert 0 <= result.active_feature_count <= result.candidate_count
|
| 343 |
+
assert all(math.isfinite(float(row[5])) for row in result.rows)
|
| 344 |
+
assert all(float(row[7]) >= 0 for row in result.rows)
|
tests/test_ui_helpers.py
CHANGED
|
@@ -64,14 +64,15 @@ def test_select_candidate_row_uses_feature_id_column() -> None:
|
|
| 64 |
assert update['value'] == '445'
|
| 65 |
|
| 66 |
|
| 67 |
-
def
|
| 68 |
app = _import_app()
|
| 69 |
assert 'featurelens_${stem' in app.INSTALL_REFLOW_JS
|
| 70 |
assert 'chart.png' in app.INSTALL_REFLOW_JS
|
| 71 |
-
assert 'featurelens-
|
| 72 |
-
assert '
|
| 73 |
assert 'stopImmediatePropagation' in app.INSTALL_REFLOW_JS
|
| 74 |
-
assert '
|
|
|
|
| 75 |
|
| 76 |
|
| 77 |
def test_dose_response_has_independent_target_control() -> None:
|
|
@@ -105,3 +106,27 @@ def test_result_tables_hide_native_labels_in_favor_of_explicit_headings() -> Non
|
|
| 105 |
assert app.discovery_table.show_label is False
|
| 106 |
assert app.dose_table.show_label is False
|
| 107 |
assert '.table-heading' in app.CSS
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
assert update['value'] == '445'
|
| 65 |
|
| 66 |
|
| 67 |
+
def test_frontend_helpers_name_exports_and_use_in_place_focus() -> None:
|
| 68 |
app = _import_app()
|
| 69 |
assert 'featurelens_${stem' in app.INSTALL_REFLOW_JS
|
| 70 |
assert 'chart.png' in app.INSTALL_REFLOW_JS
|
| 71 |
+
assert 'featurelens-inline-focus' in app.INSTALL_REFLOW_JS
|
| 72 |
+
assert 'toggleInlineFocus' in app.INSTALL_REFLOW_JS
|
| 73 |
assert 'stopImmediatePropagation' in app.INSTALL_REFLOW_JS
|
| 74 |
+
assert 'featurelens-plot-focus-overlay' not in app.INSTALL_REFLOW_JS
|
| 75 |
+
assert 'window.scrollTo' not in app.INSTALL_REFLOW_JS
|
| 76 |
|
| 77 |
|
| 78 |
def test_dose_response_has_independent_target_control() -> None:
|
|
|
|
| 106 |
assert app.discovery_table.show_label is False
|
| 107 |
assert app.dose_table.show_label is False
|
| 108 |
assert '.table-heading' in app.CSS
|
| 109 |
+
assert 'margin: 0 0 -34px' in app.CSS
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
def test_candidate_screen_markdown_is_explicitly_triage_only() -> None:
|
| 113 |
+
app = _import_app()
|
| 114 |
+
result = SimpleNamespace(
|
| 115 |
+
candidate_count=3,
|
| 116 |
+
active_feature_count=3,
|
| 117 |
+
target_tokens=['2', 'x'],
|
| 118 |
+
execution_drift_mean_logprob=1e-4,
|
| 119 |
+
execution_drift_js=2e-6,
|
| 120 |
+
rows=[[1, 16369, 29.25, True, 29.0, -0.12, -0.24, 0.004]],
|
| 121 |
+
)
|
| 122 |
+
text = app._candidate_screen_metrics_markdown(result)
|
| 123 |
+
assert 'triage screen' in text
|
| 124 |
+
assert 'no random-control ensemble is spent here' in text
|
| 125 |
+
assert 'Single-feature causal test' in text
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def test_candidate_screen_has_independent_target_and_multiselect() -> None:
|
| 129 |
+
app = _import_app()
|
| 130 |
+
assert app.candidate_screen_target.value == '2x'
|
| 131 |
+
assert app.candidate_screen_ids.multiselect is True
|
| 132 |
+
assert app.candidate_screen_ids.max_choices == 8
|