ArchitSharma commited on
Commit
393bb89
·
1 Parent(s): 1bb3265

Release FeatureLens v0.7.0

Browse files
CHANGELOG.md CHANGED
@@ -1,5 +1,30 @@
1
  # Changelog
2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ## v0.6.0
4
 
5
  ### UX / navigation
 
1
  # Changelog
2
 
3
+ ## v0.7.0
4
+
5
+ ### Research-instrument UI cleanup
6
+ - Removed collapsible wrappers from the core **scale dose-response** and **contrastive preference** experiments so section headings are no longer duplicated by accordion titles.
7
+ - Simplified the page header and removed visible footer/redundancy that did not help a reviewer use the tool.
8
+ - Strengthened result-table title and column-header typography.
9
+ - Replaced full-viewport plot stretching with a bounded top-centered **focus view**; exiting focus restores the prior page position.
10
+ - Kept native plot export but rename downloads to descriptive `featurelens_<plot-name>.png` filenames instead of a generic chart name.
11
+
12
+ ### Candidate discovery and causal readiness
13
+ - Reworked live concept-guided discovery around **Balanced selectivity**: `selectivity × target activation rate × log1p(target mean)`. This prevents very large but non-selective SAE coefficients from dominating the exploratory shortlist.
14
+ - Retained **Raw mean difference** as an explicit comparison mode rather than silently changing the old ranking.
15
+ - Candidate discovery now evaluates the current Workbench prompt in the same GPU batch and reports current-prompt maximum activation plus selected-token activation.
16
+ - The candidate selector defaults to the highest-ranked displayed candidate active at the current Workbench token when one exists.
17
+ - Candidate-table row selection is wired directly to the candidate selector, and the reuse action now confirms exactly which downstream feature selectors were updated.
18
+
19
+ ### Lexical / structural specificity
20
+ - Added a **cue × context specificity matrix**: cross several prompt stems with the same completion cues in one batched forward and measure the selected feature at every resulting final token.
21
+ - This extends the single-stem completion-cue test so a response to `is` can be separated from a broader completion-boundary or context-dependent response.
22
+
23
+ ### Controlled data and validation
24
+ - Replaced the French-language control concept one-for-one with a **German-language** concept while preserving 224 balanced discovery prompts and 28 causal tasks.
25
+ - Added regression coverage for German data, balanced/raw candidate ranking, current-Workbench candidate compatibility, cue × context scanning, candidate row selection, descriptive export naming, and focus-position preservation.
26
+ - Kept the actual Gradio `launch()` smoke test, compile gate, release checker, and deferred comprehensive adversarial suite.
27
+
28
  ## v0.6.0
29
 
30
  ### UX / navigation
README.md CHANGED
@@ -13,7 +13,7 @@ license: mit
13
 
14
  # FeatureLens — Causal Interpretability Workbench
15
 
16
- > **v0.6:** a plain-language **Start here** workflow, persistent experiment context, explicit per-experiment feature selectors, plot fullscreen/export controls, concept-guided candidate-feature discovery, and completion-cue sensitivity tests—while retaining the v0.5 causal, geometry, robustness, and prompt-wide evidence tools.
17
 
18
  **Research question:**
19
 
@@ -21,7 +21,6 @@ license: mit
21
 
22
  FeatureLens uses **Qwen3-1.7B-Base** with **Qwen-Scope residual-stream sparse autoencoders (SAEs)**. It is not an SAE viewer clone: the project separates representation quality, held-out predictiveness, robustness, and causal intervention evidence.
23
 
24
- FeatureLens is independent of thesis code and thesis datasets.
25
 
26
  ## Evidence ladder
27
 
@@ -129,7 +128,7 @@ The difference is reported as an **interaction excess**. A non-zero excess means
129
 
130
  The **Feature evidence** tab now supports four complementary live questions:
131
 
132
- 1. **Concept-guided candidate discovery** — choose one of the seven controlled concept groups and rank SAE features by the difference between that concept's prompt-wide mean maximum activation and the mean across the other six groups. The result is a candidate list, **not a semantic label**.
133
  2. **Feature-token activation trace** — show exactly where a selected feature enters the SAE TopK support across every token of the current Workbench prompt.
134
  3. **Completion-cue sensitivity** — append controlled suffixes/cues to one prompt stem and measure the selected feature at the final token. This is useful for separating concept evidence from lexical/structural completion cues.
135
  4. **Prompt-wide controlled concept contrast** — evaluate one selected feature on a small balanced batch from:
@@ -138,17 +137,17 @@ The **Feature evidence** tab now supports four complementary live questions:
138
  - mathematics;
139
  - positive sentiment;
140
  - negative sentiment;
141
- - French language;
142
  - factual entities;
143
  - uncertainty.
144
 
145
- Each controlled prompt is summarized by the feature's **maximum activation across non-padding tokens**. Candidate discovery ranks by a transparent target-minus-other mean difference and reports activation rates/selectivity alongside it. The controlled contrast reports mean/median prompt-wide maxima, prompt activation rate, mean activation when active, and maximum activation by concept.
146
 
147
  Both tools are exploratory live evidence. They never overwrite `Offline concept hint`; the held-out offline AUROC/F1 benchmark remains the stronger feature/concept claim.
148
 
149
  ### Contrastive continuation preference
150
 
151
- A probability increase for one target can reflect a broad distribution shift rather than selective behavioral control. v0.5 adds a stricter live test with two exact continuations, A and B. FeatureLens scores both teacher-forced under the same SAE intervention and random-control ensemble, then reports the causal change in:
152
 
153
  ```text
154
  log P(A) - log P(B)
@@ -173,9 +172,9 @@ The prompt-wide view reduces sensitivity to accidentally comparing semantically
173
 
174
  The trajectory view compares SAE reconstruction and sparsity/concentration diagnostics at the same prompt token across layers 4, 14 and 26. It does **not** equate feature IDs across layer-specific SAE dictionaries.
175
 
176
- ## v0.6 interface and navigation
177
 
178
- v0.6 is designed so a reviewer does not need prior mechanistic-interpretability vocabulary to find the controls:
179
 
180
  - a **Start here** tab explains the evidence workflow and glossary in plain language;
181
  - a persistent **Current Workbench context** banner states the active prompt, layer and token after inspection;
@@ -183,7 +182,7 @@ v0.6 is designed so a reviewer does not need prior mechanistic-interpretability
183
  - single-feature causal, dose-response, contrastive-preference, feature-set and evidence panels expose their own editable feature selectors instead of silently borrowing a hidden value;
184
  - concept-guided discovery gives users a route to candidate features without knowing feature IDs in advance;
185
  - headings now follow a conventional descending hierarchy and table headers use the same readable base scale as the surrounding interface;
186
- - every native plot exposes Gradio's **fullscreen** control for zoomed reading and **export** control for PNG output;
187
  - all action/copy buttons retain one muted-teal visual language and copy actions provide visible confirmation.
188
 
189
  The centered 1600 px research canvas, bounded result-table heights, dynamic resize observer and explicit bottom padding from v0.5 remain in place for Hugging Face's embedded Space layout.
@@ -192,7 +191,7 @@ The centered 1600 px research canvas, bounded result-table heights, dynamic resi
192
 
193
  Every result table that is intended for analysis has an explicit **Copy table with headers** button. The app serializes the displayed data as TSV before copying, so column names are preserved.
194
 
195
- The native Dataframe fullscreen control is retained, while the custom copy action avoids relying on browser text selection and gives immediate visual confirmation when clipboard writing succeeds.
196
 
197
  ## Reconstruction-preserving edits
198
 
@@ -304,7 +303,7 @@ FEATURELENS_CONTRAST_PROMPTS_PER_CONCEPT=4
304
 
305
  ## UI design
306
 
307
- v0.6 keeps the interface deliberately closer to a conventional research instrument than a rounded dashboard:
308
 
309
  - Times / Liberation Serif-style formal typography;
310
  - restrained teal, umber, stone, plum, and muted red chart accents;
@@ -356,7 +355,7 @@ python scripts/ui_smoke.py
356
  python scripts/release_check.py
357
  ```
358
 
359
- See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.6 validation guide uses the **exact labels shown in the UI**, focuses on new-version acceptance/regression tests, and intentionally defers the comprehensive adversarial suite to the final release.
360
 
361
  ## Limitations
362
 
@@ -372,7 +371,7 @@ See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.6 validation guide uses t
372
  ## Resume-ready description
373
 
374
  > **FeatureLens — Causal Interpretability Workbench** | PyTorch, Qwen3, Sparse Autoencoders, Mechanistic Interpretability, Gradio
375
- > Built an SAE-based interpretability system for Qwen3-1.7B with held-out concept discovery, concept-guided candidate discovery, token/prompt-wide and completion-cue feature evidence, reconstruction-preserving single and multi-feature interventions, full-continuation and contrastive preference scoring, dose-response analysis, decoder-geometry/non-additivity diagnostics, and norm-matched random-control ensembles.
376
 
377
  ## Acknowledgements
378
 
 
13
 
14
  # FeatureLens — Causal Interpretability Workbench
15
 
16
+ > **v0.7:** a cleaner research-instrument UI, bounded focus mode for tables/plots, descriptive chart exports, German-language controls, stronger concept-candidate ranking with current-prompt compatibility, click-to-select candidate handoff, and a cue × context specificity diagnostic—while retaining the causal, robustness, geometry, and prompt-wide evidence tools.
17
 
18
  **Research question:**
19
 
 
21
 
22
  FeatureLens uses **Qwen3-1.7B-Base** with **Qwen-Scope residual-stream sparse autoencoders (SAEs)**. It is not an SAE viewer clone: the project separates representation quality, held-out predictiveness, robustness, and causal intervention evidence.
23
 
 
24
 
25
  ## Evidence ladder
26
 
 
128
 
129
  The **Feature evidence** tab now supports four complementary live questions:
130
 
131
+ 1. **Concept-guided candidate discovery** — choose one of the seven controlled concept groups and rank SAE features with a selectivity-first score that combines target-vs-other contrast, target activation rate, and log activation magnitude; raw mean-difference ranking remains available for comparison. The result is a candidate list, **not a semantic label**.
132
  2. **Feature-token activation trace** — show exactly where a selected feature enters the SAE TopK support across every token of the current Workbench prompt.
133
  3. **Completion-cue sensitivity** — append controlled suffixes/cues to one prompt stem and measure the selected feature at the final token. This is useful for separating concept evidence from lexical/structural completion cues.
134
  4. **Prompt-wide controlled concept contrast** — evaluate one selected feature on a small balanced batch from:
 
137
  - mathematics;
138
  - positive sentiment;
139
  - negative sentiment;
140
+ - German language;
141
  - factual entities;
142
  - uncertainty.
143
 
144
+ Each controlled prompt is summarized by the feature's **maximum activation across non-padding tokens**. Candidate discovery defaults to **Balanced selectivity** — `positive selectivity × target activation rate × log1p(target mean activation)` — so globally large but weakly selective coefficients do not dominate purely by scale. A **Raw mean difference** mode remains available for comparison. The discovery table also reports current-Workbench prompt/token activation so candidates can be separated from candidates that are immediately causal-testable at the selected location. The controlled contrast reports mean/median prompt-wide maxima, prompt activation rate, mean activation when active, and maximum activation by concept.
145
 
146
  Both tools are exploratory live evidence. They never overwrite `Offline concept hint`; the held-out offline AUROC/F1 benchmark remains the stronger feature/concept claim.
147
 
148
  ### Contrastive continuation preference
149
 
150
+ A probability increase for one target can reflect a broad distribution shift rather than selective behavioral control. FeatureLens includes a stricter live test with two exact continuations, A and B. FeatureLens scores both teacher-forced under the same SAE intervention and random-control ensemble, then reports the causal change in:
151
 
152
  ```text
153
  log P(A) - log P(B)
 
172
 
173
  The trajectory view compares SAE reconstruction and sparsity/concentration diagnostics at the same prompt token across layers 4, 14 and 26. It does **not** equate feature IDs across layer-specific SAE dictionaries.
174
 
175
+ ## Interface and navigation
176
 
177
+ The interface is designed so a reviewer does not need prior mechanistic-interpretability vocabulary to find the controls:
178
 
179
  - a **Start here** tab explains the evidence workflow and glossary in plain language;
180
  - a persistent **Current Workbench context** banner states the active prompt, layer and token after inspection;
 
182
  - single-feature causal, dose-response, contrastive-preference, feature-set and evidence panels expose their own editable feature selectors instead of silently borrowing a hidden value;
183
  - concept-guided discovery gives users a route to candidate features without knowing feature IDs in advance;
184
  - headings now follow a conventional descending hierarchy and table headers use the same readable base scale as the surrounding interface;
185
+ - tables and plots use a bounded **focus mode** instead of stretching to the entire viewport; plot exports are renamed from the generic browser filename to descriptive `featurelens_*.png` names;
186
  - all action/copy buttons retain one muted-teal visual language and copy actions provide visible confirmation.
187
 
188
  The centered 1600 px research canvas, bounded result-table heights, dynamic resize observer and explicit bottom padding from v0.5 remain in place for Hugging Face's embedded Space layout.
 
191
 
192
  Every result table that is intended for analysis has an explicit **Copy table with headers** button. The app serializes the displayed data as TSV before copying, so column names are preserved.
193
 
194
+ The Dataframe focus control is retained, while the custom copy action avoids relying on browser text selection and gives immediate visual confirmation when clipboard writing succeeds.
195
 
196
  ## Reconstruction-preserving edits
197
 
 
303
 
304
  ## UI design
305
 
306
+ The interface deliberately stays closer to a conventional research instrument than a rounded dashboard:
307
 
308
  - Times / Liberation Serif-style formal typography;
309
  - restrained teal, umber, stone, plum, and muted red chart accents;
 
355
  python scripts/release_check.py
356
  ```
357
 
358
+ See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.7 validation guide uses the **exact labels shown in the UI**, focuses on new-version acceptance/regression tests, and intentionally defers the comprehensive adversarial suite to the final release.
359
 
360
  ## Limitations
361
 
 
371
  ## Resume-ready description
372
 
373
  > **FeatureLens — Causal Interpretability Workbench** | PyTorch, Qwen3, Sparse Autoencoders, Mechanistic Interpretability, Gradio
374
+ > Built an SAE-based interpretability system for Qwen3-1.7B with held-out concept discovery, concept-guided candidate discovery, token/prompt-wide, completion-cue, and cue × context feature evidence, reconstruction-preserving single and multi-feature interventions, full-continuation and contrastive preference scoring, dose-response analysis, decoder-geometry/non-additivity diagnostics, and norm-matched random-control ensembles.
375
 
376
  ## Acknowledgements
377
 
app.py CHANGED
@@ -1,7 +1,7 @@
1
  from __future__ import annotations
2
 
3
- import pandas as pd
4
  import gradio as gr
 
5
 
6
  from featurelens.config import SETTINGS
7
  from featurelens.hf_runtime import gpu
@@ -47,26 +47,18 @@ CSS = r"""
47
  .gradio-container h3 { font-size: 1.46rem; line-height: 1.28; }
48
  .gradio-container h4 { font-size: 1.24rem; line-height: 1.30; }
49
  .hero {
50
- padding: 15px 2px 7px;
51
  border-bottom: 1px solid var(--border-color-primary);
52
- margin-bottom: 16px;
53
  }
54
  .hero h1 {
55
  margin: 0;
56
- font-size: 2.25rem;
57
  font-weight: 600;
58
  letter-spacing: 0;
59
  }
60
- .hero .subtitle { margin-top: 4px; font-size: 1.08rem; opacity: .78; }
61
- .hero .metadata { margin-top: 8px; font-size: .92rem; opacity: .68; letter-spacing: .01em; }
62
- .research-q {
63
- margin: 14px 0 18px;
64
- padding: 11px 15px;
65
- border-left: 3px solid #708B86;
66
- border-top: 1px solid var(--border-color-primary);
67
- border-bottom: 1px solid var(--border-color-primary);
68
- background: var(--background-fill-secondary);
69
- }
70
  .section-rule {
71
  margin: 20px 0 11px;
72
  padding-top: 9px;
@@ -109,14 +101,7 @@ CSS = r"""
109
  .gradio-container textarea, .gradio-container input { border-radius: 2px !important; }
110
  .form-note { margin-top: -3px; margin-bottom: 9px; font-size: .92rem; opacity: .72; }
111
  .wide-table { width: 100% !important; }
112
- .classic-footer {
113
- margin-top: 44px;
114
- padding: 20px 2px 58px;
115
- border-top: 1px solid var(--border-color-primary);
116
- opacity: .70;
117
- font-size: .92rem;
118
- min-height: 110px;
119
- }
120
  .tabs, .tabitem { padding-bottom: 26px !important; }
121
  .context-card {
122
  border: 1px solid var(--border-color-primary);
@@ -135,8 +120,53 @@ CSS = r"""
135
  border-radius: 2px;
136
  }
137
  .start-card h3 { margin-top: 0; }
138
- .experiment-accordion > button { font-size: 1.18rem !important; font-weight: 700 !important; }
139
  .graph-note { font-size: .95rem; opacity: .72; margin-top: 2px; }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
140
  @media (max-width: 900px) {
141
  .gradio-container { width: 100% !important; padding-left: 12px !important; padding-right: 12px !important; }
142
  }
@@ -205,6 +235,56 @@ INSTALL_REFLOW_JS = r"""
205
  const mutation = new MutationObserver(kick);
206
  mutation.observe(root, {subtree: true, childList: true});
207
  window.__featurelens_mutation_observer = mutation;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
208
  kick();
209
  return [];
210
  }
@@ -439,12 +519,39 @@ def _discovery_metrics_markdown(result) -> str:
439
  f"No positively selective candidate features were found for **{result.concept}** at layer "
440
  f"**{result.layer}** in this small live batch. Increase the sample or try another layer."
441
  )
 
 
 
 
 
 
 
 
 
 
 
442
  return (
443
  f"Concept **{result.concept}** · layer **{result.layer}** · "
444
  f"{result.prompts_per_concept} prompts/concept. \n"
445
- f"Showing **{len(result.candidate_ids)}** features ranked by target-minus-other prompt-wide mean activation. \n\n"
446
- "This is **candidate discovery**, not a semantic label. Use the offline held-out AUROC/F1 pipeline "
447
- "before describing a feature as concept-selective."
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
448
  )
449
 
450
 
@@ -916,35 +1023,56 @@ def run_contrastive_causal(
916
 
917
 
918
  @gpu(duration=35)
919
- def run_concept_feature_discovery(concept: str, layer: int, prompts_per_concept: int, top_n: int):
 
 
 
 
 
 
 
 
920
  try:
 
 
 
 
 
921
  result = RUNTIME.concept_feature_discovery(
922
  concept=concept,
923
  layer=int(layer),
924
  prompts_per_concept=int(prompts_per_concept),
925
  top_n=int(top_n),
 
 
 
926
  )
927
  columns = [
928
  "Rank",
929
  "Feature id",
 
930
  "Target mean max",
931
  "Other mean max",
932
  "Mean difference",
933
- "Selectivity score",
934
  "Target activation rate",
935
  "Other activation rate",
936
- "Target max activation",
 
 
937
  ]
938
  table = pd.DataFrame(result.rows, columns=columns)
939
- chart = pd.DataFrame(result.chart_rows, columns=["Feature", "Target − other mean"])
940
- chart["Series"] = "Concept contrast"
941
  choices = [str(fid) for fid in result.candidate_ids]
942
- candidate_update = gr.update(choices=choices, value=choices[0] if choices else None)
 
943
  return _discovery_metrics_markdown(result), table, chart, candidate_update, _tsv(table)
944
  except Exception as exc:
945
  _raise_ui_error(exc)
946
 
947
 
 
948
  @gpu(duration=25)
949
  def run_feature_cue_scan(feature_id: str, layer: int, prompt_stem: str, cue_text: str):
950
  try:
@@ -966,11 +1094,47 @@ def run_feature_cue_scan(feature_id: str, layer: int, prompt_stem: str, cue_text
966
  _raise_ui_error(exc)
967
 
968
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
969
  def use_candidate_feature(candidate_id: str):
970
  if candidate_id is None or str(candidate_id).strip() == "":
971
  raise gr.Error("Run concept-guided discovery and choose a candidate first.")
972
  value = str(int(float(candidate_id)))
973
- return value, value, value, value
 
 
 
 
974
 
975
 
976
  def mode_help(mode: str):
@@ -990,13 +1154,10 @@ def set_mode_help(mode: str):
990
  with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_width=True) as demo:
991
  gr.HTML(
992
  '<header class="hero">'
993
- '<h1>FeatureLens <span style="font-size:.48em;opacity:.58;font-weight:400">v0.6</span></h1>'
994
  '<div class="subtitle">Causal Interpretability Workbench</div>'
995
- '<div class="metadata">Qwen3-1.7B-Base · Qwen-Scope residual SAEs · layers 4 / 14 / 26 · '
996
- '32,768 features · TopK=50 · ZeroGPU</div>'
997
  '</header>'
998
- '<div class="research-q"><strong>Research question.</strong> Do sparse features that predict a concept '
999
- 'also causally influence the model’s behaviour?</div>'
1000
  )
1001
 
1002
  global_context = gr.Markdown(
@@ -1035,7 +1196,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1035
  "- **Ablate:** remove that feature's current contribution. **Scale:** multiply it.\n"
1036
  "- **Random control:** an equally large residual edit in an unrelated direction.\n"
1037
  "- **Teacher-forced score:** probability assigned to exact continuation text, even if greedy generation does not change.\n\n"
1038
- "**Tables:** use fullscreen/copy controls. **Plots:** use the top-right fullscreen button to zoom and the export button for PNG."
1039
  )
1040
 
1041
  with gr.Tab("Workbench"):
@@ -1052,7 +1213,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1052
  examples=[
1053
  ["The derivative of x squared is"],
1054
  ["In Python, reverse a list using"],
1055
- ["Je voudrais réserver une table pour"],
1056
  ["I am not fully certain, but the answer may be"],
1057
  ],
1058
  inputs=[prompt],
@@ -1087,7 +1248,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1087
  label="Strongest active SAE features",
1088
  wrap=False,
1089
  max_height=380,
1090
- buttons=["fullscreen"],
1091
  )
1092
  feature_tsv = gr.Textbox(visible="hidden")
1093
  feature_copy = _copy_button()
@@ -1097,11 +1258,11 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1097
  y="Activation",
1098
  color="Series",
1099
  color_map={"Activation": INK_TEAL},
1100
- title="Activation profile",
1101
  x_title="Feature id",
1102
  y_title="Activation",
1103
  x_label_angle=-35,
1104
- buttons=["fullscreen", "export"],
1105
  height=330,
1106
  )
1107
 
@@ -1149,7 +1310,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1149
  token_prob_table = gr.Dataframe(
1150
  interactive=False,
1151
  label="Next-token distribution shift",
1152
- buttons=["fullscreen"],
1153
  wrap=False,
1154
  max_height=380,
1155
  )
@@ -1158,7 +1319,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1158
  target_token_table = gr.Dataframe(
1159
  interactive=False,
1160
  label="Target continuation token-by-token score",
1161
- buttons=["fullscreen"],
1162
  wrap=False,
1163
  max_height=380,
1164
  )
@@ -1166,8 +1327,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1166
  target_token_copy = _copy_button()
1167
 
1168
  gr.HTML('<div class="section-rule">III. Single-feature scale dose-response</div>')
1169
- with gr.Accordion("Scale dose-response", open=False, elem_classes=["experiment-accordion"]):
1170
- gr.Markdown("### Scale dose-response")
1171
  gr.Markdown("**Uses current Workbench prompt / layer / token.** Choose the feature for this experiment below.")
1172
  dose_feature_id = gr.Dropdown(
1173
  choices=[],
@@ -1187,7 +1347,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1187
  dose_table = gr.Dataframe(
1188
  interactive=False,
1189
  label="Scale dose-response measurements",
1190
- buttons=["fullscreen"],
1191
  wrap=False,
1192
  max_height=380,
1193
  )
@@ -1199,16 +1359,15 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1199
  y="Δ mean log p/token",
1200
  color="Series",
1201
  color_map={"SAE feature": INK_TEAL},
1202
- title="Scale dose-response",
1203
  x_title="Feature multiplier",
1204
  y_title="Δ mean log p/token",
1205
- buttons=["fullscreen", "export"],
1206
  height=330,
1207
  )
1208
 
1209
  gr.HTML('<div class="section-rule">IV. Contrastive continuation preference</div>')
1210
- with gr.Accordion("Contrastive causal preference test", open=False, elem_classes=["experiment-accordion"]):
1211
- gr.Markdown("### Contrastive causal preference test")
1212
  gr.Markdown("**Uses current Workbench prompt / layer / token.** Select the feature explicitly below.")
1213
  contrastive_feature_id = gr.Dropdown(
1214
  choices=[],
@@ -1246,7 +1405,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1246
  contrastive_table = gr.Dataframe(
1247
  interactive=False,
1248
  label="Contrastive continuation scores",
1249
- buttons=["fullscreen"],
1250
  wrap=False,
1251
  max_height=320,
1252
  )
@@ -1258,10 +1417,10 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1258
  y="A−B sequence log-odds",
1259
  color="Series",
1260
  color_map={"Contrastive preference": INK_TEAL},
1261
- title="Preference between exact continuations",
1262
  x_title="Execution condition",
1263
  y_title="Sequence log-odds A−B",
1264
- buttons=["fullscreen", "export"],
1265
  height=320,
1266
  )
1267
 
@@ -1313,7 +1472,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1313
  set_feature_table = gr.Dataframe(
1314
  interactive=False,
1315
  label="Joint intervention features",
1316
- buttons=["fullscreen"],
1317
  wrap=False,
1318
  max_height=380,
1319
  )
@@ -1323,7 +1482,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1323
  set_target_table = gr.Dataframe(
1324
  interactive=False,
1325
  label="Target continuation token-by-token score",
1326
- buttons=["fullscreen"],
1327
  wrap=False,
1328
  max_height=380,
1329
  )
@@ -1343,7 +1502,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1343
  set_sweep_table = gr.Dataframe(
1344
  interactive=False,
1345
  label="Feature-set size measurements",
1346
- buttons=["fullscreen"],
1347
  wrap=False,
1348
  max_height=380,
1349
  )
@@ -1358,10 +1517,10 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1358
  "Top-k SAE ablation": INK_TEAL,
1359
  "Random signed mean": INK_STONE,
1360
  },
1361
- title="Effect vs feature-set size",
1362
  x_title="Number of jointly ablated features",
1363
  y_title="Δ mean log p/token",
1364
- buttons=["fullscreen", "export"],
1365
  height=330,
1366
  )
1367
 
@@ -1379,7 +1538,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1379
  interaction_table = gr.Dataframe(
1380
  interactive=False,
1381
  label="Individual and joint ablation measurements",
1382
- buttons=["fullscreen"],
1383
  wrap=False,
1384
  max_height=380,
1385
  )
@@ -1391,18 +1550,17 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1391
  y="Δ mean log p/token",
1392
  color="Series",
1393
  color_map={"Ablation effect": INK_UMBER},
1394
- title="Individual vs joint effect",
1395
  x_title="Intervention condition",
1396
  y_title="Δ mean log p/token",
1397
  x_label_angle=-25,
1398
- buttons=["fullscreen", "export"],
1399
  height=330,
1400
  )
1401
 
1402
 
1403
  gr.HTML('<div class="section-rule">Decoder geometry</div>')
1404
- with gr.Accordion("Feature-set decoder geometry", open=False, elem_classes=["experiment-accordion"]):
1405
- gr.Markdown("### Feature-set decoder geometry")
1406
  gr.Markdown(
1407
  "Joint causal effects can reflect both **decoder-direction geometry** and downstream non-linearity. "
1408
  "This zero-extra-generation diagnostic measures pairwise decoder cosines and compares the actual "
@@ -1417,7 +1575,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1417
  geometry_table = gr.Dataframe(
1418
  interactive=False,
1419
  label="Pairwise decoder geometry",
1420
- buttons=["fullscreen"],
1421
  wrap=False,
1422
  max_height=340,
1423
  )
@@ -1429,11 +1587,11 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1429
  y="Decoder cosine",
1430
  color="Series",
1431
  color_map={"Decoder cosine": INK_PLUM},
1432
- title="Pairwise SAE decoder cosine",
1433
  x_title="Feature pair",
1434
  y_title="Cosine similarity",
1435
  x_label_angle=-30,
1436
- buttons=["fullscreen", "export"],
1437
  height=320,
1438
  )
1439
 
@@ -1446,15 +1604,16 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1446
  )
1447
  gr.HTML('<div class="section-rule">A. Concept-guided candidate discovery</div>')
1448
  gr.Markdown(
1449
- "Don't know which feature id to inspect? Choose a controlled concept and let FeatureLens rank candidate "
1450
- "features whose **prompt-wide maximum activation** is higher for that concept than for the other six groups. "
1451
- "This is a live screening step, not a semantic label."
 
1452
  )
1453
  with gr.Row(equal_height=True):
1454
  discovery_concept = gr.Dropdown(
1455
  choices=[
1456
  "code", "mathematics", "positive_sentiment", "negative_sentiment",
1457
- "french_language", "factual_entities", "uncertainty"
1458
  ],
1459
  value="mathematics",
1460
  label="Target concept",
@@ -1462,25 +1621,46 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1462
  discovery_layer = gr.Dropdown(choices=list(SETTINGS.layers), value=SETTINGS.layers[1], label="Residual layer")
1463
  discovery_n = gr.Slider(2, 6, value=SETTINGS.contrast_prompts_per_concept, step=1, label="Prompts per concept")
1464
  discovery_top_n = gr.Slider(5, 20, value=12, step=1, label="Candidate features")
 
 
 
 
 
 
 
 
 
 
 
 
1465
  discovery_btn = gr.Button("Discover concept-associated candidates", variant="primary", elem_classes=["action-btn"])
1466
  discovery_metrics = gr.Markdown()
1467
  with gr.Row(equal_height=False):
1468
  with gr.Column(scale=3):
1469
  discovery_table = gr.Dataframe(
1470
- interactive=False, label="Candidate feature contrast", buttons=["fullscreen"], wrap=False, max_height=380
 
1471
  )
1472
  discovery_tsv = gr.Textbox(visible="hidden")
1473
  discovery_copy = _copy_button()
1474
  with gr.Column(scale=2):
1475
  discovery_plot = gr.BarPlot(
1476
- x="Feature", y="Target − other mean", color="Series",
1477
- color_map={"Concept contrast": INK_TEAL}, title="Candidate concept contrast",
1478
- x_title="Feature id", y_title="Target other mean max activation", x_label_angle=-35,
1479
- buttons=["fullscreen", "export"], height=330
1480
  )
 
 
 
 
 
1481
  with gr.Row(equal_height=True):
1482
- discovery_candidate = gr.Dropdown(choices=[], label="Discovered candidate feature id", allow_custom_value=True, scale=3)
1483
- use_candidate_btn = gr.Button("Use candidate in feature experiments", variant="primary", elem_classes=["action-btn"], scale=2)
 
 
 
1484
 
1485
  gr.HTML('<div class="section-rule">B. Inspect one feature</div>')
1486
  contrast_location = gr.Markdown(
@@ -1520,7 +1700,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1520
  trace_table = gr.Dataframe(
1521
  interactive=False,
1522
  label="Feature activation by prompt token",
1523
- buttons=["fullscreen"],
1524
  wrap=False,
1525
  max_height=340,
1526
  )
@@ -1532,11 +1712,11 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1532
  y="Activation",
1533
  color="Series",
1534
  color_map={"Feature activation": INK_TEAL},
1535
- title="Feature activation across prompt tokens",
1536
  x_title="Prompt token",
1537
  y_title="Activation",
1538
  x_label_angle=-35,
1539
- buttons=["fullscreen", "export"],
1540
  height=320,
1541
  )
1542
 
@@ -1553,17 +1733,64 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1553
  cue_metrics = gr.Markdown()
1554
  with gr.Row(equal_height=False):
1555
  with gr.Column(scale=3):
1556
- cue_table = gr.Dataframe(interactive=False, label="Feature response by completion cue", buttons=["fullscreen"], wrap=False, max_height=340)
1557
  cue_tsv = gr.Textbox(visible="hidden")
1558
  cue_copy = _copy_button()
1559
  with gr.Column(scale=2):
1560
  cue_plot = gr.BarPlot(
1561
  x="Cue", y="Activation", color="Series", color_map={"Cue response": INK_UMBER},
1562
- title="Completion-cue feature response", x_title="Cue", y_title="Final-token activation",
1563
- buttons=["fullscreen", "export"], height=320
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1564
  )
1565
 
1566
- gr.HTML('<div class="section-rule">D. Controlled concept contrast for selected feature</div>')
1567
  gr.Markdown("### Prompt-wide controlled concept contrast")
1568
  contrast_btn = gr.Button("Run controlled concept contrast", variant="primary", elem_classes=["action-btn"])
1569
  contrast_metrics = gr.Markdown()
@@ -1572,7 +1799,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1572
  contrast_table = gr.Dataframe(
1573
  interactive=False,
1574
  label="Feature activation by controlled concept",
1575
- buttons=["fullscreen"],
1576
  wrap=False,
1577
  max_height=380,
1578
  )
@@ -1584,11 +1811,11 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1584
  y="Mean prompt-wide max",
1585
  color="Series",
1586
  color_map={"Prompt-wide max": INK_BLUEGREY},
1587
- title="Prompt-wide controlled concept contrast",
1588
  x_title="Concept",
1589
  y_title="Mean max activation",
1590
  x_label_angle=-25,
1591
- buttons=["fullscreen", "export"],
1592
  height=330,
1593
  )
1594
 
@@ -1633,7 +1860,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1633
  para_table = gr.Dataframe(
1634
  interactive=False,
1635
  label="Top-feature overlap at selected tokens",
1636
- buttons=["fullscreen"],
1637
  wrap=False,
1638
  max_height=380,
1639
  )
@@ -1645,11 +1872,11 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1645
  y="Activation",
1646
  color="Prompt",
1647
  color_map={"Original": INK_TEAL, "Paraphrase": INK_PLUM},
1648
- title="Selected-token feature activations",
1649
  x_title="Feature id",
1650
  y_title="Activation",
1651
  x_label_angle=-35,
1652
- buttons=["fullscreen", "export"],
1653
  height=330,
1654
  )
1655
 
@@ -1680,7 +1907,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1680
  trajectory_table = gr.Dataframe(
1681
  interactive=False,
1682
  label="Layer diagnostics",
1683
- buttons=["fullscreen"],
1684
  wrap=False,
1685
  max_height=380,
1686
  )
@@ -1696,10 +1923,10 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1696
  "Top-5 mass": INK_UMBER,
1697
  "Activation entropy": INK_RED,
1698
  },
1699
- title="Representation trajectory",
1700
  x_title="Layer",
1701
  y_title="Normalized value",
1702
- buttons=["fullscreen", "export"],
1703
  height=330,
1704
  )
1705
 
@@ -1708,7 +1935,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
1708
  gr.Markdown(
1709
  "The offline pipeline remains the source of held-out feature/concept AUROC + F1, reconstruction "
1710
  "quality, paraphrase stability, dense residual linear probes, and causal intervention conclusions. "
1711
- "v0.6 strengthens the live workbench with batch-context null references and random-control ensembles; "
1712
  "the saved report should be generated only from real experiment artifacts."
1713
  )
1714
 
@@ -1729,7 +1956,7 @@ $$h' = h + \sum_{i \in S}\Delta z_i d_i.$$
1729
 
1730
  FeatureLens patches the delta into the **original residual**; it never replaces the residual with the complete SAE reconstruction.
1731
 
1732
- ### v0.6 control discipline
1733
 
1734
  Batched experiments include an explicit **zero-edit row**. Causal effects are measured against that row rather than a separately executed baseline, which removes batch-vs-single floating-point drift from the measured effect. Random specificity uses an ensemble of norm-matched residual directions rather than one arbitrary seed.
1735
 
@@ -1738,7 +1965,7 @@ Batched experiments include an explicit **zero-edit row**. Causal effects are me
1738
  1. SAE reconstruction quality.
1739
  2. Held-out feature/concept prediction.
1740
  3. Concept-guided candidate discovery and selected-feature concept contrast.
1741
- 4. Token-local activation traces and completion-cue sensitivity.
1742
  5. Local and prompt-wide paraphrase robustness.
1743
  6. Single-feature causal intervention and dose-response.
1744
  7. Contrastive continuation preference under intervention.
@@ -1750,11 +1977,7 @@ Association, robustness, geometry, and intervention evidence remain separate cla
1750
  """
1751
  )
1752
 
1753
- gr.HTML(
1754
- '<footer class="classic-footer">Built with PyTorch, Transformers, Qwen3-1.7B-Base and Qwen-Scope '
1755
- 'residual-stream SAEs. FeatureLens is independent of thesis code and thesis datasets.<br><br>'
1756
- 'End of workbench.</footer>'
1757
- )
1758
 
1759
  demo.load(fn=None, js=INSTALL_REFLOW_JS, queue=False)
1760
 
@@ -1837,13 +2060,22 @@ Association, robustness, geometry, and intervention evidence remain separate cla
1837
  )
1838
  discovery_btn.click(
1839
  run_concept_feature_discovery,
1840
- inputs=[discovery_concept, discovery_layer, discovery_n, discovery_top_n],
 
 
 
1841
  outputs=[discovery_metrics, discovery_table, discovery_plot, discovery_candidate, discovery_tsv],
1842
  )
 
 
 
 
 
 
1843
  use_candidate_btn.click(
1844
  use_candidate_feature,
1845
  inputs=[discovery_candidate],
1846
- outputs=[feature_id, dose_feature_id, contrastive_feature_id, contrast_feature_id],
1847
  queue=False,
1848
  )
1849
  cue_btn.click(
@@ -1851,6 +2083,11 @@ Association, robustness, geometry, and intervention evidence remain separate cla
1851
  inputs=[contrast_feature_id, contrast_layer, cue_stem, cue_text],
1852
  outputs=[cue_metrics, cue_table, cue_plot, cue_tsv],
1853
  )
 
 
 
 
 
1854
  para_btn.click(
1855
  run_paraphrase_compare,
1856
  inputs=[para_a, para_b, para_layer, para_idx_a, para_idx_b, para_top_n],
@@ -1877,6 +2114,7 @@ Association, robustness, geometry, and intervention evidence remain separate cla
1877
  (contrast_copy, contrast_tsv),
1878
  (discovery_copy, discovery_tsv),
1879
  (cue_copy, cue_tsv),
 
1880
  (para_copy, para_tsv),
1881
  (trajectory_copy, trajectory_tsv),
1882
  ]:
@@ -1889,4 +2127,4 @@ if __name__ == "__main__":
1889
  theme=THEME,
1890
  ssr_mode=False,
1891
  show_error=True,
1892
- )
 
1
  from __future__ import annotations
2
 
 
3
  import gradio as gr
4
+ import pandas as pd
5
 
6
  from featurelens.config import SETTINGS
7
  from featurelens.hf_runtime import gpu
 
47
  .gradio-container h3 { font-size: 1.46rem; line-height: 1.28; }
48
  .gradio-container h4 { font-size: 1.24rem; line-height: 1.30; }
49
  .hero {
50
+ padding: 14px 2px 10px;
51
  border-bottom: 1px solid var(--border-color-primary);
52
+ margin-bottom: 14px;
53
  }
54
  .hero h1 {
55
  margin: 0;
56
+ font-size: 2.18rem;
57
  font-weight: 600;
58
  letter-spacing: 0;
59
  }
60
+ .hero .subtitle { margin-top: 4px; font-size: 1.08rem; opacity: .82; }
61
+ .hero .question { margin-top: 6px; font-size: 1rem; opacity: .72; }
 
 
 
 
 
 
 
 
62
  .section-rule {
63
  margin: 20px 0 11px;
64
  padding-top: 9px;
 
101
  .gradio-container textarea, .gradio-container input { border-radius: 2px !important; }
102
  .form-note { margin-top: -3px; margin-bottom: 9px; font-size: .92rem; opacity: .72; }
103
  .wide-table { width: 100% !important; }
104
+ .bottom-spacer { height: 110px; width: 100%; }
 
 
 
 
 
 
 
105
  .tabs, .tabitem { padding-bottom: 26px !important; }
106
  .context-card {
107
  border: 1px solid var(--border-color-primary);
 
120
  border-radius: 2px;
121
  }
122
  .start-card h3 { margin-top: 0; }
 
123
  .graph-note { font-size: .95rem; opacity: .72; margin-top: 2px; }
124
+ .result-table .label-wrap,
125
+ .result-table .label-wrap span,
126
+ .result-table label,
127
+ .result-table label span,
128
+ .result-table [data-testid="block-label"],
129
+ .result-table [data-testid="block-label"] span,
130
+ .result-table .block-label,
131
+ .result-table .block-title {
132
+ font-size: 19px !important;
133
+ font-weight: 700 !important;
134
+ line-height: 1.30 !important;
135
+ }
136
+ .result-table table thead th,
137
+ .result-table table thead th *,
138
+ .result-table [role="columnheader"],
139
+ .result-table [role="columnheader"] * {
140
+ font-size: 16.75px !important;
141
+ font-weight: 700 !important;
142
+ line-height: 1.25 !important;
143
+ }
144
+ /* Native fullscreen is converted into a bounded focus surface. Tables may be wider; plots stay
145
+ narrower so chart labels can be read without scanning across an entire monitor. */
146
+ .fl-plot.fullscreen {
147
+ top: 6vh !important;
148
+ left: 50% !important;
149
+ transform: translateX(-50%) !important;
150
+ width: min(90vw, 1120px) !important;
151
+ height: 88vh !important;
152
+ padding: 18px 22px 22px !important;
153
+ border-radius: 4px !important;
154
+ overflow: auto !important;
155
+ box-shadow: 0 0 0 100vmax rgba(0, 0, 0, .66), 0 14px 40px rgba(0, 0, 0, .35) !important;
156
+ }
157
+ .result-table.fullscreen {
158
+ top: 5vh !important;
159
+ left: 50% !important;
160
+ transform: translateX(-50%) !important;
161
+ width: min(94vw, 1420px) !important;
162
+ height: 90vh !important;
163
+ padding: 18px 22px 24px !important;
164
+ border-radius: 4px !important;
165
+ overflow: auto !important;
166
+ box-shadow: 0 0 0 100vmax rgba(0, 0, 0, .66), 0 14px 40px rgba(0, 0, 0, .35) !important;
167
+ }
168
+ .fl-plot.fullscreen svg { max-width: 100% !important; }
169
+ .candidate-help { opacity: .78; font-size: .97rem; margin-top: -2px; }
170
  @media (max-width: 900px) {
171
  .gradio-container { width: 100% !important; padding-left: 12px !important; padding-right: 12px !important; }
172
  }
 
235
  const mutation = new MutationObserver(kick);
236
  mutation.observe(root, {subtree: true, childList: true});
237
  window.__featurelens_mutation_observer = mutation;
238
+
239
+ // Keep the page anchored when entering/exiting native fullscreen. Gradio toggles fullscreen on the
240
+ // same component node, so FeatureLens can focus the component without navigating away from its source location.
241
+ document.addEventListener("click", (event) => {
242
+ const button = event.target && event.target.closest ? event.target.closest("button") : null;
243
+ if (!button) return;
244
+ const label = `${button.getAttribute("aria-label") || ""} ${button.getAttribute("title") || ""} ${button.textContent || ""}`.toLowerCase();
245
+ if (!label.includes("fullscreen")) return;
246
+ const block = button.closest(".fl-plot, .result-table");
247
+ if (!block) return;
248
+ const entering = !block.classList.contains("fullscreen");
249
+ if (entering) {
250
+ window.__featurelens_focus_scroll_y = window.scrollY;
251
+ window.__featurelens_focus_block = block;
252
+ window.setTimeout(() => {
253
+ block.scrollTop = 0;
254
+ if (window.__featurelens_focus_scroll_y != null) window.scrollTo(0, window.__featurelens_focus_scroll_y);
255
+ }, 80);
256
+ } else {
257
+ const y = window.__featurelens_focus_scroll_y;
258
+ window.setTimeout(() => {
259
+ if (y != null) window.scrollTo(0, y);
260
+ window.__featurelens_focus_scroll_y = null;
261
+ window.__featurelens_focus_block = null;
262
+ }, 80);
263
+ }
264
+ }, true);
265
+
266
+ // Gradio's native plot exporter currently emits chart.png. Remember which FeatureLens plot initiated
267
+ // the export, then rename the transient download anchor before the browser handles it.
268
+ document.addEventListener("click", (event) => {
269
+ const button = event.target && event.target.closest ? event.target.closest("button") : null;
270
+ if (button) {
271
+ const label = `${button.getAttribute("aria-label") || ""} ${button.getAttribute("title") || ""} ${button.textContent || ""}`.toLowerCase();
272
+ if (label.includes("export")) {
273
+ const block = button.closest(".fl-plot");
274
+ if (block) {
275
+ const id = block.id || "plot-featurelens-chart";
276
+ const stem = id.replace(/^plot-/, "").replace(/[^a-z0-9_-]+/gi, "-").replace(/-+/g, "-").replace(/^-|-$/g, "");
277
+ window.__featurelens_export_name = `featurelens_${stem || "chart"}.png`;
278
+ }
279
+ }
280
+ }
281
+ const anchor = event.target && event.target.closest ? event.target.closest('a[download="chart.png"]') : null;
282
+ if (anchor && window.__featurelens_export_name) {
283
+ anchor.setAttribute("download", window.__featurelens_export_name);
284
+ window.setTimeout(() => { window.__featurelens_export_name = null; }, 500);
285
+ }
286
+ }, true);
287
+
288
  kick();
289
  return [];
290
  }
 
519
  f"No positively selective candidate features were found for **{result.concept}** at layer "
520
  f"**{result.layer}** in this small live batch. Increase the sample or try another layer."
521
  )
522
+ ranking = (
523
+ "balanced selectivity × coverage × log-magnitude"
524
+ if result.ranking_mode == "balanced_selectivity"
525
+ else "raw target-minus-other mean activation"
526
+ )
527
+ context = (
528
+ f"Current Workbench token **{result.current_token_index}** was checked in the same model batch. "
529
+ "The default candidate prefers the highest-ranked feature active at that token when one exists."
530
+ if result.current_context_available
531
+ else "No Workbench prompt was available, so current-prompt compatibility was not evaluated."
532
+ )
533
  return (
534
  f"Concept **{result.concept}** · layer **{result.layer}** · "
535
  f"{result.prompts_per_concept} prompts/concept. \n"
536
+ f"Showing **{len(result.candidate_ids)}** candidates ranked by **{ranking}**. \n"
537
+ f"{context} \n\n"
538
+ "This is **candidate discovery**, not a semantic label. The offline held-out AUROC/F1 pipeline remains "
539
+ "the standard for a concept-selectivity claim."
540
+ )
541
+
542
+
543
+ def _cue_context_metrics_markdown(result) -> str:
544
+ active = ", ".join(
545
+ f"{cue}: {count}/{len(result.stems)} contexts"
546
+ for cue, count in result.cue_active_context_counts.items()
547
+ )
548
+ return (
549
+ f"Feature **{result.feature_id}**, layer **{result.layer}** · active in "
550
+ f"**{result.active_condition_count}/{result.condition_count}** stem × cue conditions. \n"
551
+ f"Cue coverage — {active}. \n\n"
552
+ "A cue that activates across unrelated stems looks more lexical/cue-specific; activation only in a subset "
553
+ "of semantically related stems is more consistent with context-sensitive completion structure. This is still "
554
+ "a diagnostic, not a semantic label."
555
  )
556
 
557
 
 
1023
 
1024
 
1025
  @gpu(duration=35)
1026
+ def run_concept_feature_discovery(
1027
+ concept: str,
1028
+ layer: int,
1029
+ prompts_per_concept: int,
1030
+ top_n: int,
1031
+ ranking_label: str,
1032
+ workbench_prompt: str,
1033
+ workbench_token_index: int,
1034
+ ):
1035
  try:
1036
+ ranking_mode = (
1037
+ "balanced_selectivity"
1038
+ if ranking_label == "Balanced selectivity"
1039
+ else "raw_mean_difference"
1040
+ )
1041
  result = RUNTIME.concept_feature_discovery(
1042
  concept=concept,
1043
  layer=int(layer),
1044
  prompts_per_concept=int(prompts_per_concept),
1045
  top_n=int(top_n),
1046
+ ranking_mode=ranking_mode,
1047
+ current_text=workbench_prompt,
1048
+ current_token_index=int(workbench_token_index),
1049
  )
1050
  columns = [
1051
  "Rank",
1052
  "Feature id",
1053
+ "Candidate score",
1054
  "Target mean max",
1055
  "Other mean max",
1056
  "Mean difference",
1057
+ "Selectivity",
1058
  "Target activation rate",
1059
  "Other activation rate",
1060
+ "Current prompt max",
1061
+ "Current token activation",
1062
+ "Active at current token",
1063
  ]
1064
  table = pd.DataFrame(result.rows, columns=columns)
1065
+ chart = pd.DataFrame(result.chart_rows, columns=["Feature", "Candidate score"])
1066
+ chart["Series"] = "Candidate score"
1067
  choices = [str(fid) for fid in result.candidate_ids]
1068
+ default = str(result.default_candidate_id) if result.default_candidate_id is not None else (choices[0] if choices else None)
1069
+ candidate_update = gr.update(choices=choices, value=default)
1070
  return _discovery_metrics_markdown(result), table, chart, candidate_update, _tsv(table)
1071
  except Exception as exc:
1072
  _raise_ui_error(exc)
1073
 
1074
 
1075
+
1076
  @gpu(duration=25)
1077
  def run_feature_cue_scan(feature_id: str, layer: int, prompt_stem: str, cue_text: str):
1078
  try:
 
1094
  _raise_ui_error(exc)
1095
 
1096
 
1097
+ @gpu(duration=30)
1098
+ def run_feature_cue_context_scan(feature_id: str, layer: int, stems_text: str, cue_text: str):
1099
+ try:
1100
+ if feature_id is None or str(feature_id).strip() == "":
1101
+ raise ValueError("Choose a feature id first.")
1102
+ stems = [line for line in str(stems_text).splitlines() if line.strip()]
1103
+ cues = [line for line in str(cue_text).splitlines() if line.strip()]
1104
+ result = RUNTIME.feature_cue_context_scan(
1105
+ feature_id=int(float(feature_id)),
1106
+ layer=int(layer),
1107
+ stems=stems,
1108
+ cues=cues,
1109
+ )
1110
+ columns = ["Prompt stem", "Cue", "Full prompt", "Final token", "Activation", "Active in TopK"]
1111
+ table = pd.DataFrame(result.rows, columns=columns)
1112
+ chart = pd.DataFrame(result.chart_rows, columns=["Prompt stem", "Cue", "Activation"])
1113
+ return _cue_context_metrics_markdown(result), table, chart, _tsv(table)
1114
+ except Exception as exc:
1115
+ _raise_ui_error(exc)
1116
+
1117
+
1118
+ def select_candidate_row(table: pd.DataFrame, evt: gr.SelectData):
1119
+ if table is None or len(table) == 0:
1120
+ return gr.update()
1121
+ index = evt.index
1122
+ row_index = int(index[0] if isinstance(index, (tuple, list)) else index)
1123
+ if row_index < 0 or row_index >= len(table):
1124
+ return gr.update()
1125
+ value = str(int(float(table.iloc[row_index]["Feature id"])))
1126
+ return gr.update(value=value)
1127
+
1128
+
1129
  def use_candidate_feature(candidate_id: str):
1130
  if candidate_id is None or str(candidate_id).strip() == "":
1131
  raise gr.Error("Run concept-guided discovery and choose a candidate first.")
1132
  value = str(int(float(candidate_id)))
1133
+ status = (
1134
+ f"**Feature {value} loaded.** It is now selected in Single-feature causal test, "
1135
+ "Scale dose-response, Contrastive preference, and Feature evidence. You can still change any selector independently."
1136
+ )
1137
+ return value, value, value, value, status
1138
 
1139
 
1140
  def mode_help(mode: str):
 
1154
  with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_width=True) as demo:
1155
  gr.HTML(
1156
  '<header class="hero">'
1157
+ '<h1>FeatureLens <span style="font-size:.48em;opacity:.58;font-weight:400">v0.7</span></h1>'
1158
  '<div class="subtitle">Causal Interpretability Workbench</div>'
1159
+ '<div class="question">Discover sparse features, test robustness, and separate correlation from causal influence.</div>'
 
1160
  '</header>'
 
 
1161
  )
1162
 
1163
  global_context = gr.Markdown(
 
1196
  "- **Ablate:** remove that feature's current contribution. **Scale:** multiply it.\n"
1197
  "- **Random control:** an equally large residual edit in an unrelated direction.\n"
1198
  "- **Teacher-forced score:** probability assigned to exact continuation text, even if greedy generation does not change.\n\n"
1199
+ "**Tables:** use the top-right focus control or copy-with-headers action. **Plots:** use the top-right focus control for a bounded reading view; PNG exports use descriptive FeatureLens filenames."
1200
  )
1201
 
1202
  with gr.Tab("Workbench"):
 
1213
  examples=[
1214
  ["The derivative of x squared is"],
1215
  ["In Python, reverse a list using"],
1216
+ ["Ich möchte einen Tisch für zwei reservieren."],
1217
  ["I am not fully certain, but the answer may be"],
1218
  ],
1219
  inputs=[prompt],
 
1248
  label="Strongest active SAE features",
1249
  wrap=False,
1250
  max_height=380,
1251
+ buttons=["fullscreen"], elem_classes=["result-table"],
1252
  )
1253
  feature_tsv = gr.Textbox(visible="hidden")
1254
  feature_copy = _copy_button()
 
1258
  y="Activation",
1259
  color="Series",
1260
  color_map={"Activation": INK_TEAL},
1261
+ title="Activation profile", elem_id="plot-activation-profile",
1262
  x_title="Feature id",
1263
  y_title="Activation",
1264
  x_label_angle=-35,
1265
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1266
  height=330,
1267
  )
1268
 
 
1310
  token_prob_table = gr.Dataframe(
1311
  interactive=False,
1312
  label="Next-token distribution shift",
1313
+ buttons=["fullscreen"], elem_classes=["result-table"],
1314
  wrap=False,
1315
  max_height=380,
1316
  )
 
1319
  target_token_table = gr.Dataframe(
1320
  interactive=False,
1321
  label="Target continuation token-by-token score",
1322
+ buttons=["fullscreen"], elem_classes=["result-table"],
1323
  wrap=False,
1324
  max_height=380,
1325
  )
 
1327
  target_token_copy = _copy_button()
1328
 
1329
  gr.HTML('<div class="section-rule">III. Single-feature scale dose-response</div>')
1330
+ with gr.Group():
 
1331
  gr.Markdown("**Uses current Workbench prompt / layer / token.** Choose the feature for this experiment below.")
1332
  dose_feature_id = gr.Dropdown(
1333
  choices=[],
 
1347
  dose_table = gr.Dataframe(
1348
  interactive=False,
1349
  label="Scale dose-response measurements",
1350
+ buttons=["fullscreen"], elem_classes=["result-table"],
1351
  wrap=False,
1352
  max_height=380,
1353
  )
 
1359
  y="Δ mean log p/token",
1360
  color="Series",
1361
  color_map={"SAE feature": INK_TEAL},
1362
+ title="Scale dose-response", elem_id="plot-scale-dose-response",
1363
  x_title="Feature multiplier",
1364
  y_title="Δ mean log p/token",
1365
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1366
  height=330,
1367
  )
1368
 
1369
  gr.HTML('<div class="section-rule">IV. Contrastive continuation preference</div>')
1370
+ with gr.Group():
 
1371
  gr.Markdown("**Uses current Workbench prompt / layer / token.** Select the feature explicitly below.")
1372
  contrastive_feature_id = gr.Dropdown(
1373
  choices=[],
 
1405
  contrastive_table = gr.Dataframe(
1406
  interactive=False,
1407
  label="Contrastive continuation scores",
1408
+ buttons=["fullscreen"], elem_classes=["result-table"],
1409
  wrap=False,
1410
  max_height=320,
1411
  )
 
1417
  y="A−B sequence log-odds",
1418
  color="Series",
1419
  color_map={"Contrastive preference": INK_TEAL},
1420
+ title="Preference between exact continuations", elem_id="plot-contrastive-preference",
1421
  x_title="Execution condition",
1422
  y_title="Sequence log-odds A−B",
1423
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1424
  height=320,
1425
  )
1426
 
 
1472
  set_feature_table = gr.Dataframe(
1473
  interactive=False,
1474
  label="Joint intervention features",
1475
+ buttons=["fullscreen"], elem_classes=["result-table"],
1476
  wrap=False,
1477
  max_height=380,
1478
  )
 
1482
  set_target_table = gr.Dataframe(
1483
  interactive=False,
1484
  label="Target continuation token-by-token score",
1485
+ buttons=["fullscreen"], elem_classes=["result-table"],
1486
  wrap=False,
1487
  max_height=380,
1488
  )
 
1502
  set_sweep_table = gr.Dataframe(
1503
  interactive=False,
1504
  label="Feature-set size measurements",
1505
+ buttons=["fullscreen"], elem_classes=["result-table"],
1506
  wrap=False,
1507
  max_height=380,
1508
  )
 
1517
  "Top-k SAE ablation": INK_TEAL,
1518
  "Random signed mean": INK_STONE,
1519
  },
1520
+ title="Effect vs feature-set size", elem_id="plot-feature-set-size",
1521
  x_title="Number of jointly ablated features",
1522
  y_title="Δ mean log p/token",
1523
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1524
  height=330,
1525
  )
1526
 
 
1538
  interaction_table = gr.Dataframe(
1539
  interactive=False,
1540
  label="Individual and joint ablation measurements",
1541
+ buttons=["fullscreen"], elem_classes=["result-table"],
1542
  wrap=False,
1543
  max_height=380,
1544
  )
 
1550
  y="Δ mean log p/token",
1551
  color="Series",
1552
  color_map={"Ablation effect": INK_UMBER},
1553
+ title="Individual vs joint effect", elem_id="plot-individual-vs-joint",
1554
  x_title="Intervention condition",
1555
  y_title="Δ mean log p/token",
1556
  x_label_angle=-25,
1557
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1558
  height=330,
1559
  )
1560
 
1561
 
1562
  gr.HTML('<div class="section-rule">Decoder geometry</div>')
1563
+ with gr.Group():
 
1564
  gr.Markdown(
1565
  "Joint causal effects can reflect both **decoder-direction geometry** and downstream non-linearity. "
1566
  "This zero-extra-generation diagnostic measures pairwise decoder cosines and compares the actual "
 
1575
  geometry_table = gr.Dataframe(
1576
  interactive=False,
1577
  label="Pairwise decoder geometry",
1578
+ buttons=["fullscreen"], elem_classes=["result-table"],
1579
  wrap=False,
1580
  max_height=340,
1581
  )
 
1587
  y="Decoder cosine",
1588
  color="Series",
1589
  color_map={"Decoder cosine": INK_PLUM},
1590
+ title="Pairwise SAE decoder cosine", elem_id="plot-decoder-geometry",
1591
  x_title="Feature pair",
1592
  y_title="Cosine similarity",
1593
  x_label_angle=-30,
1594
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1595
  height=320,
1596
  )
1597
 
 
1604
  )
1605
  gr.HTML('<div class="section-rule">A. Concept-guided candidate discovery</div>')
1606
  gr.Markdown(
1607
+ "Start from a concept instead of an anonymous feature id. The default **Balanced selectivity** ranking "
1608
+ "downweights globally high-activation features and favors candidates that are both selective and consistently "
1609
+ "active for the target concept. The table also checks whether each candidate is active in the current Workbench "
1610
+ "prompt and selected token, so you can tell whether it is immediately usable for a causal test."
1611
  )
1612
  with gr.Row(equal_height=True):
1613
  discovery_concept = gr.Dropdown(
1614
  choices=[
1615
  "code", "mathematics", "positive_sentiment", "negative_sentiment",
1616
+ "german_language", "factual_entities", "uncertainty"
1617
  ],
1618
  value="mathematics",
1619
  label="Target concept",
 
1621
  discovery_layer = gr.Dropdown(choices=list(SETTINGS.layers), value=SETTINGS.layers[1], label="Residual layer")
1622
  discovery_n = gr.Slider(2, 6, value=SETTINGS.contrast_prompts_per_concept, step=1, label="Prompts per concept")
1623
  discovery_top_n = gr.Slider(5, 20, value=12, step=1, label="Candidate features")
1624
+ with gr.Row(equal_height=True):
1625
+ discovery_ranking = gr.Dropdown(
1626
+ choices=["Balanced selectivity", "Raw mean difference"],
1627
+ value="Balanced selectivity",
1628
+ label="Candidate ranking",
1629
+ info="Balanced selectivity is the recommended exploratory ranking; raw mean difference reproduces the earlier scale-dominated ordering.",
1630
+ scale=2,
1631
+ )
1632
+ gr.Markdown(
1633
+ "**Input source:** controlled concept prompts + the current Workbench prompt/token, evaluated in the same GPU batch.",
1634
+ elem_classes=["instrument-note"],
1635
+ )
1636
  discovery_btn = gr.Button("Discover concept-associated candidates", variant="primary", elem_classes=["action-btn"])
1637
  discovery_metrics = gr.Markdown()
1638
  with gr.Row(equal_height=False):
1639
  with gr.Column(scale=3):
1640
  discovery_table = gr.Dataframe(
1641
+ interactive=False, label="Candidate feature evidence", buttons=["fullscreen"], elem_classes=["result-table"],
1642
+ wrap=False, max_height=420
1643
  )
1644
  discovery_tsv = gr.Textbox(visible="hidden")
1645
  discovery_copy = _copy_button()
1646
  with gr.Column(scale=2):
1647
  discovery_plot = gr.BarPlot(
1648
+ x="Feature", y="Candidate score", color="Series",
1649
+ color_map={"Candidate score": INK_TEAL}, title="Candidate evidence score", elem_id="plot-candidate-discovery",
1650
+ x_title="Feature id", y_title="Exploratory ranking score", x_label_angle=-35,
1651
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"], height=330
1652
  )
1653
+ gr.Markdown(
1654
+ "Click a row in **Candidate feature evidence** to place that feature in the selector below. "
1655
+ "The selector defaults to the highest-ranked candidate active at the current Workbench token when available.",
1656
+ elem_classes=["candidate-help"],
1657
+ )
1658
  with gr.Row(equal_height=True):
1659
+ discovery_candidate = gr.Dropdown(choices=[], label="Selected candidate feature id", allow_custom_value=True, scale=3)
1660
+ use_candidate_btn = gr.Button(
1661
+ "Use selected candidate across feature tests", variant="primary", elem_classes=["action-btn"], scale=2
1662
+ )
1663
+ candidate_use_status = gr.Markdown()
1664
 
1665
  gr.HTML('<div class="section-rule">B. Inspect one feature</div>')
1666
  contrast_location = gr.Markdown(
 
1700
  trace_table = gr.Dataframe(
1701
  interactive=False,
1702
  label="Feature activation by prompt token",
1703
+ buttons=["fullscreen"], elem_classes=["result-table"],
1704
  wrap=False,
1705
  max_height=340,
1706
  )
 
1712
  y="Activation",
1713
  color="Series",
1714
  color_map={"Feature activation": INK_TEAL},
1715
+ title="Feature activation across prompt tokens", elem_id="plot-feature-token-trace",
1716
  x_title="Prompt token",
1717
  y_title="Activation",
1718
  x_label_angle=-35,
1719
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1720
  height=320,
1721
  )
1722
 
 
1733
  cue_metrics = gr.Markdown()
1734
  with gr.Row(equal_height=False):
1735
  with gr.Column(scale=3):
1736
+ cue_table = gr.Dataframe(interactive=False, label="Feature response by completion cue", buttons=["fullscreen"], elem_classes=["result-table"], wrap=False, max_height=340)
1737
  cue_tsv = gr.Textbox(visible="hidden")
1738
  cue_copy = _copy_button()
1739
  with gr.Column(scale=2):
1740
  cue_plot = gr.BarPlot(
1741
  x="Cue", y="Activation", color="Series", color_map={"Cue response": INK_UMBER},
1742
+ title="Completion-cue feature response", elem_id="plot-completion-cue-response", x_title="Cue", y_title="Final-token activation",
1743
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"], height=320
1744
+ )
1745
+
1746
+ gr.HTML('<div class="section-rule">D. Cue × context specificity</div>')
1747
+ gr.Markdown(
1748
+ "The single-stem cue scan can tell you that a feature prefers `is`, but not whether that preference is lexical "
1749
+ "or context-sensitive. This matrix crosses several prompt stems with the same completion cues in one batch."
1750
+ )
1751
+ with gr.Row(equal_height=True):
1752
+ cue_context_stems = gr.Textbox(
1753
+ label="Prompt stems (one per line)",
1754
+ value="The derivative of x squared\nThe capital of Germany\nThe weather today\nMy name",
1755
+ lines=5,
1756
+ scale=3,
1757
+ )
1758
+ cue_context_cues = gr.Textbox(
1759
+ label="Completion cues (one per line)",
1760
+ value="is\n=\n:\nequals\ntherefore",
1761
+ lines=5,
1762
+ scale=2,
1763
+ )
1764
+ cue_context_btn = gr.Button("Run cue × context scan", variant="primary", elem_classes=["action-btn"])
1765
+ cue_context_metrics = gr.Markdown()
1766
+ with gr.Row(equal_height=False):
1767
+ with gr.Column(scale=3):
1768
+ cue_context_table = gr.Dataframe(
1769
+ interactive=False,
1770
+ label="Cue × context feature response",
1771
+ buttons=["fullscreen"],
1772
+ elem_classes=["result-table"],
1773
+ wrap=False,
1774
+ max_height=420,
1775
+ )
1776
+ cue_context_tsv = gr.Textbox(visible="hidden")
1777
+ cue_context_copy = _copy_button()
1778
+ with gr.Column(scale=2):
1779
+ cue_context_plot = gr.BarPlot(
1780
+ x="Prompt stem",
1781
+ y="Activation",
1782
+ color="Cue",
1783
+ title="Cue response across contexts",
1784
+ elem_id="plot-cue-context-matrix",
1785
+ x_title="Prompt stem",
1786
+ y_title="Final-token activation",
1787
+ x_label_angle=-25,
1788
+ buttons=["fullscreen", "export"],
1789
+ elem_classes=["fl-plot"],
1790
+ height=340,
1791
  )
1792
 
1793
+ gr.HTML('<div class="section-rule">E. Controlled concept contrast for selected feature</div>')
1794
  gr.Markdown("### Prompt-wide controlled concept contrast")
1795
  contrast_btn = gr.Button("Run controlled concept contrast", variant="primary", elem_classes=["action-btn"])
1796
  contrast_metrics = gr.Markdown()
 
1799
  contrast_table = gr.Dataframe(
1800
  interactive=False,
1801
  label="Feature activation by controlled concept",
1802
+ buttons=["fullscreen"], elem_classes=["result-table"],
1803
  wrap=False,
1804
  max_height=380,
1805
  )
 
1811
  y="Mean prompt-wide max",
1812
  color="Series",
1813
  color_map={"Prompt-wide max": INK_BLUEGREY},
1814
+ title="Prompt-wide controlled concept contrast", elem_id="plot-controlled-concept-contrast",
1815
  x_title="Concept",
1816
  y_title="Mean max activation",
1817
  x_label_angle=-25,
1818
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1819
  height=330,
1820
  )
1821
 
 
1860
  para_table = gr.Dataframe(
1861
  interactive=False,
1862
  label="Top-feature overlap at selected tokens",
1863
+ buttons=["fullscreen"], elem_classes=["result-table"],
1864
  wrap=False,
1865
  max_height=380,
1866
  )
 
1872
  y="Activation",
1873
  color="Prompt",
1874
  color_map={"Original": INK_TEAL, "Paraphrase": INK_PLUM},
1875
+ title="Selected-token feature activations", elem_id="plot-paraphrase-selected-token",
1876
  x_title="Feature id",
1877
  y_title="Activation",
1878
  x_label_angle=-35,
1879
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1880
  height=330,
1881
  )
1882
 
 
1907
  trajectory_table = gr.Dataframe(
1908
  interactive=False,
1909
  label="Layer diagnostics",
1910
+ buttons=["fullscreen"], elem_classes=["result-table"],
1911
  wrap=False,
1912
  max_height=380,
1913
  )
 
1923
  "Top-5 mass": INK_UMBER,
1924
  "Activation entropy": INK_RED,
1925
  },
1926
+ title="Representation trajectory", elem_id="plot-layer-trajectory",
1927
  x_title="Layer",
1928
  y_title="Normalized value",
1929
+ buttons=["fullscreen", "export"], elem_classes=["fl-plot"],
1930
  height=330,
1931
  )
1932
 
 
1935
  gr.Markdown(
1936
  "The offline pipeline remains the source of held-out feature/concept AUROC + F1, reconstruction "
1937
  "quality, paraphrase stability, dense residual linear probes, and causal intervention conclusions. "
1938
+ "The live workbench uses batch-context null references and random-control ensembles; "
1939
  "the saved report should be generated only from real experiment artifacts."
1940
  )
1941
 
 
1956
 
1957
  FeatureLens patches the delta into the **original residual**; it never replaces the residual with the complete SAE reconstruction.
1958
 
1959
+ ### Control discipline
1960
 
1961
  Batched experiments include an explicit **zero-edit row**. Causal effects are measured against that row rather than a separately executed baseline, which removes batch-vs-single floating-point drift from the measured effect. Random specificity uses an ensemble of norm-matched residual directions rather than one arbitrary seed.
1962
 
 
1965
  1. SAE reconstruction quality.
1966
  2. Held-out feature/concept prediction.
1967
  3. Concept-guided candidate discovery and selected-feature concept contrast.
1968
+ 4. Token-local activation traces, completion-cue sensitivity, and cue × context specificity.
1969
  5. Local and prompt-wide paraphrase robustness.
1970
  6. Single-feature causal intervention and dose-response.
1971
  7. Contrastive continuation preference under intervention.
 
1977
  """
1978
  )
1979
 
1980
+ gr.HTML('<div class="bottom-spacer" aria-hidden="true"></div>')
 
 
 
 
1981
 
1982
  demo.load(fn=None, js=INSTALL_REFLOW_JS, queue=False)
1983
 
 
2060
  )
2061
  discovery_btn.click(
2062
  run_concept_feature_discovery,
2063
+ inputs=[
2064
+ discovery_concept, discovery_layer, discovery_n, discovery_top_n, discovery_ranking,
2065
+ prompt, token_index,
2066
+ ],
2067
  outputs=[discovery_metrics, discovery_table, discovery_plot, discovery_candidate, discovery_tsv],
2068
  )
2069
+ discovery_table.select(
2070
+ select_candidate_row,
2071
+ inputs=[discovery_table],
2072
+ outputs=[discovery_candidate],
2073
+ queue=False,
2074
+ )
2075
  use_candidate_btn.click(
2076
  use_candidate_feature,
2077
  inputs=[discovery_candidate],
2078
+ outputs=[feature_id, dose_feature_id, contrastive_feature_id, contrast_feature_id, candidate_use_status],
2079
  queue=False,
2080
  )
2081
  cue_btn.click(
 
2083
  inputs=[contrast_feature_id, contrast_layer, cue_stem, cue_text],
2084
  outputs=[cue_metrics, cue_table, cue_plot, cue_tsv],
2085
  )
2086
+ cue_context_btn.click(
2087
+ run_feature_cue_context_scan,
2088
+ inputs=[contrast_feature_id, contrast_layer, cue_context_stems, cue_context_cues],
2089
+ outputs=[cue_context_metrics, cue_context_table, cue_context_plot, cue_context_tsv],
2090
+ )
2091
  para_btn.click(
2092
  run_paraphrase_compare,
2093
  inputs=[para_a, para_b, para_layer, para_idx_a, para_idx_b, para_top_n],
 
2114
  (contrast_copy, contrast_tsv),
2115
  (discovery_copy, discovery_tsv),
2116
  (cue_copy, cue_tsv),
2117
+ (cue_context_copy, cue_context_tsv),
2118
  (para_copy, para_tsv),
2119
  (trajectory_copy, trajectory_tsv),
2120
  ]:
 
2127
  theme=THEME,
2128
  ssr_mode=False,
2129
  show_error=True,
2130
+ )
data/causal_tasks.jsonl CHANGED
@@ -14,10 +14,10 @@
14
  {"id": 13, "concept": "code", "prompt": "In Python, an exception handler begins with the keyword", "target": " except"}
15
  {"id": 14, "concept": "code", "prompt": "SQL keyword used to retrieve rows:", "target": " SELECT"}
16
  {"id": 15, "concept": "code", "prompt": "A Python conditional branch commonly starts with", "target": " if"}
17
- {"id": 16, "concept": "french_language", "prompt": "Translate 'hello' into French:", "target": " bonjour"}
18
- {"id": 17, "concept": "french_language", "prompt": "Translate 'thank you' into French:", "target": " merci"}
19
- {"id": 18, "concept": "french_language", "prompt": "Translate 'yes' into French:", "target": " oui"}
20
- {"id": 19, "concept": "french_language", "prompt": "Translate 'good evening' into French:", "target": " bonsoir"}
21
  {"id": 20, "concept": "factual_entities", "prompt": "The scientist associated with radium research, Marie", "target": " Curie"}
22
  {"id": 21, "concept": "factual_entities", "prompt": "The composer of the Fifth Symphony, Ludwig van", "target": " Beethoven"}
23
  {"id": 22, "concept": "factual_entities", "prompt": "The computer scientist known for the Turing machine, Alan", "target": " Turing"}
 
14
  {"id": 13, "concept": "code", "prompt": "In Python, an exception handler begins with the keyword", "target": " except"}
15
  {"id": 14, "concept": "code", "prompt": "SQL keyword used to retrieve rows:", "target": " SELECT"}
16
  {"id": 15, "concept": "code", "prompt": "A Python conditional branch commonly starts with", "target": " if"}
17
+ {"id": 16, "concept": "german_language", "prompt": "Translate 'hello' into German:", "target": " hallo"}
18
+ {"id": 17, "concept": "german_language", "prompt": "Translate 'thank you' into German:", "target": " danke"}
19
+ {"id": 18, "concept": "german_language", "prompt": "Translate 'yes' into German:", "target": " ja"}
20
+ {"id": 19, "concept": "german_language", "prompt": "Translate 'good evening' into German:", "target": " guten Abend"}
21
  {"id": 20, "concept": "factual_entities", "prompt": "The scientist associated with radium research, Marie", "target": " Curie"}
22
  {"id": 21, "concept": "factual_entities", "prompt": "The composer of the Fifth Symphony, Ludwig van", "target": " Beethoven"}
23
  {"id": 22, "concept": "factual_entities", "prompt": "The computer scientist known for the Turing machine, Alan", "target": " Turing"}
data/prompts.jsonl CHANGED
@@ -126,38 +126,38 @@
126
  {"id": 125, "concept": "negative_sentiment", "pair_id": "negative_sentiment-14", "variant": 1, "text": "I thought the exhibition was dull and unrewarding."}
127
  {"id": 126, "concept": "negative_sentiment", "pair_id": "negative_sentiment-15", "variant": 0, "text": "The train journey was stressful and uncomfortable."}
128
  {"id": 127, "concept": "negative_sentiment", "pair_id": "negative_sentiment-15", "variant": 1, "text": "The trip by train felt frustrating, noisy, and unpleasant."}
129
- {"id": 128, "concept": "french_language", "pair_id": "french_language-00", "variant": 0, "text": "Bonjour, comment allez-vous aujourd’hui ?"}
130
- {"id": 129, "concept": "french_language", "pair_id": "french_language-00", "variant": 1, "text": "Salut, comment vas-tu aujourd’hui ?"}
131
- {"id": 130, "concept": "french_language", "pair_id": "french_language-01", "variant": 0, "text": "Je voudrais réserver une table pour deux personnes."}
132
- {"id": 131, "concept": "french_language", "pair_id": "french_language-01", "variant": 1, "text": "Puis-je réserver une table pour deux, s’il vous plaît ?"}
133
- {"id": 132, "concept": "french_language", "pair_id": "french_language-02", "variant": 0, "text": "La bibliothèque ferme à dix-huit heures."}
134
- {"id": 133, "concept": "french_language", "pair_id": "french_language-02", "variant": 1, "text": "La bibliothèque est ouverte jusqu’à dix-huit heures."}
135
- {"id": 134, "concept": "french_language", "pair_id": "french_language-03", "variant": 0, "text": "Ce livre raconte une histoire très intéressante."}
136
- {"id": 135, "concept": "french_language", "pair_id": "french_language-03", "variant": 1, "text": "L’histoire racontée dans ce livre est vraiment intéressante."}
137
- {"id": 136, "concept": "french_language", "pair_id": "french_language-04", "variant": 0, "text": "Nous allons prendre le train demain matin."}
138
- {"id": 137, "concept": "french_language", "pair_id": "french_language-04", "variant": 1, "text": "Demain matin, nous voyagerons en train."}
139
- {"id": 138, "concept": "french_language", "pair_id": "french_language-05", "variant": 0, "text": "Pouvez-vous m’indiquer le chemin vers la gare ?"}
140
- {"id": 139, "concept": "french_language", "pair_id": "french_language-05", "variant": 1, "text": "Comment puis-je aller jusqu’à la gare ?"}
141
- {"id": 140, "concept": "french_language", "pair_id": "french_language-06", "variant": 0, "text": "J’aime apprendre de nouvelles langues."}
142
- {"id": 141, "concept": "french_language", "pair_id": "french_language-06", "variant": 1, "text": "Apprendre des langues nouvelles me plaît beaucoup."}
143
- {"id": 142, "concept": "french_language", "pair_id": "french_language-07", "variant": 0, "text": "Le temps est magnifique au bord de la mer."}
144
- {"id": 143, "concept": "french_language", "pair_id": "french_language-07", "variant": 1, "text": "Il fait très beau près de la mer."}
145
- {"id": 144, "concept": "french_language", "pair_id": "french_language-08", "variant": 0, "text": "Elle prépare le dîner dans la cuisine."}
146
- {"id": 145, "concept": "french_language", "pair_id": "french_language-08", "variant": 1, "text": "Dans la cuisine, elle est en train de préparer le repas du soir."}
147
- {"id": 146, "concept": "french_language", "pair_id": "french_language-09", "variant": 0, "text": "Nous avons visité un musée pendant le week-end."}
148
- {"id": 147, "concept": "french_language", "pair_id": "french_language-09", "variant": 1, "text": "Ce week-end, nous sommes allés voir un musée."}
149
- {"id": 148, "concept": "french_language", "pair_id": "french_language-10", "variant": 0, "text": "La réunion commence à neuf heures précises."}
150
- {"id": 149, "concept": "french_language", "pair_id": "french_language-10", "variant": 1, "text": "Le rendez-vous débute exactement à neuf heures."}
151
- {"id": 150, "concept": "french_language", "pair_id": "french_language-11", "variant": 0, "text": "Mon ordinateur ne fonctionne plus correctement."}
152
- {"id": 151, "concept": "french_language", "pair_id": "french_language-11", "variant": 1, "text": "J’ai un problème : mon ordinateur marche mal maintenant."}
153
- {"id": 152, "concept": "french_language", "pair_id": "french_language-12", "variant": 0, "text": "Cette ville possède de nombreux bâtiments historiques."}
154
- {"id": 153, "concept": "french_language", "pair_id": "french_language-12", "variant": 1, "text": "On trouve beaucoup d’édifices historiques dans cette ville."}
155
- {"id": 154, "concept": "french_language", "pair_id": "french_language-13", "variant": 0, "text": "Il faut acheter du pain et des légumes."}
156
- {"id": 155, "concept": "french_language", "pair_id": "french_language-13", "variant": 1, "text": "Nous devons prendre du pain ainsi que des légumes."}
157
- {"id": 156, "concept": "french_language", "pair_id": "french_language-14", "variant": 0, "text": "Je cherche un appartement près de l’université."}
158
- {"id": 157, "concept": "french_language", "pair_id": "french_language-14", "variant": 1, "text": "Je voudrais trouver un logement proche de l’université."}
159
- {"id": 158, "concept": "french_language", "pair_id": "french_language-15", "variant": 0, "text": "Merci beaucoup pour votre aide."}
160
- {"id": 159, "concept": "french_language", "pair_id": "french_language-15", "variant": 1, "text": "Je vous remercie sincèrement de votre aide."}
161
  {"id": 160, "concept": "factual_entities", "pair_id": "factual_entities-00", "variant": 0, "text": "Tell me about Marie Curie and her scientific work."}
162
  {"id": 161, "concept": "factual_entities", "pair_id": "factual_entities-00", "variant": 1, "text": "Summarize the scientific contributions of Marie Curie."}
163
  {"id": 162, "concept": "factual_entities", "pair_id": "factual_entities-01", "variant": 0, "text": "What is notable about the city of Kyoto?"}
 
126
  {"id": 125, "concept": "negative_sentiment", "pair_id": "negative_sentiment-14", "variant": 1, "text": "I thought the exhibition was dull and unrewarding."}
127
  {"id": 126, "concept": "negative_sentiment", "pair_id": "negative_sentiment-15", "variant": 0, "text": "The train journey was stressful and uncomfortable."}
128
  {"id": 127, "concept": "negative_sentiment", "pair_id": "negative_sentiment-15", "variant": 1, "text": "The trip by train felt frustrating, noisy, and unpleasant."}
129
+ {"id": 128, "concept": "german_language", "pair_id": "german_language-00", "variant": 0, "text": "Guten Tag, wie geht es Ihnen heute?"}
130
+ {"id": 129, "concept": "german_language", "pair_id": "german_language-00", "variant": 1, "text": "Hallo, wie geht es dir heute?"}
131
+ {"id": 130, "concept": "german_language", "pair_id": "german_language-01", "variant": 0, "text": "Ich möchte einen Tisch für zwei Personen reservieren."}
132
+ {"id": 131, "concept": "german_language", "pair_id": "german_language-01", "variant": 1, "text": "Kann ich bitte einen Tisch für zwei reservieren?"}
133
+ {"id": 132, "concept": "german_language", "pair_id": "german_language-02", "variant": 0, "text": "Die Bibliothek schließt um achtzehn Uhr."}
134
+ {"id": 133, "concept": "german_language", "pair_id": "german_language-02", "variant": 1, "text": "Die Bibliothek ist bis achtzehn Uhr geöffnet."}
135
+ {"id": 134, "concept": "german_language", "pair_id": "german_language-03", "variant": 0, "text": "Dieses Buch erzählt eine sehr interessante Geschichte."}
136
+ {"id": 135, "concept": "german_language", "pair_id": "german_language-03", "variant": 1, "text": "Die Geschichte in diesem Buch ist wirklich interessant."}
137
+ {"id": 136, "concept": "german_language", "pair_id": "german_language-04", "variant": 0, "text": "Wir fahren morgen früh mit dem Zug."}
138
+ {"id": 137, "concept": "german_language", "pair_id": "german_language-04", "variant": 1, "text": "Morgen früh werden wir mit dem Zug reisen."}
139
+ {"id": 138, "concept": "german_language", "pair_id": "german_language-05", "variant": 0, "text": "Können Sie mir den Weg zum Bahnhof zeigen?"}
140
+ {"id": 139, "concept": "german_language", "pair_id": "german_language-05", "variant": 1, "text": "Wie komme ich zum Bahnhof?"}
141
+ {"id": 140, "concept": "german_language", "pair_id": "german_language-06", "variant": 0, "text": "Ich lerne gern neue Sprachen."}
142
+ {"id": 141, "concept": "german_language", "pair_id": "german_language-06", "variant": 1, "text": "Neue Sprachen zu lernen macht mir viel Spaß."}
143
+ {"id": 142, "concept": "german_language", "pair_id": "german_language-07", "variant": 0, "text": "Das Wetter am Meer ist wunderschön."}
144
+ {"id": 143, "concept": "german_language", "pair_id": "german_language-07", "variant": 1, "text": "Am Meer ist das Wetter heute sehr schön."}
145
+ {"id": 144, "concept": "german_language", "pair_id": "german_language-08", "variant": 0, "text": "Sie bereitet das Abendessen in der Küche zu."}
146
+ {"id": 145, "concept": "german_language", "pair_id": "german_language-08", "variant": 1, "text": "In der Küche kocht sie gerade das Abendessen."}
147
+ {"id": 146, "concept": "german_language", "pair_id": "german_language-09", "variant": 0, "text": "Wir haben am Wochenende ein Museum besucht."}
148
+ {"id": 147, "concept": "german_language", "pair_id": "german_language-09", "variant": 1, "text": "Am Wochenende waren wir in einem Museum."}
149
+ {"id": 148, "concept": "german_language", "pair_id": "german_language-10", "variant": 0, "text": "Die Besprechung beginnt genau um neun Uhr."}
150
+ {"id": 149, "concept": "german_language", "pair_id": "german_language-10", "variant": 1, "text": "Das Treffen fängt pünktlich um neun Uhr an."}
151
+ {"id": 150, "concept": "german_language", "pair_id": "german_language-11", "variant": 0, "text": "Mein Computer funktioniert nicht mehr richtig."}
152
+ {"id": 151, "concept": "german_language", "pair_id": "german_language-11", "variant": 1, "text": "Mit meinem Computer stimmt etwas nicht; er läuft nicht richtig."}
153
+ {"id": 152, "concept": "german_language", "pair_id": "german_language-12", "variant": 0, "text": "Diese Stadt hat viele historische Gebäude."}
154
+ {"id": 153, "concept": "german_language", "pair_id": "german_language-12", "variant": 1, "text": "In dieser Stadt gibt es zahlreiche historische Bauwerke."}
155
+ {"id": 154, "concept": "german_language", "pair_id": "german_language-13", "variant": 0, "text": "Wir müssen Brot und Gemüse kaufen."}
156
+ {"id": 155, "concept": "german_language", "pair_id": "german_language-13", "variant": 1, "text": "Wir sollten noch Brot sowie Gemüse einkaufen."}
157
+ {"id": 156, "concept": "german_language", "pair_id": "german_language-14", "variant": 0, "text": "Ich suche eine Wohnung in der Nähe der Universität."}
158
+ {"id": 157, "concept": "german_language", "pair_id": "german_language-14", "variant": 1, "text": "Ich möchte eine Unterkunft nahe der Universität finden."}
159
+ {"id": 158, "concept": "german_language", "pair_id": "german_language-15", "variant": 0, "text": "Vielen Dank für Ihre Hilfe."}
160
+ {"id": 159, "concept": "german_language", "pair_id": "german_language-15", "variant": 1, "text": "Ich danke Ihnen herzlich für Ihre Unterstützung."}
161
  {"id": 160, "concept": "factual_entities", "pair_id": "factual_entities-00", "variant": 0, "text": "Tell me about Marie Curie and her scientific work."}
162
  {"id": 161, "concept": "factual_entities", "pair_id": "factual_entities-00", "variant": 1, "text": "Summarize the scientific contributions of Marie Curie."}
163
  {"id": 162, "concept": "factual_entities", "pair_id": "factual_entities-01", "variant": 0, "text": "What is notable about the city of Kyoto?"}
docs/HF_DEPLOY.md CHANGED
@@ -15,11 +15,11 @@ The live app does not need every SAE layer from the full repository.
15
 
16
  ## GPU-decorated actions
17
 
18
- Current v0.6 live actions include:
19
 
20
  - concept-guided candidate feature discovery and candidate reuse;
21
  - completion-cue sensitivity scans;
22
- - plot fullscreen/export controls and a persistent Workbench context banner;
23
  - **Inspect sparse features**;
24
  - **Run single-feature causal test**;
25
  - **Run scale dose-response**;
@@ -73,7 +73,7 @@ The app uses:
73
  - restrained serif typography with normalized control/table sizes;
74
  - consistent muted-teal action and copy buttons;
75
  - bounded result-table heights;
76
- - explicit bottom padding and a visible `End of workbench.` footer;
77
  - a browser-side ResizeObserver/MutationObserver that requests a resize reflow after dynamic result-height changes.
78
 
79
  These changes reduce wasted horizontal space and mitigate the embedded-Space case where the outer page stopped extending after a large dynamic result. Hugging Face still owns the outer embedding frame, so compare with the direct `*.hf.space` URL if the parent page ever behaves differently.
 
15
 
16
  ## GPU-decorated actions
17
 
18
+ Current live actions include:
19
 
20
  - concept-guided candidate feature discovery and candidate reuse;
21
  - completion-cue sensitivity scans;
22
+ - bounded table/plot focus controls, descriptive PNG exports, and a persistent Workbench context banner;
23
  - **Inspect sparse features**;
24
  - **Run single-feature causal test**;
25
  - **Run scale dose-response**;
 
73
  - restrained serif typography with normalized control/table sizes;
74
  - consistent muted-teal action and copy buttons;
75
  - bounded result-table heights;
76
+ - explicit bottom padding with no visible footer clutter;
77
  - a browser-side ResizeObserver/MutationObserver that requests a resize reflow after dynamic result-height changes.
78
 
79
  These changes reduce wasted horizontal space and mitigate the embedded-Space case where the outer page stopped extending after a large dynamic result. Hugging Face still owns the outer embedding frame, so compare with the direct `*.hf.space` URL if the parent page ever behaves differently.
docs/METHODOLOGY.md CHANGED
@@ -29,7 +29,7 @@ The controlled discovery benchmark contains seven concept groups:
29
  - mathematics;
30
  - positive sentiment;
31
  - negative sentiment;
32
- - French language;
33
  - factual entities;
34
  - uncertainty.
35
 
@@ -348,9 +348,9 @@ A large point-estimate ratio alone is not sufficient for a strong causal-specifi
348
  FeatureLens keeps the raw rows even when the resulting conclusion is null, mixed, or contrary to the original hypothesis.
349
 
350
 
351
- ## v0.6 live candidate discovery
352
 
353
- The selected-feature concept contrast asks **given a feature, where does it activate?** v0.6 adds the reverse live question: **given a controlled concept, which SAE features are plausible candidates to investigate?**
354
 
355
  For a balanced batch with `n` prompts from each of the seven controlled groups, FeatureLens encodes every non-padding token and forms a prompt-wide feature profile by taking the maximum TopK activation of each SAE feature over the prompt. For target concept `c` and feature `f`, it reports:
356
 
@@ -361,9 +361,9 @@ mean_difference(f) = target_mean(f) - other_mean(f)
361
  selectivity(f) = mean_difference(f) / (target_mean(f) + other_mean(f) + eps)
362
  ```
363
 
364
- Candidates are ranked by positive `mean_difference`. The table also reports target/other activation rates and target maximum activation. This is intentionally **not a held-out labeler**: the same small batch is used for live screening. Semantic claims still require the offline grouped train/test feature evaluation.
365
 
366
- ## v0.6 completion-cue sensitivity
367
 
368
  The v0.5 math example showed feature `22632` active only at the final `is` token. That pattern motivates a lexical/structural control. For one prompt stem and a user-supplied list of completion cues, FeatureLens appends each cue, encodes the resulting prompt, and measures the selected feature at the final non-padding token.
369
 
@@ -375,3 +375,19 @@ The cue scan is intended to distinguish hypotheses such as:
375
  - broad activation across several continuation cues.
376
 
377
  It is a controlled diagnostic only. A cue response cannot establish the feature's complete semantics.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
  - mathematics;
30
  - positive sentiment;
31
  - negative sentiment;
32
+ - German language;
33
  - factual entities;
34
  - uncertainty.
35
 
 
348
  FeatureLens keeps the raw rows even when the resulting conclusion is null, mixed, or contrary to the original hypothesis.
349
 
350
 
351
+ ## Live candidate discovery
352
 
353
+ The selected-feature concept contrast asks **given a feature, where does it activate?** FeatureLens also asks the reverse live question: **given a controlled concept, which SAE features are plausible candidates to investigate?**
354
 
355
  For a balanced batch with `n` prompts from each of the seven controlled groups, FeatureLens encodes every non-padding token and forms a prompt-wide feature profile by taking the maximum TopK activation of each SAE feature over the prompt. For target concept `c` and feature `f`, it reports:
356
 
 
361
  selectivity(f) = mean_difference(f) / (target_mean(f) + other_mean(f) + eps)
362
  ```
363
 
364
+ The default live ranking uses the exploratory score `max(0, selectivity) × target_activation_rate × log1p(target_mean)`, which prevents very large but broadly active SAE coefficients from dominating merely because of scale. A raw positive `mean_difference` ordering remains available for comparison. The table also reports target/other activation rates plus current-Workbench prompt-wide and selected-token activation. This is intentionally **not a held-out labeler**: the same small controlled batch is used for live screening. Semantic claims still require the offline grouped train/test feature evaluation.
365
 
366
+ ## Completion-cue sensitivity
367
 
368
  The v0.5 math example showed feature `22632` active only at the final `is` token. That pattern motivates a lexical/structural control. For one prompt stem and a user-supplied list of completion cues, FeatureLens appends each cue, encodes the resulting prompt, and measures the selected feature at the final non-padding token.
369
 
 
375
  - broad activation across several continuation cues.
376
 
377
  It is a controlled diagnostic only. A cue response cannot establish the feature's complete semantics.
378
+
379
+ ## Cue × context specificity
380
+
381
+ A single completion-cue scan can show that a feature responds to a token such as `is`, but cannot distinguish a lexical cue feature from a context-dependent completion-boundary feature. v0.7 therefore crosses several prompt stems with the same cue set in one batched forward pass.
382
+
383
+ For feature $f$, stem $s$, and cue $c$, the diagnostic records the final-token SAE activation $z_f(s+c)$. A cue that activates across unrelated stems is more consistent with lexical/cue specificity; activation restricted to a subset of semantically related stems is more consistent with context-sensitive completion structure. This remains a diagnostic rather than a semantic label.
384
+
385
+ ## Live candidate ranking and causal readiness
386
+
387
+ Large SAE coefficients can dominate a raw mean-difference ranking even when they are common across many concepts. The default candidate score is therefore
388
+
389
+ $$
390
+ S_f = \max(0,\mathrm{selectivity}_f) \cdot \mathrm{target\ rate}_f \cdot \log(1 + \mathrm{target\ mean}_f).
391
+ $$
392
+
393
+ This ranking is still exploratory. It is designed to triage candidates, not replace held-out AUROC/F1. The same forward pass optionally includes the current Workbench prompt, allowing the candidate table to report current-prompt maximum activation and selected-token activation. A candidate can therefore be concept-associated in the live batch but visibly inactive at the current causal location.
docs/VALIDATION.md CHANGED
@@ -1,8 +1,6 @@
1
- # FeatureLens v0.6 validation
2
 
3
- This guide tests the **new v0.6 behavior** plus a few high-value regressions. The comprehensive edge/adversarial suite remains deferred until the final hardening release, as agreed.
4
-
5
- Use the exact UI labels below.
6
 
7
  ## A. Local release gates
8
 
@@ -18,7 +16,7 @@ python3 scripts/release_check.py
18
  Expected:
19
 
20
  ```text
21
- 40 passed
22
  FeatureLens UI launch smoke: PASS
23
  FeatureLens release check: PASS
24
  discovery prompts: 224
@@ -26,21 +24,32 @@ FeatureLens release check: PASS
26
  layers: [4, 14, 26]
27
  feature-set sizes: [1, 3, 5]
28
  random controls: 8
29
- release: v0.6.0
30
  ```
31
 
32
- ## B. Navigation / typography / plot controls
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
- 1. Open **Start here**.
35
- 2. Confirm the three workflow cards and the plain-language glossary are readable without tiny heading text.
36
- 3. Open **Workbench**.
37
- 4. Confirm the hierarchy is visually descending: section title > subsection heading > explanatory body text.
38
- 5. Confirm result-table column headers are not noticeably smaller than normal body text.
39
- 6. After any plot is produced, use its top-right **fullscreen** button. The plot should open large enough for labels to be read comfortably.
40
- 7. Confirm the adjacent **export** control produces a PNG.
41
- 8. Confirm copy buttons still show `✓ Copied with headers` briefly.
42
 
43
- Pass condition: no heading level appears smaller than the body text it introduces, tables remain readable, and every native plot has fullscreen/export controls.
 
44
 
45
  ## C. Establish the shared Workbench context
46
 
@@ -65,108 +74,141 @@ NMSE: 0.1874
65
  Top-5 activation mass: 26.7%
66
  ```
67
 
68
- The persistent **Current Workbench context** banner above the tabs must now state the inspected prompt, layer 14, and token 5. Switch to **Feature sets** and **Feature evidence**; the banner should remain visible.
69
-
70
- ## D. Explicit experiment feature selectors
71
 
72
- ### D1. Scale dose-response
73
 
74
- Open **WorkbenchIII. Single-feature scale dose-response Scale dose-response**.
75
 
76
- Confirm there is a visible field named:
77
-
78
- ```text
79
- Dose-response feature id
80
- ```
81
 
82
- It should be populated from the latest inspection but editable independently. Set it to `22632` and run with target continuation `2x` from the single-feature causal panel.
 
83
 
84
- The `1.0` multiplier row must remain the exact numerical null:
85
 
86
  ```text
87
- Δ feature coefficient = 0
88
- Perturbation L2 = 0
89
- Δ mean log p/token = 0
90
- Δ sequence log p = 0
91
- Next-token JS = 0
92
  ```
93
 
94
- ### D2. Contrastive continuation preference
95
 
96
- Open **Workbench → IV. Contrastive continuation preference → Contrastive causal preference test**.
97
 
98
- Confirm there is now a visible field named:
 
 
99
 
100
  ```text
101
- Contrastive feature id
 
 
 
 
102
  ```
103
 
104
- Set:
 
 
105
 
106
  ```text
107
- Contrastive feature id: 22632
108
- Continuation A (preferred): 2x
109
- Continuation B (comparison): x
110
- Contrastive intervention: ablate
 
 
 
 
 
 
 
 
111
  ```
112
 
113
- Click **Run contrastive preference test**.
114
 
115
- The v0.5 reference result was a causal sequence-log-odds shift of about `-0.0882`; small numerical variation is acceptable. The important regression is that the selected feature is explicit and the panel no longer silently inherits a hidden feature id.
 
 
 
116
 
117
- ## E. Concept-guided candidate feature discovery
118
 
119
- Open **Feature evidence → A. Concept-guided candidate discovery**.
120
 
121
- Set:
122
 
123
  ```text
124
- Target concept: mathematics
125
- Residual layer: 14
126
- Prompts per concept: 4
127
- Candidate features: 12
128
  ```
129
 
130
- Click **Discover concept-associated candidates**.
 
 
 
 
 
 
 
 
131
 
132
- Verify:
 
 
 
 
 
 
 
133
 
134
- - the result is a table named **Candidate feature contrast**;
135
- - rows contain `Feature id`, target/other prompt-wide means, mean difference, selectivity score, and activation rates;
136
- - candidates are ordered by positive `Target − other mean` evidence;
137
- - the UI calls them **candidates**, not semantic labels;
138
- - **Discovered candidate feature id** is populated when candidates exist;
139
- - the plot has fullscreen/export controls.
140
 
141
- An empty candidate list is not itself a software failure if the UI explicitly reports that no positively selective candidate was found. It must not invent a label.
142
 
143
- ## F. Reuse a discovered candidate
144
 
145
- After Test E, choose any value in **Discovered candidate feature id** and click:
146
 
147
  ```text
148
- Use candidate in feature experiments
 
 
 
 
 
 
 
 
149
  ```
150
 
151
- Verify that the same id appears in:
 
 
152
 
153
- - **Workbench → Single feature id**;
154
- - **Workbench → Dose-response feature id**;
155
- - **Workbench Contrastive feature id**;
156
- - **Feature evidence → Feature id**.
157
 
158
- This is a UI/state-routing test only; it should not invoke the GPU.
159
 
160
- ## G. Completion-cue sensitivity
161
 
162
- Open **Feature evidence → C. Completion-cue sensitivity**.
163
 
164
  Use:
165
 
166
  ```text
167
  Feature id: 22632
168
  Residual layer: 14
169
- Prompt stem: The derivative of x squared
 
 
 
 
 
 
170
  Completion cues:
171
  is
172
  =
@@ -175,40 +217,56 @@ equals
175
  therefore
176
  ```
177
 
178
- Click **Run completion-cue scan**.
179
 
180
- The `is` row reconstructs the original prompt `The derivative of x squared is`, so its final-token activation should be approximately `32.90625` and `Active in TopK = True`.
181
 
182
- Record the remaining cue activations. Their scientific purpose is to test whether feature 22632 is strongly tied to a completion cue such as `is`, rather than to mathematics generally. Do **not** require the other cues to be zero; that is an empirical result.
 
 
183
 
184
- ## H. Prompt-wide selected-feature contrast regression
185
 
186
- Still in **Feature evidence**, use feature `22632`, layer `14`, 4 prompts/concept and click **Run controlled concept contrast**.
187
 
188
- Reference v0.5 behavior:
 
 
189
 
190
  ```text
191
- active in 7/28 prompts
192
- uncertainty mean ≈ 7.55
193
- mathematics mean 5.42
194
- french_language mean = 0
195
  ```
196
 
197
- Small numerical variation is acceptable. This verifies the existing prompt-wide selected-feature scan remains unchanged while the new candidate-discovery scan answers the reverse question: *given a concept, which features should I investigate?*
 
 
 
 
198
 
199
- ## I. Quick existing regressions
200
 
201
- You do not need to resend full tables unless something changes unexpectedly.
202
 
203
- 1. **Paraphrase robustness**: default pair should remain roughly selected-token cosine `0.301`, prompt-wide cosine `0.981`.
204
- 2. **Identical-prompt control**: all four robustness metrics should remain `1.000`.
205
- 3. **Layer trajectory**: layer-14 reconstruction should remain the weakest of layers 4/14/26 for the default prompt.
206
- 4. **Feature sets → Set-size sensitivity → Run 1/3/5-feature ablation sweep**: this is the “top-k sweep.” It is **not** the basic Workbench inspection. Only confirm the experiment still runs and produces k = 1, 3, 5 rows.
207
- 5. Scroll all the way to **End of workbench** after dynamic outputs expand; no tab switch should be necessary.
 
 
 
 
 
 
 
 
 
208
 
209
- ## J. Deferred final hardening
210
 
211
- Do not spend ZeroGPU quota on the complete adversarial suite yet. Keep these for the final release:
212
 
213
  - empty prompts;
214
  - out-of-range token indices;
@@ -221,4 +279,4 @@ Do not spend ZeroGPU quota on the complete adversarial suite yet. Keep these for
221
  - long-prompt truncation;
222
  - rapid repeated clicks / queue behavior;
223
  - narrow/mobile-width layout;
224
- - final table-copy and plot fullscreen/export sweep.
 
1
+ # FeatureLens v0.7 validation
2
 
3
+ This guide tests the **v0.7 changes** plus a small number of high-value regressions. Use the exact UI labels below. The comprehensive edge/adversarial suite remains deferred until final hardening.
 
 
4
 
5
  ## A. Local release gates
6
 
 
16
  Expected:
17
 
18
  ```text
19
+ 45 passed
20
  FeatureLens UI launch smoke: PASS
21
  FeatureLens release check: PASS
22
  discovery prompts: 224
 
24
  layers: [4, 14, 26]
25
  feature-set sizes: [1, 3, 5]
26
  random controls: 8
27
+ release: v0.7.0
28
  ```
29
 
30
+ ## B. UI cleanup, table typography, focus mode, and export names
31
+
32
+ 1. Open **Workbench**.
33
+ 2. Confirm these are normal always-visible sections, **not collapsible dropdowns/accordions**:
34
+ - **III. Single-feature scale dose-response**
35
+ - **IV. Contrastive continuation preference**
36
+ 3. Confirm there is only one experiment heading before each section. A plot title such as `Scale dose-response` is fine; there should not be a second accordion heading with the same text.
37
+ 4. After any result table appears, confirm its table title (for example **Strongest active SAE features**) is visually larger/bolder than ordinary body text, and column headings are clearly readable.
38
+ 5. Produce a plot and click its top-right **fullscreen/focus** icon.
39
+ - The focused plot should open near the top-center of the viewport.
40
+ - It should be bounded rather than stretching across the entire monitor.
41
+ - Axis labels should remain readable without needing to pan horizontally across a huge plot.
42
+ 6. Exit focus mode. You should return to approximately the same page location you were at before opening it.
43
+ 7. Repeat the focus test on one wide result table.
44
+ 8. Click the plot **export** icon. Verify the downloaded filename is descriptive, for example:
45
 
46
+ ```text
47
+ featurelens_activation-profile.png
48
+ featurelens_scale-dose-response.png
49
+ ```
 
 
 
 
50
 
51
+ It should not be the generic `chart.png`.
52
+ 9. Click **Copy table with headers** and confirm the temporary `✓ Copied with headers` feedback still appears.
53
 
54
  ## C. Establish the shared Workbench context
55
 
 
74
  Top-5 activation mass: 26.7%
75
  ```
76
 
77
+ The persistent **Current Workbench context** banner must state the inspected prompt, layer 14, and token 5. It should remain visible when switching tabs.
 
 
78
 
79
+ ## D. German-language control regression
80
 
81
+ Open **Feature evidence A. Concept-guided candidate discovery** and inspect **Target concept**.
82
 
83
+ Pass conditions:
 
 
 
 
84
 
85
+ - `german_language` is available;
86
+ - `french_language` is not available.
87
 
88
+ The controlled examples in **Workbench** should also include a German example such as:
89
 
90
  ```text
91
+ Ich möchte einen Tisch für zwei reservieren.
 
 
 
 
92
  ```
93
 
94
+ No GPU run is required for this check.
95
 
96
+ ## E. Balanced concept-guided candidate discovery
97
 
98
+ Open **Feature evidence A. Concept-guided candidate discovery**.
99
+
100
+ Set:
101
 
102
  ```text
103
+ Target concept: mathematics
104
+ Residual layer: 14
105
+ Prompts per concept: 4
106
+ Candidate features: 12
107
+ Candidate ranking: Balanced selectivity
108
  ```
109
 
110
+ Keep the Workbench context from Test C and click **Discover concept-associated candidates**.
111
+
112
+ Verify that **Candidate feature evidence** includes these fields:
113
 
114
  ```text
115
+ Rank
116
+ Feature id
117
+ Candidate score
118
+ Target mean max
119
+ Other mean max
120
+ Mean difference
121
+ Selectivity
122
+ Target activation rate
123
+ Other activation rate
124
+ Current prompt max
125
+ Current token activation
126
+ Active at current token
127
  ```
128
 
129
+ Interpretation check:
130
 
131
+ - `Candidate score` is an exploratory balanced score, not a semantic label.
132
+ - Globally huge but weakly selective features should no longer automatically dominate merely because their raw coefficients are enormous.
133
+ - `Current prompt max` and `Current token activation` show whether a candidate can immediately be tested in the current Workbench context.
134
+ - **Selected candidate feature id** should default to the highest-ranked displayed candidate active at the current Workbench token when such a candidate exists; otherwise it may default to the top-ranked candidate.
135
 
136
+ Record the top 12 rows. These are the most scientifically useful new v0.7 results.
137
 
138
+ ### E2. Raw-ranking comparison
139
 
140
+ Without changing the other fields, set:
141
 
142
  ```text
143
+ Candidate ranking: Raw mean difference
 
 
 
144
  ```
145
 
146
+ and rerun **Discover concept-associated candidates**.
147
+
148
+ This should reproduce the scale-sensitive style of ranking used previously. The purpose of this test is to verify that v0.7 exposes the methodological choice explicitly instead of silently changing it.
149
+
150
+ You do **not** need to investigate all 12 raw-ranking features causally.
151
+
152
+ ## F. Candidate row selection and explicit reuse
153
+
154
+ After Test E:
155
 
156
+ 1. Click a row in **Candidate feature evidence**. The row's feature id should appear in **Selected candidate feature id**.
157
+ 2. Click **Use selected candidate across feature tests**.
158
+ 3. A visible confirmation should say that the feature was loaded.
159
+ 4. Verify the same id appears in:
160
+ - **Workbench → Single feature id**;
161
+ - **Workbench → Dose-response feature id**;
162
+ - **Workbench → Contrastive feature id**;
163
+ - **Feature evidence → Feature id**.
164
 
165
+ This is a state-routing operation and should not consume another GPU call.
 
 
 
 
 
166
 
167
+ ## G. Completion-cue regression for feature 22632
168
 
169
+ Open **Feature evidence C. Completion-cue sensitivity**.
170
 
171
+ Set:
172
 
173
  ```text
174
+ Feature id: 22632
175
+ Residual layer: 14
176
+ Prompt stem: The derivative of x squared
177
+ Completion cues:
178
+ is
179
+ =
180
+ :
181
+ equals
182
+ therefore
183
  ```
184
 
185
+ Click **Run completion-cue scan**.
186
+
187
+ The `is` condition reconstructs the original prompt and should remain approximately:
188
 
189
+ ```text
190
+ activation: 32.90625
191
+ Active in TopK: True
192
+ ```
193
 
194
+ The previous reference result had zero activation for `=`, `:`, `equals`, and `therefore`. Treat that as a regression reference, not as a hard scientific requirement if tiny numerical differences appear.
195
 
196
+ ## H. Cue × context specificity
197
 
198
+ Open **Feature evidence → D. Cue × context specificity**.
199
 
200
  Use:
201
 
202
  ```text
203
  Feature id: 22632
204
  Residual layer: 14
205
+
206
+ Prompt stems:
207
+ The derivative of x squared
208
+ The capital of Germany
209
+ The weather today
210
+ My name
211
+
212
  Completion cues:
213
  is
214
  =
 
217
  therefore
218
  ```
219
 
220
+ Click **Run cue × context scan**.
221
 
222
+ Expected output:
223
 
224
+ - 20 rows = 4 stems × 5 cues;
225
+ - columns **Prompt stem**, **Cue**, **Full prompt**, **Final token**, **Activation**, **Active in TopK**;
226
+ - a **Cue response across contexts** plot.
227
 
228
+ Scientific question: does feature `22632` fire specifically on the literal `is`, on `is` across many contexts, on several completion-boundary cues, or only in the original mathematics context?
229
 
230
+ Send the active rows and the per-cue context summary. This is the other key new v0.7 result.
231
 
232
+ ## I. Prompt-wide selected-feature contrast with German control
233
+
234
+ Still in **Feature evidence**, set:
235
 
236
  ```text
237
+ Feature id: 22632
238
+ Residual layer: 14
239
+ Prompts per concept: 4
 
240
  ```
241
 
242
+ Click **Run controlled concept contrast**.
243
+
244
+ The result should contain `german_language` rather than `french_language` while preserving seven balanced concepts and 28 sampled prompts total.
245
+
246
+ Do not expect the exact old French-based concept means to remain identical because one control group has intentionally changed.
247
 
248
+ ## J. High-value existing regressions
249
 
250
+ Only confirm these run; do not resend full tables unless something changes unexpectedly.
251
 
252
+ 1. **Workbench III. Single-feature scale dose-response** with feature `22632`, target `2x`:
253
+ - multiplier `1.0` must remain the exact same-batch numerical null with all causal deltas equal to zero.
254
+ 2. **Paraphrase robustness** default pair:
255
+ - selected-token cosine should remain around `0.301`;
256
+ - prompt-wide cosine should remain around `0.981`.
257
+ 3. **Paraphrase robustness** identical-prompt control:
258
+ - all four robustness metrics should remain `1.000`.
259
+ 4. **Layer trajectory**:
260
+ - layer 14 should remain the weakest reconstruction of layers 4/14/26 on the default prompt.
261
+ 5. **Feature sets → Set-size sensitivity → Run 1/3/5-feature ablation sweep**:
262
+ - this is the top-k causal sweep;
263
+ - it is **not** the Workbench `Strongest active SAE features` table;
264
+ - confirm k = 1, 3, 5 rows appear.
265
+ 6. After dynamic results expand, scroll to the actual bottom without switching tabs. The v0.5 reflow fix should remain intact.
266
 
267
+ ## K. Deferred final hardening
268
 
269
+ Do not spend ZeroGPU quota on the complete adversarial suite yet. Preserve these for the final hardening release:
270
 
271
  - empty prompts;
272
  - out-of-range token indices;
 
279
  - long-prompt truncation;
280
  - rapid repeated clicks / queue behavior;
281
  - narrow/mobile-width layout;
282
+ - final copy/focus/export sweep.
experiments/build_dataset.py CHANGED
@@ -79,23 +79,23 @@ PAIRS: dict[str, list[tuple[str, str]]] = {
79
  ('The museum exhibition was painfully boring.', 'I thought the exhibition was dull and unrewarding.'),
80
  ('The train journey was stressful and uncomfortable.', 'The trip by train felt frustrating, noisy, and unpleasant.'),
81
  ],
82
- 'french_language': [
83
- ('Bonjour, comment allez-vous aujourd’hui ?', 'Salut, comment vas-tu aujourd’hui ?'),
84
- ('Je voudrais réserver une table pour deux personnes.', 'Puis-je réserver une table pour deux, s’il vous plaît ?'),
85
- ('La bibliothèque ferme à dix-huit heures.', 'La bibliothèque est ouverte jusqu’à dix-huit heures.'),
86
- ('Ce livre raconte une histoire très intéressante.', 'L’histoire racontée dans ce livre est vraiment intéressante.'),
87
- ('Nous allons prendre le train demain matin.', 'Demain matin, nous voyagerons en train.'),
88
- ('Pouvez-vous m’indiquer le chemin vers la gare ?', 'Comment puis-je aller jusqu’à la gare ?'),
89
- ('J’aime apprendre de nouvelles langues.', 'Apprendre des langues nouvelles me plaît beaucoup.'),
90
- ('Le temps est magnifique au bord de la mer.', 'Il fait très beau près de la mer.'),
91
- ('Elle prépare le dîner dans la cuisine.', 'Dans la cuisine, elle est en train de préparer le repas du soir.'),
92
- ('Nous avons visité un musée pendant le week-end.', 'Ce week-end, nous sommes allés voir un musée.'),
93
- ('La réunion commence à neuf heures précises.', 'Le rendez-vous débute exactement à neuf heures.'),
94
- ('Mon ordinateur ne fonctionne plus correctement.', 'J’ai un problème : mon ordinateur marche mal maintenant.'),
95
- ('Cette ville possède de nombreux bâtiments historiques.', 'On trouve beaucoup d’édifices historiques dans cette ville.'),
96
- ('Il faut acheter du pain et des légumes.', 'Nous devons prendre du pain ainsi que des légumes.'),
97
- ('Je cherche un appartement près de l’université.', 'Je voudrais trouver un logement proche de l’université.'),
98
- ('Merci beaucoup pour votre aide.', 'Je vous remercie sincèrement de votre aide.'),
99
  ],
100
  'factual_entities': [
101
  ('Tell me about Marie Curie and her scientific work.', 'Summarize the scientific contributions of Marie Curie.'),
@@ -152,10 +152,10 @@ CAUSAL_TASKS = [
152
  {'concept': 'code', 'prompt': 'In Python, an exception handler begins with the keyword', 'target': ' except'},
153
  {'concept': 'code', 'prompt': 'SQL keyword used to retrieve rows:', 'target': ' SELECT'},
154
  {'concept': 'code', 'prompt': 'A Python conditional branch commonly starts with', 'target': ' if'},
155
- {'concept': 'french_language', 'prompt': "Translate 'hello' into French:", 'target': ' bonjour'},
156
- {'concept': 'french_language', 'prompt': "Translate 'thank you' into French:", 'target': ' merci'},
157
- {'concept': 'french_language', 'prompt': "Translate 'yes' into French:", 'target': ' oui'},
158
- {'concept': 'french_language', 'prompt': "Translate 'good evening' into French:", 'target': ' bonsoir'},
159
  {'concept': 'factual_entities', 'prompt': 'The scientist associated with radium research, Marie', 'target': ' Curie'},
160
  {'concept': 'factual_entities', 'prompt': 'The composer of the Fifth Symphony, Ludwig van', 'target': ' Beethoven'},
161
  {'concept': 'factual_entities', 'prompt': 'The computer scientist known for the Turing machine, Alan', 'target': ' Turing'},
 
79
  ('The museum exhibition was painfully boring.', 'I thought the exhibition was dull and unrewarding.'),
80
  ('The train journey was stressful and uncomfortable.', 'The trip by train felt frustrating, noisy, and unpleasant.'),
81
  ],
82
+ 'german_language': [
83
+ ('Guten Tag, wie geht es Ihnen heute?', 'Hallo, wie geht es dir heute?'),
84
+ ('Ich möchte einen Tisch für zwei Personen reservieren.', 'Kann ich bitte einen Tisch für zwei reservieren?'),
85
+ ('Die Bibliothek schließt um achtzehn Uhr.', 'Die Bibliothek ist bis achtzehn Uhr geöffnet.'),
86
+ ('Dieses Buch erzählt eine sehr interessante Geschichte.', 'Die Geschichte in diesem Buch ist wirklich interessant.'),
87
+ ('Wir fahren morgen früh mit dem Zug.', 'Morgen früh werden wir mit dem Zug reisen.'),
88
+ ('Können Sie mir den Weg zum Bahnhof zeigen?', 'Wie komme ich zum Bahnhof?'),
89
+ ('Ich lerne gern neue Sprachen.', 'Neue Sprachen zu lernen macht mir viel Spaß.'),
90
+ ('Das Wetter am Meer ist wunderschön.', 'Am Meer ist das Wetter heute sehr schön.'),
91
+ ('Sie bereitet das Abendessen in der Küche zu.', 'In der Küche kocht sie gerade das Abendessen.'),
92
+ ('Wir haben am Wochenende ein Museum besucht.', 'Am Wochenende waren wir in einem Museum.'),
93
+ ('Die Besprechung beginnt genau um neun Uhr.', 'Das Treffen fängt pünktlich um neun Uhr an.'),
94
+ ('Mein Computer funktioniert nicht mehr richtig.', 'Mit meinem Computer stimmt etwas nicht; er läuft nicht richtig.'),
95
+ ('Diese Stadt hat viele historische Gebäude.', 'In dieser Stadt gibt es zahlreiche historische Bauwerke.'),
96
+ ('Wir müssen Brot und Gemüse kaufen.', 'Wir sollten noch Brot sowie Gemüse einkaufen.'),
97
+ ('Ich suche eine Wohnung in der Nähe der Universität.', 'Ich möchte eine Unterkunft nahe der Universität finden.'),
98
+ ('Vielen Dank für Ihre Hilfe.', 'Ich danke Ihnen herzlich für Ihre Unterstützung.'),
99
  ],
100
  'factual_entities': [
101
  ('Tell me about Marie Curie and her scientific work.', 'Summarize the scientific contributions of Marie Curie.'),
 
152
  {'concept': 'code', 'prompt': 'In Python, an exception handler begins with the keyword', 'target': ' except'},
153
  {'concept': 'code', 'prompt': 'SQL keyword used to retrieve rows:', 'target': ' SELECT'},
154
  {'concept': 'code', 'prompt': 'A Python conditional branch commonly starts with', 'target': ' if'},
155
+ {'concept': 'german_language', 'prompt': "Translate 'hello' into German:", 'target': ' hallo'},
156
+ {'concept': 'german_language', 'prompt': "Translate 'thank you' into German:", 'target': ' danke'},
157
+ {'concept': 'german_language', 'prompt': "Translate 'yes' into German:", 'target': ' ja'},
158
+ {'concept': 'german_language', 'prompt': "Translate 'good evening' into German:", 'target': ' guten Abend'},
159
  {'concept': 'factual_entities', 'prompt': 'The scientist associated with radium research, Marie', 'target': ' Curie'},
160
  {'concept': 'factual_entities', 'prompt': 'The composer of the Fifth Symphony, Ludwig van', 'target': ' Beethoven'},
161
  {'concept': 'factual_entities', 'prompt': 'The computer scientist known for the Turing machine, Alan', 'target': ' Turing'},
experiments/collect_activations.py CHANGED
@@ -40,6 +40,19 @@ def _build_sparse(encodings: list, n_rows: int, width: int) -> sp.csr_matrix:
40
  return sp.csr_matrix((values, (row_ids, col_ids)), shape=(n_rows, width), dtype=np.float32)
41
 
42
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
  @torch.inference_mode()
44
  def main() -> None:
45
  args = parse_args()
@@ -85,15 +98,12 @@ def main() -> None:
85
  captured: dict[int, torch.Tensor] = {}
86
  handles = []
87
 
88
- def make_hook(layer: int):
89
- def hook(_module, _inputs, output):
90
- hidden = output[0] if isinstance(output, tuple) else output
91
- captured[layer] = hidden.detach()
92
-
93
- return hook
94
-
95
  for layer in args.layers:
96
- handles.append(model.model.layers[layer].register_forward_hook(make_hook(layer)))
 
 
 
 
97
  model(**batch, use_cache=False)
98
  for handle in handles:
99
  handle.remove()
@@ -148,4 +158,4 @@ def main() -> None:
148
 
149
 
150
  if __name__ == '__main__':
151
- main()
 
40
  return sp.csr_matrix((values, (row_ids, col_ids)), shape=(n_rows, width), dtype=np.float32)
41
 
42
 
43
+ def _make_capture_hook(
44
+ captured: dict[int, torch.Tensor],
45
+ layer: int,
46
+ ):
47
+ """Bind the capture dictionary and layer before registering the hook."""
48
+
49
+ def hook(_module, _inputs, output):
50
+ hidden = output[0] if isinstance(output, tuple) else output
51
+ captured[layer] = hidden.detach()
52
+
53
+ return hook
54
+
55
+
56
  @torch.inference_mode()
57
  def main() -> None:
58
  args = parse_args()
 
98
  captured: dict[int, torch.Tensor] = {}
99
  handles = []
100
 
 
 
 
 
 
 
 
101
  for layer in args.layers:
102
+ handles.append(
103
+ model.model.layers[layer].register_forward_hook(
104
+ _make_capture_hook(captured, layer)
105
+ )
106
+ )
107
  model(**batch, use_cache=False)
108
  for handle in handles:
109
  handle.remove()
 
158
 
159
 
160
  if __name__ == '__main__':
161
+ main()
experiments/run_causal.py CHANGED
@@ -54,6 +54,37 @@ def replace_hidden(output, hidden):
54
  return (hidden, *output[1:]) if isinstance(output, tuple) else hidden
55
 
56
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  def append_target(inputs: dict[str, torch.Tensor], target_ids: list[int]) -> dict[str, torch.Tensor]:
58
  prompt_ids = inputs['input_ids']
59
  target = torch.tensor(target_ids, dtype=prompt_ids.dtype, device=prompt_ids.device).unsqueeze(0)
@@ -121,12 +152,9 @@ def main() -> None:
121
  target_ids = [int(x) for x in target_ids]
122
  full_inputs = append_target(prompt_inputs, target_ids)
123
  capture: dict = {}
124
-
125
- def capture_hook(_module, _inp, output):
126
- if 'hidden' not in capture:
127
- capture['hidden'] = hidden_from_output(output).detach()
128
-
129
- handle = model.model.layers[layer].register_forward_hook(capture_hook)
130
  single_baseline_out = model(**full_inputs, use_cache=False)
131
  handle.remove()
132
  single_baseline_logits = single_baseline_out.logits[0]
@@ -178,19 +206,9 @@ def main() -> None:
178
  deltas = torch.stack([zero, *[item[4] for item in condition_meta]], dim=0)
179
  repeated = {key: value.repeat(deltas.shape[0], 1) for key, value in full_inputs.items()}
180
  applied = {'done': False}
181
-
182
- def batch_edit_hook(_module, _inp, output):
183
- if applied['done']:
184
- return output
185
- hidden = hidden_from_output(output)
186
- modified = hidden.clone()
187
- modified[:, prompt_len - 1, :] = (
188
- modified[:, prompt_len - 1, :] + deltas.to(hidden.device, hidden.dtype)
189
- )
190
- applied['done'] = True
191
- return replace_hidden(output, modified)
192
-
193
- hook = model.model.layers[layer].register_forward_hook(batch_edit_hook)
194
  edited_out = model(**repeated, use_cache=False)
195
  hook.remove()
196
 
@@ -210,7 +228,7 @@ def main() -> None:
210
  baseline_rank = int((baseline_next > baseline_next[target_id]).sum().item()) + 1
211
  baseline_top1 = int(torch.argmax(baseline_next).item())
212
 
213
- for row_idx, (intervention_name, condition, control_id, spec, applied_delta, delta_activation) in enumerate(
214
  condition_meta,
215
  start=1,
216
  ):
 
54
  return (hidden, *output[1:]) if isinstance(output, tuple) else hidden
55
 
56
 
57
+ def _make_capture_hook(capture: dict):
58
+ """Bind a per-task capture dictionary before registering the hook."""
59
+
60
+ def capture_hook(_module, _inp, output):
61
+ if 'hidden' not in capture:
62
+ capture['hidden'] = hidden_from_output(output).detach()
63
+
64
+ return capture_hook
65
+
66
+
67
+ def _make_batch_edit_hook(
68
+ applied: dict[str, bool],
69
+ prompt_len: int,
70
+ deltas: torch.Tensor,
71
+ ):
72
+ """Bind per-task edit state so hooks cannot capture a later loop iteration."""
73
+
74
+ def batch_edit_hook(_module, _inp, output):
75
+ if applied['done']:
76
+ return output
77
+ hidden = hidden_from_output(output)
78
+ modified = hidden.clone()
79
+ modified[:, prompt_len - 1, :] = (
80
+ modified[:, prompt_len - 1, :] + deltas.to(hidden.device, hidden.dtype)
81
+ )
82
+ applied['done'] = True
83
+ return replace_hidden(output, modified)
84
+
85
+ return batch_edit_hook
86
+
87
+
88
  def append_target(inputs: dict[str, torch.Tensor], target_ids: list[int]) -> dict[str, torch.Tensor]:
89
  prompt_ids = inputs['input_ids']
90
  target = torch.tensor(target_ids, dtype=prompt_ids.dtype, device=prompt_ids.device).unsqueeze(0)
 
152
  target_ids = [int(x) for x in target_ids]
153
  full_inputs = append_target(prompt_inputs, target_ids)
154
  capture: dict = {}
155
+ handle = model.model.layers[layer].register_forward_hook(
156
+ _make_capture_hook(capture)
157
+ )
 
 
 
158
  single_baseline_out = model(**full_inputs, use_cache=False)
159
  handle.remove()
160
  single_baseline_logits = single_baseline_out.logits[0]
 
206
  deltas = torch.stack([zero, *[item[4] for item in condition_meta]], dim=0)
207
  repeated = {key: value.repeat(deltas.shape[0], 1) for key, value in full_inputs.items()}
208
  applied = {'done': False}
209
+ hook = model.model.layers[layer].register_forward_hook(
210
+ _make_batch_edit_hook(applied, prompt_len, deltas)
211
+ )
 
 
 
 
 
 
 
 
 
 
212
  edited_out = model(**repeated, use_cache=False)
213
  hook.remove()
214
 
 
228
  baseline_rank = int((baseline_next > baseline_next[target_id]).sum().item()) + 1
229
  baseline_top1 = int(torch.argmax(baseline_next).item())
230
 
231
+ for row_idx, (intervention_name, condition, control_id, _spec, applied_delta, delta_activation) in enumerate(
232
  condition_meta,
233
  start=1,
234
  ):
experiments/run_feature_sets.py CHANGED
@@ -9,7 +9,11 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
9
 
10
  from experiments.common import ARTIFACT_DIR, DATA_DIR, load_jsonl, set_seed
11
  from featurelens.config import SETTINGS
12
- from featurelens.interventions import InterventionSpec, joint_residual_delta, normalized_random_control
 
 
 
 
13
  from featurelens.metrics import js_divergence_from_logits, sequence_logprob_summary
14
  from featurelens.sae import SAEStore
15
  from featurelens.selection import load_feature_sets
@@ -34,6 +38,37 @@ def replace_hidden(output, hidden):
34
  return (hidden, *output[1:]) if isinstance(output, tuple) else hidden
35
 
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  def append_target(inputs: dict[str, torch.Tensor], target_ids: list[int]) -> dict[str, torch.Tensor]:
38
  prompt_ids = inputs['input_ids']
39
  target = torch.tensor(target_ids, dtype=prompt_ids.dtype, device=prompt_ids.device).unsqueeze(0)
@@ -103,12 +138,9 @@ def main() -> None:
103
  target_ids = [int(x) for x in target_ids]
104
  full_inputs = append_target(prompt_inputs, target_ids)
105
  capture: dict = {}
106
-
107
- def capture_hook(_module, _inp, output):
108
- if 'hidden' not in capture:
109
- capture['hidden'] = hidden_from_output(output).detach()
110
-
111
- handle = model.model.layers[layer].register_forward_hook(capture_hook)
112
  single_baseline_out = model(**full_inputs, use_cache=False)
113
  handle.remove()
114
  single_logits = single_baseline_out.logits[0]
@@ -146,19 +178,9 @@ def main() -> None:
146
  deltas = torch.stack([zero, *[item[4] for item in condition_meta]], dim=0)
147
  repeated = {key: value.repeat(deltas.shape[0], 1) for key, value in full_inputs.items()}
148
  applied = {'done': False}
149
-
150
- def edit_hook(_module, _inp, output):
151
- if applied['done']:
152
- return output
153
- hidden = hidden_from_output(output)
154
- modified = hidden.clone()
155
- modified[:, prompt_len - 1, :] = (
156
- modified[:, prompt_len - 1, :] + deltas.to(hidden.device, hidden.dtype)
157
- )
158
- applied['done'] = True
159
- return replace_hidden(output, modified)
160
-
161
- hook = model.model.layers[layer].register_forward_hook(edit_hook)
162
  edited_out = model(**repeated, use_cache=False)
163
  hook.remove()
164
 
@@ -223,4 +245,4 @@ def main() -> None:
223
 
224
 
225
  if __name__ == '__main__':
226
- main()
 
9
 
10
  from experiments.common import ARTIFACT_DIR, DATA_DIR, load_jsonl, set_seed
11
  from featurelens.config import SETTINGS
12
+ from featurelens.interventions import (
13
+ InterventionSpec,
14
+ joint_residual_delta,
15
+ normalized_random_control,
16
+ )
17
  from featurelens.metrics import js_divergence_from_logits, sequence_logprob_summary
18
  from featurelens.sae import SAEStore
19
  from featurelens.selection import load_feature_sets
 
38
  return (hidden, *output[1:]) if isinstance(output, tuple) else hidden
39
 
40
 
41
+ def _make_capture_hook(capture: dict):
42
+ """Bind a per-task capture dictionary before registering the hook."""
43
+
44
+ def capture_hook(_module, _inp, output):
45
+ if 'hidden' not in capture:
46
+ capture['hidden'] = hidden_from_output(output).detach()
47
+
48
+ return capture_hook
49
+
50
+
51
+ def _make_batch_edit_hook(
52
+ applied: dict[str, bool],
53
+ prompt_len: int,
54
+ deltas: torch.Tensor,
55
+ ):
56
+ """Bind per-task edit state so hooks cannot capture a later loop iteration."""
57
+
58
+ def edit_hook(_module, _inp, output):
59
+ if applied['done']:
60
+ return output
61
+ hidden = hidden_from_output(output)
62
+ modified = hidden.clone()
63
+ modified[:, prompt_len - 1, :] = (
64
+ modified[:, prompt_len - 1, :] + deltas.to(hidden.device, hidden.dtype)
65
+ )
66
+ applied['done'] = True
67
+ return replace_hidden(output, modified)
68
+
69
+ return edit_hook
70
+
71
+
72
  def append_target(inputs: dict[str, torch.Tensor], target_ids: list[int]) -> dict[str, torch.Tensor]:
73
  prompt_ids = inputs['input_ids']
74
  target = torch.tensor(target_ids, dtype=prompt_ids.dtype, device=prompt_ids.device).unsqueeze(0)
 
138
  target_ids = [int(x) for x in target_ids]
139
  full_inputs = append_target(prompt_inputs, target_ids)
140
  capture: dict = {}
141
+ handle = model.model.layers[layer].register_forward_hook(
142
+ _make_capture_hook(capture)
143
+ )
 
 
 
144
  single_baseline_out = model(**full_inputs, use_cache=False)
145
  handle.remove()
146
  single_logits = single_baseline_out.logits[0]
 
178
  deltas = torch.stack([zero, *[item[4] for item in condition_meta]], dim=0)
179
  repeated = {key: value.repeat(deltas.shape[0], 1) for key, value in full_inputs.items()}
180
  applied = {'done': False}
181
+ hook = model.model.layers[layer].register_forward_hook(
182
+ _make_batch_edit_hook(applied, prompt_len, deltas)
183
+ )
 
 
 
 
 
 
 
 
 
 
184
  edited_out = model(**repeated, use_cache=False)
185
  hook.remove()
186
 
 
245
 
246
 
247
  if __name__ == '__main__':
248
+ main()
featurelens/interventions.py CHANGED
@@ -1,7 +1,7 @@
1
  from __future__ import annotations
2
 
3
- from dataclasses import dataclass
4
  from collections.abc import Sequence
 
5
 
6
  import torch
7
 
 
1
  from __future__ import annotations
2
 
 
3
  from collections.abc import Sequence
4
+ from dataclasses import dataclass
5
 
6
  import torch
7
 
featurelens/runtime.py CHANGED
@@ -2,13 +2,13 @@ from __future__ import annotations
2
 
3
  import hashlib
4
  import html
 
5
  import math
6
  import os
7
- import json
8
- from pathlib import Path
9
  from collections.abc import Iterator, Sequence
10
  from contextlib import contextmanager
11
  from dataclasses import dataclass
 
12
 
13
  import torch
14
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -233,9 +233,13 @@ class ConceptFeatureDiscoveryResult:
233
  layer: int
234
  prompts_per_concept: int
235
  top_n: int
 
236
  rows: list[list[object]]
237
  chart_rows: list[list[object]]
238
  candidate_ids: list[int]
 
 
 
239
 
240
 
241
  @dataclass
@@ -249,6 +253,19 @@ class FeatureCueScanResult:
249
  cue_count: int
250
 
251
 
 
 
 
 
 
 
 
 
 
 
 
 
 
252
  @dataclass
253
  class ParaphraseResult:
254
  tokens_a: list[str]
@@ -474,7 +491,7 @@ class FeatureLensRuntime:
474
 
475
  @staticmethod
476
  def _control_seed(text: str, layer: int, key: str, mode: str, coefficient: float) -> int:
477
- payload = f'{text}\0{layer}\0{key}\0{mode}\0{coefficient:.8g}'.encode('utf-8')
478
  return int.from_bytes(hashlib.sha256(payload).digest()[:4], 'big', signed=False)
479
 
480
  @staticmethod
@@ -1778,8 +1795,18 @@ class FeatureLensRuntime:
1778
  layer: int,
1779
  prompts_per_concept: int | None = None,
1780
  top_n: int = 12,
 
 
 
1781
  ) -> ConceptFeatureDiscoveryResult:
1782
- """Find SAE features whose prompt-wide activation is higher for one controlled concept."""
 
 
 
 
 
 
 
1783
  self.ensure_ready(preload_saes=False)
1784
  assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
1785
  concept = str(concept).strip()
@@ -1791,12 +1818,20 @@ class FeatureLensRuntime:
1791
  top_n = int(top_n)
1792
  if top_n < 1 or top_n > 25:
1793
  raise ValueError('Number of candidate features must be between 1 and 25.')
 
 
 
1794
  rows = self._contrast_prompt_rows(n)
1795
  available = sorted({str(row['concept']) for row in rows})
1796
  if concept not in available:
1797
  raise ValueError(f'Concept must be one of {available}.')
1798
 
 
 
1799
  texts = [str(row['text']) for row in rows]
 
 
 
1800
  batch = self.tokenizer(
1801
  texts,
1802
  return_tensors='pt',
@@ -1826,14 +1861,17 @@ class FeatureLensRuntime:
1826
  reduce='amax',
1827
  include_self=True,
1828
  )
 
 
 
1829
  target_mask = torch.tensor(
1830
  [str(row['concept']) == concept for row in rows],
1831
  device=dense.device,
1832
  dtype=torch.bool,
1833
  )
1834
  other_mask = ~target_mask
1835
- target = dense[target_mask]
1836
- other = dense[other_mask]
1837
  target_mean = target.mean(dim=0)
1838
  other_mean = other.mean(dim=0)
1839
  target_rate = (target > 0).float().mean(dim=0)
@@ -1841,6 +1879,31 @@ class FeatureLensRuntime:
1841
  mean_diff = target_mean - other_mean
1842
  selectivity = mean_diff / (target_mean + other_mean + 1e-8)
1843
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1844
  eligible = (target_mean > 0) & (mean_diff > 0)
1845
  candidate_idx = torch.nonzero(eligible, as_tuple=False).flatten()
1846
  if candidate_idx.numel() == 0:
@@ -1849,38 +1912,61 @@ class FeatureLensRuntime:
1849
  layer=int(layer),
1850
  prompts_per_concept=n,
1851
  top_n=top_n,
 
1852
  rows=[],
1853
  chart_rows=[],
1854
  candidate_ids=[],
 
 
 
1855
  )
1856
- order = torch.argsort(mean_diff[candidate_idx], descending=True)
 
 
1857
  candidate_idx = candidate_idx[order[:top_n]]
1858
 
1859
  table_rows: list[list[object]] = []
1860
  chart_rows: list[list[object]] = []
 
1861
  for rank, feature_tensor in enumerate(candidate_idx, start=1):
1862
  fid = int(feature_tensor.item())
 
 
 
 
 
1863
  row = [
1864
  rank,
1865
  fid,
 
1866
  float(target_mean[fid].item()),
1867
  float(other_mean[fid].item()),
1868
  float(mean_diff[fid].item()),
1869
  float(selectivity[fid].item()),
1870
  float(target_rate[fid].item()),
1871
  float(other_rate[fid].item()),
1872
- float(target[:, fid].max().item()),
 
 
1873
  ]
1874
  table_rows.append(row)
1875
- chart_rows.append([str(fid), float(mean_diff[fid].item())])
 
 
 
 
1876
  return ConceptFeatureDiscoveryResult(
1877
  concept=concept,
1878
  layer=int(layer),
1879
  prompts_per_concept=n,
1880
  top_n=top_n,
 
1881
  rows=table_rows,
1882
  chart_rows=chart_rows,
1883
  candidate_ids=[int(x.item()) for x in candidate_idx],
 
 
 
1884
  )
1885
 
1886
  @staticmethod
@@ -1964,6 +2050,102 @@ class FeatureLensRuntime:
1964
  cue_count=len(cue_list),
1965
  )
1966
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1967
 
1968
  RUNTIME = FeatureLensRuntime()
1969
 
 
2
 
3
  import hashlib
4
  import html
5
+ import json
6
  import math
7
  import os
 
 
8
  from collections.abc import Iterator, Sequence
9
  from contextlib import contextmanager
10
  from dataclasses import dataclass
11
+ from pathlib import Path
12
 
13
  import torch
14
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
233
  layer: int
234
  prompts_per_concept: int
235
  top_n: int
236
+ ranking_mode: str
237
  rows: list[list[object]]
238
  chart_rows: list[list[object]]
239
  candidate_ids: list[int]
240
+ default_candidate_id: int | None
241
+ current_context_available: bool
242
+ current_token_index: int | None
243
 
244
 
245
  @dataclass
 
253
  cue_count: int
254
 
255
 
256
+ @dataclass
257
+ class FeatureCueContextResult:
258
+ feature_id: int
259
+ layer: int
260
+ stems: list[str]
261
+ cues: list[str]
262
+ rows: list[list[object]]
263
+ chart_rows: list[list[object]]
264
+ active_condition_count: int
265
+ condition_count: int
266
+ cue_active_context_counts: dict[str, int]
267
+
268
+
269
  @dataclass
270
  class ParaphraseResult:
271
  tokens_a: list[str]
 
491
 
492
  @staticmethod
493
  def _control_seed(text: str, layer: int, key: str, mode: str, coefficient: float) -> int:
494
+ payload = f'{text}\0{layer}\0{key}\0{mode}\0{coefficient:.8g}'.encode()
495
  return int.from_bytes(hashlib.sha256(payload).digest()[:4], 'big', signed=False)
496
 
497
  @staticmethod
 
1795
  layer: int,
1796
  prompts_per_concept: int | None = None,
1797
  top_n: int = 12,
1798
+ ranking_mode: str = 'balanced_selectivity',
1799
+ current_text: str | None = None,
1800
+ current_token_index: int = -1,
1801
  ) -> ConceptFeatureDiscoveryResult:
1802
+ """Find exploratory concept candidates and show whether they are usable in the current Workbench context.
1803
+
1804
+ ``balanced_selectivity`` downweights globally high-activation features by combining target selectivity,
1805
+ target coverage, and target activation magnitude. ``raw_mean_difference`` preserves the simpler raw
1806
+ mean-difference ranking for comparison. If a Workbench prompt is supplied it is appended to the same
1807
+ model batch, so
1808
+ current-prompt compatibility does not require another forward pass.
1809
+ """
1810
  self.ensure_ready(preload_saes=False)
1811
  assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
1812
  concept = str(concept).strip()
 
1818
  top_n = int(top_n)
1819
  if top_n < 1 or top_n > 25:
1820
  raise ValueError('Number of candidate features must be between 1 and 25.')
1821
+ if ranking_mode not in {'balanced_selectivity', 'raw_mean_difference'}:
1822
+ raise ValueError("ranking_mode must be 'balanced_selectivity' or 'raw_mean_difference'.")
1823
+
1824
  rows = self._contrast_prompt_rows(n)
1825
  available = sorted({str(row['concept']) for row in rows})
1826
  if concept not in available:
1827
  raise ValueError(f'Concept must be one of {available}.')
1828
 
1829
+ current_text = str(current_text or '').strip()
1830
+ current_context_available = bool(current_text)
1831
  texts = [str(row['text']) for row in rows]
1832
+ if current_context_available:
1833
+ texts.append(current_text)
1834
+
1835
  batch = self.tokenizer(
1836
  texts,
1837
  return_tensors='pt',
 
1861
  reduce='amax',
1862
  include_self=True,
1863
  )
1864
+
1865
+ controlled_count = len(rows)
1866
+ controlled_dense = dense[:controlled_count]
1867
  target_mask = torch.tensor(
1868
  [str(row['concept']) == concept for row in rows],
1869
  device=dense.device,
1870
  dtype=torch.bool,
1871
  )
1872
  other_mask = ~target_mask
1873
+ target = controlled_dense[target_mask]
1874
+ other = controlled_dense[other_mask]
1875
  target_mean = target.mean(dim=0)
1876
  other_mean = other.mean(dim=0)
1877
  target_rate = (target > 0).float().mean(dim=0)
 
1879
  mean_diff = target_mean - other_mean
1880
  selectivity = mean_diff / (target_mean + other_mean + 1e-8)
1881
 
1882
+ # A scale-aware but selectivity-first exploratory score. log1p prevents very large SAE coefficients from
1883
+ # overwhelming features that are much more exclusive to the target concept.
1884
+ balanced_score = selectivity.clamp_min(0) * target_rate * torch.log1p(target_mean.clamp_min(0))
1885
+
1886
+ current_prompt_max = torch.zeros(self.settings.sae_width, device=dense.device, dtype=torch.float32)
1887
+ current_token_dense = torch.zeros_like(current_prompt_max)
1888
+ resolved_current_idx: int | None = None
1889
+ if current_context_available:
1890
+ current_row = controlled_count
1891
+ current_prompt_max = dense[current_row]
1892
+ valid_positions = torch.nonzero(attention[current_row], as_tuple=False).flatten()
1893
+ prompt_len = int(valid_positions.numel())
1894
+ if prompt_len:
1895
+ resolved_current_idx = self._resolve_index(int(current_token_index), prompt_len)
1896
+ padded_position = int(valid_positions[resolved_current_idx].item())
1897
+ token_ids = encoding.indices[current_row, padded_position]
1898
+ token_values = encoding.values[current_row, padded_position].float().clamp_min(0)
1899
+ current_token_dense.scatter_reduce_(
1900
+ 0,
1901
+ token_ids,
1902
+ token_values,
1903
+ reduce='amax',
1904
+ include_self=True,
1905
+ )
1906
+
1907
  eligible = (target_mean > 0) & (mean_diff > 0)
1908
  candidate_idx = torch.nonzero(eligible, as_tuple=False).flatten()
1909
  if candidate_idx.numel() == 0:
 
1912
  layer=int(layer),
1913
  prompts_per_concept=n,
1914
  top_n=top_n,
1915
+ ranking_mode=ranking_mode,
1916
  rows=[],
1917
  chart_rows=[],
1918
  candidate_ids=[],
1919
+ default_candidate_id=None,
1920
+ current_context_available=current_context_available,
1921
+ current_token_index=resolved_current_idx,
1922
  )
1923
+
1924
+ ranking_values = balanced_score if ranking_mode == 'balanced_selectivity' else mean_diff
1925
+ order = torch.argsort(ranking_values[candidate_idx], descending=True)
1926
  candidate_idx = candidate_idx[order[:top_n]]
1927
 
1928
  table_rows: list[list[object]] = []
1929
  chart_rows: list[list[object]] = []
1930
+ default_candidate_id: int | None = None
1931
  for rank, feature_tensor in enumerate(candidate_idx, start=1):
1932
  fid = int(feature_tensor.item())
1933
+ current_max = float(current_prompt_max[fid].item()) if current_context_available else 0.0
1934
+ current_token = float(current_token_dense[fid].item()) if current_context_available else 0.0
1935
+ if default_candidate_id is None and current_token > 0:
1936
+ default_candidate_id = fid
1937
+ score = float(ranking_values[fid].item())
1938
  row = [
1939
  rank,
1940
  fid,
1941
+ score,
1942
  float(target_mean[fid].item()),
1943
  float(other_mean[fid].item()),
1944
  float(mean_diff[fid].item()),
1945
  float(selectivity[fid].item()),
1946
  float(target_rate[fid].item()),
1947
  float(other_rate[fid].item()),
1948
+ current_max,
1949
+ current_token,
1950
+ bool(current_token > 0),
1951
  ]
1952
  table_rows.append(row)
1953
+ chart_rows.append([str(fid), score])
1954
+
1955
+ if default_candidate_id is None and candidate_idx.numel():
1956
+ default_candidate_id = int(candidate_idx[0].item())
1957
+
1958
  return ConceptFeatureDiscoveryResult(
1959
  concept=concept,
1960
  layer=int(layer),
1961
  prompts_per_concept=n,
1962
  top_n=top_n,
1963
+ ranking_mode=ranking_mode,
1964
  rows=table_rows,
1965
  chart_rows=chart_rows,
1966
  candidate_ids=[int(x.item()) for x in candidate_idx],
1967
+ default_candidate_id=default_candidate_id,
1968
+ current_context_available=current_context_available,
1969
+ current_token_index=resolved_current_idx,
1970
  )
1971
 
1972
  @staticmethod
 
2050
  cue_count=len(cue_list),
2051
  )
2052
 
2053
+ @torch.inference_mode()
2054
+ def feature_cue_context_scan(
2055
+ self,
2056
+ feature_id: int,
2057
+ layer: int,
2058
+ stems: Sequence[str],
2059
+ cues: Sequence[str],
2060
+ ) -> FeatureCueContextResult:
2061
+ """Cross completion cues with multiple prompt stems in one batch.
2062
+
2063
+ This distinguishes a cue-specific response (for example a feature that responds to ``is`` everywhere)
2064
+ from a context-sensitive completion-boundary response.
2065
+ """
2066
+ stem_list: list[str] = []
2067
+ seen_stems: set[str] = set()
2068
+ for raw in stems:
2069
+ stem = str(raw).strip()
2070
+ if not stem or stem in seen_stems:
2071
+ continue
2072
+ seen_stems.add(stem)
2073
+ stem_list.append(stem)
2074
+ cue_list: list[str] = []
2075
+ seen_cues: set[str] = set()
2076
+ for raw in cues:
2077
+ cue = str(raw).strip()
2078
+ if not cue or cue in seen_cues:
2079
+ continue
2080
+ seen_cues.add(cue)
2081
+ cue_list.append(cue)
2082
+ if not stem_list:
2083
+ raise ValueError('Enter at least one prompt stem.')
2084
+ if not cue_list:
2085
+ raise ValueError('Enter at least one completion cue.')
2086
+ if len(stem_list) > 8:
2087
+ raise ValueError('Cue-context scan supports at most 8 prompt stems.')
2088
+ if len(cue_list) > 8:
2089
+ raise ValueError('Cue-context scan supports at most 8 completion cues.')
2090
+ if len(stem_list) * len(cue_list) > 40:
2091
+ raise ValueError('Cue-context scan supports at most 40 stem × cue conditions per run.')
2092
+ if int(feature_id) < 0 or int(feature_id) >= self.settings.sae_width:
2093
+ raise ValueError(f'Feature id must be in [0, {self.settings.sae_width - 1}].')
2094
+
2095
+ self.ensure_ready(preload_saes=False)
2096
+ assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
2097
+ conditions: list[tuple[str, str, str]] = []
2098
+ for stem in stem_list:
2099
+ for cue in cue_list:
2100
+ conditions.append((stem, cue, self._cue_prompt(stem, cue)))
2101
+ prompts = [full for _, _, full in conditions]
2102
+ batch = self.tokenizer(
2103
+ prompts,
2104
+ return_tensors='pt',
2105
+ padding=True,
2106
+ truncation=True,
2107
+ max_length=self.settings.max_prompt_tokens,
2108
+ )
2109
+ batch = {key: value.to(self.device) for key, value in batch.items()}
2110
+ capture: dict = {}
2111
+ with self._capture_hook(int(layer), capture):
2112
+ self.model(**batch, use_cache=False)
2113
+ sae = self.sae_store.get(int(layer))
2114
+ encoding = sae.encode(capture['hidden'])
2115
+ attention = batch.get('attention_mask', torch.ones_like(batch['input_ids'])).bool()
2116
+
2117
+ rows: list[list[object]] = []
2118
+ chart_rows: list[list[object]] = []
2119
+ active_count = 0
2120
+ cue_active_context_counts = {cue: 0 for cue in cue_list}
2121
+ for row_idx, (stem, cue, full_prompt) in enumerate(conditions):
2122
+ valid_positions = torch.nonzero(attention[row_idx], as_tuple=False).flatten()
2123
+ final_pos = int(valid_positions[-1].item())
2124
+ indices = encoding.indices[row_idx, final_pos]
2125
+ values = encoding.values[row_idx, final_pos]
2126
+ mask = indices == int(feature_id)
2127
+ activation = float(values[mask][0].item()) if bool(mask.any()) else 0.0
2128
+ active = activation > 0
2129
+ active_count += int(active)
2130
+ cue_active_context_counts[cue] += int(active)
2131
+ token_id = int(batch['input_ids'][row_idx, final_pos].item())
2132
+ final_token = self.tokenizer.decode([token_id])
2133
+ short_stem = stem if len(stem) <= 42 else stem[:39] + '…'
2134
+ rows.append([stem, cue, full_prompt, repr(final_token), activation, active])
2135
+ chart_rows.append([short_stem, cue, activation])
2136
+
2137
+ return FeatureCueContextResult(
2138
+ feature_id=int(feature_id),
2139
+ layer=int(layer),
2140
+ stems=stem_list,
2141
+ cues=cue_list,
2142
+ rows=rows,
2143
+ chart_rows=chart_rows,
2144
+ active_condition_count=active_count,
2145
+ condition_count=len(conditions),
2146
+ cue_active_context_counts=cue_active_context_counts,
2147
+ )
2148
+
2149
 
2150
  RUNTIME = FeatureLensRuntime()
2151
 
pyproject.toml CHANGED
@@ -1,6 +1,6 @@
1
  [project]
2
  name = "featurelens"
3
- version = "0.6.0"
4
  description = "Causal sparse-feature interpretability workbench for Qwen3 and Qwen-Scope SAEs"
5
  requires-python = ">=3.10"
6
 
 
1
  [project]
2
  name = "featurelens"
3
+ version = "0.7.0"
4
  description = "Causal sparse-feature interpretability workbench for Qwen3 and Qwen-Scope SAEs"
5
  requires-python = ">=3.10"
6
 
research_config.json CHANGED
@@ -14,7 +14,7 @@
14
  "mathematics",
15
  "positive_sentiment",
16
  "negative_sentiment",
17
- "french_language",
18
  "factual_entities",
19
  "uncertainty"
20
  ],
@@ -90,7 +90,7 @@
90
  "contrastive_continuation_preference_test",
91
  "feature_decoder_geometry"
92
  ],
93
- "concept_candidate_discovery_metric": "prompt-wide target-minus-other mean max activation; exploratory only",
94
  "completion_cue_scan": "final-token feature activation after controlled suffix/cue substitution",
95
  "live_features_v0_6": [
96
  "start_here_plain_language_onboarding",
@@ -100,5 +100,14 @@
100
  "consistent_heading_and_table_typography",
101
  "concept_guided_candidate_feature_discovery",
102
  "completion_cue_sensitivity_scan"
 
 
 
 
 
 
 
 
 
103
  ]
104
  }
 
14
  "mathematics",
15
  "positive_sentiment",
16
  "negative_sentiment",
17
+ "german_language",
18
  "factual_entities",
19
  "uncertainty"
20
  ],
 
90
  "contrastive_continuation_preference_test",
91
  "feature_decoder_geometry"
92
  ],
93
+ "concept_candidate_discovery_metric": "balanced exploratory score = selectivity × target activation rate × log1p(target mean), with raw-contrast alternative and current-Workbench compatibility",
94
  "completion_cue_scan": "final-token feature activation after controlled suffix/cue substitution",
95
  "live_features_v0_6": [
96
  "start_here_plain_language_onboarding",
 
100
  "consistent_heading_and_table_typography",
101
  "concept_guided_candidate_feature_discovery",
102
  "completion_cue_sensitivity_scan"
103
+ ],
104
+ "live_features_v0_7": [
105
+ "cleaned_nonaccordion_experiment_layout",
106
+ "focused_fullscreen_modal_for_tables_and_plots",
107
+ "descriptive_plot_export_filenames",
108
+ "german_language_control_concept",
109
+ "balanced_candidate_ranking_and_current_prompt_compatibility",
110
+ "click_to_select_candidate_rows",
111
+ "completion_cue_context_matrix"
112
  ]
113
  }
scripts/release_check.py CHANGED
@@ -5,7 +5,6 @@ import subprocess
5
  from collections import Counter
6
  from pathlib import Path
7
 
8
-
9
  ROOT = Path(__file__).resolve().parents[1]
10
  MAX_FILE_SIZE_BYTES = 5_000_000 # 5 MB
11
 
@@ -136,6 +135,24 @@ def check_config(config: dict) -> None:
136
  f'{sorted(actual_live_v06)}'
137
  )
138
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
139
 
140
  def check_datasets(config: dict) -> tuple[list[dict], list[dict]]:
141
  prompts = load_jsonl(ROOT / 'data' / 'prompts.jsonl')
@@ -201,17 +218,20 @@ def check_readme() -> None:
201
  'token activation',
202
  'concept-guided candidate',
203
  'completion-cue',
 
 
 
204
  'start here',
205
  ]
206
  missing = [value for value in required_strings if value.lower() not in readme.lower()]
207
  if missing:
208
- raise SystemExit(f'README.md is missing required v0.6 content: {missing}')
209
 
210
 
211
  def check_pyproject() -> None:
212
  text = (ROOT / 'pyproject.toml').read_text(encoding='utf-8')
213
- if 'version = "0.6.0"' not in text:
214
- raise SystemExit('pyproject.toml must declare version 0.6.0.')
215
 
216
 
217
  def main() -> None:
@@ -229,7 +249,7 @@ def main() -> None:
229
  print(f' layers: {config["layers"]}')
230
  print(f' feature-set sizes: {config["feature_set_sizes"]}')
231
  print(f' random controls: {config["live_random_controls"]}')
232
- print(' release: v0.6.0')
233
 
234
 
235
  if __name__ == '__main__':
 
5
  from collections import Counter
6
  from pathlib import Path
7
 
 
8
  ROOT = Path(__file__).resolve().parents[1]
9
  MAX_FILE_SIZE_BYTES = 5_000_000 # 5 MB
10
 
 
135
  f'{sorted(actual_live_v06)}'
136
  )
137
 
138
+ required_live_v07 = {
139
+ 'cleaned_nonaccordion_experiment_layout',
140
+ 'focused_fullscreen_modal_for_tables_and_plots',
141
+ 'descriptive_plot_export_filenames',
142
+ 'german_language_control_concept',
143
+ 'balanced_candidate_ranking_and_current_prompt_compatibility',
144
+ 'click_to_select_candidate_rows',
145
+ 'completion_cue_context_matrix',
146
+ }
147
+ actual_live_v07 = set(config.get('live_features_v0_7', []))
148
+ if actual_live_v07 != required_live_v07:
149
+ raise SystemExit(
150
+ 'research_config.json live_features_v0_7 mismatch: '
151
+ f'{sorted(actual_live_v07)}'
152
+ )
153
+ if 'german_language' not in config.get('concepts', []) or 'french_language' in config.get('concepts', []):
154
+ raise SystemExit('research_config.json must use german_language and must not contain french_language.')
155
+
156
 
157
  def check_datasets(config: dict) -> tuple[list[dict], list[dict]]:
158
  prompts = load_jsonl(ROOT / 'data' / 'prompts.jsonl')
 
218
  'token activation',
219
  'concept-guided candidate',
220
  'completion-cue',
221
+ 'cue × context',
222
+ 'balanced selectivity',
223
+ 'german',
224
  'start here',
225
  ]
226
  missing = [value for value in required_strings if value.lower() not in readme.lower()]
227
  if missing:
228
+ raise SystemExit(f'README.md is missing required v0.7 content: {missing}')
229
 
230
 
231
  def check_pyproject() -> None:
232
  text = (ROOT / 'pyproject.toml').read_text(encoding='utf-8')
233
+ if 'version = "0.7.0"' not in text:
234
+ raise SystemExit('pyproject.toml must declare version 0.7.0.')
235
 
236
 
237
  def main() -> None:
 
249
  print(f' layers: {config["layers"]}')
250
  print(f' feature-set sizes: {config["feature_set_sizes"]}')
251
  print(f' random controls: {config["live_random_controls"]}')
252
+ print(' release: v0.7.0')
253
 
254
 
255
  if __name__ == '__main__':
tests/test_data.py CHANGED
@@ -16,6 +16,11 @@ def test_discovery_dataset_is_balanced_and_paired() -> None:
16
  assert len(rows) == 224
17
  concept_counts = Counter(row['concept'] for row in rows)
18
  assert set(concept_counts.values()) == {32}
 
 
 
 
 
19
  pair_counts = Counter(row['pair_id'] for row in rows)
20
  assert set(pair_counts.values()) == {2}
21
 
@@ -25,3 +30,6 @@ def test_causal_dataset_covers_every_discovery_concept() -> None:
25
  causal = read_jsonl(ROOT / 'data' / 'causal_tasks.jsonl')
26
  assert len(causal) == 28
27
  assert {row['concept'] for row in causal} == {row['concept'] for row in discovery}
 
 
 
 
16
  assert len(rows) == 224
17
  concept_counts = Counter(row['concept'] for row in rows)
18
  assert set(concept_counts.values()) == {32}
19
+ assert 'german_language' in concept_counts
20
+ assert 'french_language' not in concept_counts
21
+ german = [row for row in rows if row['concept'] == 'german_language']
22
+ assert len(german) == 32
23
+ assert any('Tisch' in row['text'] or 'Deutsch' in row['text'] or 'Hamburg' in row['text'] for row in german)
24
  pair_counts = Counter(row['pair_id'] for row in rows)
25
  assert set(pair_counts.values()) == {2}
26
 
 
30
  causal = read_jsonl(ROOT / 'data' / 'causal_tasks.jsonl')
31
  assert len(causal) == 28
32
  assert {row['concept'] for row in causal} == {row['concept'] for row in discovery}
33
+ german = [row for row in causal if row['concept'] == 'german_language']
34
+ assert len(german) == 4
35
+ assert all(row['target'].strip() in {'hallo', 'danke', 'ja', 'guten Abend'} for row in german)
tests/test_live_runtime_helpers.py CHANGED
@@ -219,12 +219,41 @@ def test_concept_contrast_promptwide_scan_runs_on_toy_runtime() -> None:
219
  def test_concept_feature_discovery_runs_on_toy_runtime() -> None:
220
  runtime = make_runtime()
221
  result = runtime.concept_feature_discovery(
222
- concept='mathematics', layer=0, prompts_per_concept=1, top_n=3
 
 
 
 
 
 
223
  )
224
  assert result.concept == 'mathematics'
 
 
 
225
  assert len(result.rows) <= 3
226
  assert result.candidate_ids == [int(row[1]) for row in result.rows]
227
- assert all(len(row) == 9 for row in result.rows)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
228
 
229
 
230
  def test_feature_cue_scan_runs_on_toy_runtime() -> None:
@@ -239,3 +268,19 @@ def test_feature_cue_scan_runs_on_toy_runtime() -> None:
239
  assert len(result.rows) == 3
240
  assert all(len(row) == 5 for row in result.rows)
241
  assert 0 <= result.active_cue_count <= result.cue_count
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
219
  def test_concept_feature_discovery_runs_on_toy_runtime() -> None:
220
  runtime = make_runtime()
221
  result = runtime.concept_feature_discovery(
222
+ concept='mathematics',
223
+ layer=0,
224
+ prompts_per_concept=1,
225
+ top_n=3,
226
+ ranking_mode='balanced_selectivity',
227
+ current_text='abc',
228
+ current_token_index=-1,
229
  )
230
  assert result.concept == 'mathematics'
231
+ assert result.ranking_mode == 'balanced_selectivity'
232
+ assert result.current_context_available is True
233
+ assert result.current_token_index == 2
234
  assert len(result.rows) <= 3
235
  assert result.candidate_ids == [int(row[1]) for row in result.rows]
236
+ assert all(len(row) == 12 for row in result.rows)
237
+ if result.rows:
238
+ assert result.default_candidate_id in result.candidate_ids
239
+ assert all(math.isfinite(float(row[2])) for row in result.rows)
240
+ assert all(float(row[9]) >= 0 for row in result.rows) # current prompt max
241
+ assert all(float(row[10]) >= 0 for row in result.rows) # current token activation
242
+
243
+
244
+ def test_concept_feature_discovery_supports_raw_mean_difference() -> None:
245
+ runtime = make_runtime()
246
+ result = runtime.concept_feature_discovery(
247
+ concept='mathematics',
248
+ layer=0,
249
+ prompts_per_concept=1,
250
+ top_n=3,
251
+ ranking_mode='raw_mean_difference',
252
+ )
253
+ assert result.ranking_mode == 'raw_mean_difference'
254
+ assert result.current_context_available is False
255
+ assert result.current_token_index is None
256
+ assert all(len(row) == 12 for row in result.rows)
257
 
258
 
259
  def test_feature_cue_scan_runs_on_toy_runtime() -> None:
 
268
  assert len(result.rows) == 3
269
  assert all(len(row) == 5 for row in result.rows)
270
  assert 0 <= result.active_cue_count <= result.cue_count
271
+
272
+
273
+ def test_feature_cue_context_scan_runs_on_toy_runtime() -> None:
274
+ runtime = make_runtime()
275
+ result = runtime.feature_cue_context_scan(
276
+ feature_id=0,
277
+ layer=0,
278
+ stems=['abc', 'xyz'],
279
+ cues=['is', '=', ':'],
280
+ )
281
+ assert result.condition_count == 6
282
+ assert len(result.rows) == 6
283
+ assert all(len(row) == 6 for row in result.rows)
284
+ assert 0 <= result.active_condition_count <= result.condition_count
285
+ assert set(result.cue_active_context_counts) == {'is', '=', ':'}
286
+ assert len(result.chart_rows) == 6
tests/test_ui_helpers.py CHANGED
@@ -42,3 +42,31 @@ def test_tsv_copy_payload_keeps_headers() -> None:
42
  payload = app._tsv(frame)
43
  assert payload.startswith('Feature id\tActivation\n')
44
  assert payload.endswith('1\t2.0\n')
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  payload = app._tsv(frame)
43
  assert payload.startswith('Feature id\tActivation\n')
44
  assert payload.endswith('1\t2.0\n')
45
+
46
+
47
+ def test_use_candidate_feature_returns_explicit_handoff_status() -> None:
48
+ app = _import_app()
49
+ outputs = app.use_candidate_feature('21885')
50
+ assert outputs[:4] == ('21885', '21885', '21885', '21885')
51
+ assert 'Feature 21885 loaded' in outputs[4]
52
+ assert 'Single-feature causal test' in outputs[4]
53
+
54
+
55
+ def test_select_candidate_row_uses_feature_id_column() -> None:
56
+ app = _import_app()
57
+ table = pd.DataFrame(
58
+ [[1, 21885, 3.4], [2, 445, 3.3]],
59
+ columns=['Rank', 'Feature id', 'Candidate score'],
60
+ )
61
+ event = SimpleNamespace(index=(1, 0))
62
+ update = app.select_candidate_row(table, event)
63
+ # Gradio returns an update dictionary-like object in current releases.
64
+ assert update['value'] == '445'
65
+
66
+
67
+ def test_frontend_helpers_name_exports_and_preserve_focus_position() -> None:
68
+ app = _import_app()
69
+ assert 'featurelens_${stem' in app.INSTALL_REFLOW_JS
70
+ assert 'chart.png' in app.INSTALL_REFLOW_JS
71
+ assert '__featurelens_focus_scroll_y' in app.INSTALL_REFLOW_JS
72
+ assert 'window.scrollTo' in app.INSTALL_REFLOW_JS