Release FeatureLens v0.11.0

CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## v0.11.0
+
+- Added **Controlled candidate specificity**, a one-batch follow-up that compares up to three candidate SAE ablations against each candidate's own 8-direction norm-matched random ensemble.
+- Added a strategic controlled shortlist that preserves the discovery leader, target-effect leader, and distribution-shift leader when they differ, then fills remaining slots by triage target rank.
+- Added target-specificity and JS-specificity ratios plus coarse empirical random-control tails for every controlled candidate.
+- Added **Association vs controlled causality**, joining concept-discovery evidence to random-normalized causal specificity instead of relying only on raw triage magnitude.
+- Kept target-specific and whole-distribution causal influence separate rather than collapsing them into one score.
+- Kept the validated in-place focus/zoom implementation unchanged.
+- Reduced HF acceptance to one new GPU call; unchanged discovery/triage and other regression paths remain covered by automated tests.
+
 ## v0.10.0
 
 - Added a zero-extra-GPU **Discovery–causality alignment** panel after candidate triage.

README.md

@@ -13,7 +13,7 @@ license: mit
 
 # FeatureLens — Causal Interpretability Workbench
 
-> **v0.10:** an association-to-causality synthesis workflow that joins concept discovery and batched causal triage, exposes rank disagreement, and quantifies descriptive concordance without spending another GPU call.
+> **v0.11:** a controlled association-to-causality workflow that follows cheap candidate triage with per-feature norm-matched random ensembles, separating raw target effect from random-normalized causal specificity.
 
 **Research question:**
 
@@ -30,14 +30,15 @@ FeatureLens uses **Qwen3-1.7B-Base** with **Qwen-Scope residual-stream sparse au
 4. **Candidate discovery** — which features are more active for a chosen controlled concept than for the other groups?
 5. **Controlled contrast** — does a selected feature prefer one concept group in a balanced live mini-batch?
 6. **Cue sensitivity** — is a feature responding to a lexical/completion cue rather than the underlying concept?
-7. **Single-feature causality** — does ablation/scaling change downstream probability?
-8. **Dose-response** — does effect size vary coherently with feature coefficient?
-9. **Feature-set causality** — do small sparse subspaces matter more than one feature?
-10. **Non-additivity** — does a joint edit differ from the sum of individual effects?
-11. **Specificity** — are targeted effects larger than norm-matched random residual perturbations?
-12. **Contrastive specificity** — does an edit change preference between a desired and comparison continuation?
-13. **Geometry** — are selected SAE decoder directions aligned, orthogonal, or cancelling before downstream model non-linearity?
-14. **Association–causality concordance** — do features ranked highly by concept evidence also rank highly by downstream target effect or distribution shift?
+7. **Candidate causal specificity** — do shortlisted candidate ablations exceed their own norm-matched random perturbation ensembles?
+8. **Single-feature causality** — does ablation/scaling change downstream probability?
+9. **Dose-response** — does effect size vary coherently with feature coefficient?
+10. **Feature-set causality** — do small sparse subspaces matter more than one feature?
+11. **Non-additivity** — does a joint edit differ from the sum of individual effects?
+12. **Specificity** — are targeted effects larger than norm-matched random residual perturbations?
+13. **Contrastive specificity** — does an edit change preference between a desired and comparison continuation?
+14. **Geometry** — are selected SAE decoder directions aligned, orthogonal, or cancelling before downstream model non-linearity?
+15. **Association–causality concordance** — do features ranked highly by concept evidence also rank highly by downstream target effect or distribution shift?
 
 A high AUROC, high paraphrase overlap, or a large activation is still correlational evidence. Causal evidence comes from downstream change under controlled intervention.
 
@@ -358,6 +359,15 @@ FeatureLens/
 - **Association-vs-target-effect scatter** is generated from the same two existing GPU calls; the synthesis itself costs no additional GPU time.
 - Existing v0.9 causal-ready discovery, candidate triage, in-place focus, and GPU-budget-aware validation remain unchanged.
 
+## v0.11 controlled candidate specificity
+
+- **Controlled candidate specificity** compares up to three shortlisted SAE ablations against each candidate's own eight-direction norm-matched random ensemble in one shared batched execution context.
+- **Strategic shortlist selection** preserves the discovery leader, target-effect leader, and distribution-shift leader when they differ, then fills remaining slots by triage target rank. This avoids confirming only whichever metric already won.
+- **Two controlled causal notions remain separate:** target-specificity ratio normalizes the specified continuation effect; JS-specificity ratio normalizes whole next-token distribution shift.
+- **Association vs controlled causality** joins discovery score/rank to random-normalized target and JS specificity, with descriptive Spearman concordance and rank shifts.
+- The controlled live screen remains deliberately small: three candidates × eight controls each plus one zero-edit reference. Empirical tails are therefore coarse diagnostics rather than conventional significance tests.
+- The validated in-place focus behavior and unchanged paraphrase/trajectory/feature-set paths are not touched. HF acceptance for v0.11 requires only the new controlled-candidate GPU path.
+
 ## Validation
 
 ```bash
@@ -368,7 +378,7 @@ python scripts/ui_smoke.py
 python scripts/release_check.py
 ```
 
-See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.10 validation guide uses the **exact labels shown in the UI**. It reuses the existing discovery + triage workflow and does not ask for paraphrase, trajectory, feature-set, dose-response, or focus regressions because those implementations were not changed.
+See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.11 validation guide uses the **exact labels shown in the UI** and asks for only the new controlled-candidate comparison on HF. Discovery, triage, paraphrase, trajectory, feature-set, dose-response, cue, and focus paths are not rerun because their implementations were not changed.
 
 ## Limitations
 
@@ -384,7 +394,7 @@ See [`docs/VALIDATION.md`](docs/VALIDATION.md). The v0.10 validation guide uses
 ## Resume-ready description
 
 > **FeatureLens — Causal Interpretability Workbench** | PyTorch, Qwen3, Sparse Autoencoders, Mechanistic Interpretability, Gradio
-> Built an SAE-based interpretability system for Qwen3-1.7B with held-out concept discovery, concept-guided candidate discovery, token/prompt-wide, completion-cue, and cue × context feature evidence, reconstruction-preserving single and multi-feature interventions, full-continuation and contrastive preference scoring, dose-response analysis, decoder-geometry/non-additivity diagnostics, discovery-to-causality rank analysis, and norm-matched random-control ensembles.
+> Built an SAE-based interpretability system for Qwen3-1.7B with held-out concept discovery, concept-guided candidate discovery, token/prompt-wide, completion-cue, and cue × context feature evidence, reconstruction-preserving single and multi-feature interventions, full-continuation and contrastive preference scoring, dose-response analysis, decoder-geometry/non-additivity diagnostics, discovery-to-causality rank analysis, multi-candidate random-controlled specificity screening, and norm-matched random-control ensembles.
 
 ## Acknowledgements
 

app.py

@@ -749,6 +749,218 @@ def _candidate_alignment_outputs(
     return summary, table, chart
 
 
+
+def _controlled_candidate_shortlist(
+    discovery_table: pd.DataFrame | None,
+    screen_table: pd.DataFrame | None,
+    limit: int = 3,
+) -> list[str]:
+    """Pick a small controlled follow-up set without another model call.
+
+    Prefer the strongest discovery candidate, strongest target-effect candidate,
+    and strongest distribution-shift candidate. Fill any duplicate slots using
+    target-effect rank. This preserves the association-vs-causality contrast
+    instead of blindly testing only the triage top-k.
+    """
+    if screen_table is None:
+        return []
+    screen = pd.DataFrame(screen_table).copy()
+    if screen.empty or "Feature id" not in screen:
+        return []
+    screen["Feature id"] = pd.to_numeric(screen["Feature id"], errors="coerce")
+    screen = screen.dropna(subset=["Feature id"]).copy()
+    screen["Feature id"] = screen["Feature id"].astype(int)
+    if screen.empty:
+        return []
+
+    selected: list[int] = []
+
+    def add(feature_id: int) -> None:
+        if feature_id not in selected and len(selected) < int(limit):
+            selected.append(feature_id)
+
+    if discovery_table is not None:
+        discovery = pd.DataFrame(discovery_table).copy()
+        if not discovery.empty and {"Feature id", "Rank"}.issubset(discovery.columns):
+            discovery["Feature id"] = pd.to_numeric(discovery["Feature id"], errors="coerce")
+            discovery = discovery.dropna(subset=["Feature id"]).copy()
+            discovery["Feature id"] = discovery["Feature id"].astype(int)
+            screened_ids = set(screen["Feature id"].astype(int).tolist())
+            discovery = discovery[discovery["Feature id"].isin(screened_ids)]
+            if not discovery.empty:
+                top_discovery = discovery.sort_values(["Rank", "Feature id"]).iloc[0]
+                add(int(top_discovery["Feature id"]))
+
+    target_sorted = screen.sort_values(["Rank", "Feature id"])
+    add(int(target_sorted.iloc[0]["Feature id"]))
+
+    if "Next-token JS" in screen.columns:
+        js_sorted = screen.sort_values(["Next-token JS", "Feature id"], ascending=[False, True])
+        add(int(js_sorted.iloc[0]["Feature id"]))
+
+    for feature_id in target_sorted["Feature id"].astype(int).tolist():
+        add(feature_id)
+        if len(selected) >= int(limit):
+            break
+    return [str(feature_id) for feature_id in selected]
+
+
+def _candidate_specificity_metrics_markdown(result) -> str:
+    tokens = " ".join(repr(token) for token in result.target_tokens)
+    if result.rows:
+        top = result.rows[0]
+        strongest = (
+            f"Highest target-specificity ratio: feature **{int(top[1])}** at **{float(top[9]):.2f}×** "
+            f"the random mean |effect| (empirical tail **{float(top[10]):.3f}**)."
+        )
+    else:
+        strongest = "No controlled candidate rows were produced."
+    return (
+        f"Compared **{result.candidate_count}** candidate feature(s); **{result.active_feature_count}** were active "
+        f"at the selected Workbench token. Each candidate uses **{result.random_control_count}** norm-matched random controls.  \n"
+        f"Target continuation: {len(result.target_tokens)} token(s): {tokens}.  \n"
+        f"{strongest}  \n"
+        f"Execution-context null drift: mean log p/token **{result.execution_drift_mean_logprob:+.2e}**, "
+        f"JS **{result.execution_drift_js:.2e}**.  \n\n"
+        "This is the controlled follow-up to triage: target-specificity compares |Δ mean log p/token| against the "
+        "candidate's own norm-matched random ensemble; JS-specificity does the same for the local next-token distribution. "
+        "With eight controls, empirical tail values remain intentionally coarse."
+    )
+
+
+def _controlled_alignment_outputs(
+    discovery_table: pd.DataFrame | None,
+    specificity_table: pd.DataFrame | None,
+) -> tuple[str, pd.DataFrame, pd.DataFrame]:
+    """Join concept evidence to random-controlled causal specificity."""
+    columns = [
+        "Feature id",
+        "Discovery rank",
+        "Specificity rank",
+        "Target-effect rank",
+        "JS-specificity rank",
+        "Candidate score",
+        "|SAE Δ mean log p/token|",
+        "Random mean |Δ|",
+        "Target specificity ratio",
+        "Target empirical tail p",
+        "SAE next-token JS",
+        "Random mean JS",
+        "JS specificity ratio",
+        "JS empirical tail p",
+        "Discovery→specificity rank shift",
+    ]
+    if discovery_table is None or specificity_table is None:
+        return "", pd.DataFrame(columns=columns), pd.DataFrame()
+    discovery = pd.DataFrame(discovery_table).copy()
+    controlled = pd.DataFrame(specificity_table).copy()
+    if discovery.empty or controlled.empty or "Feature id" not in discovery or "Feature id" not in controlled:
+        return "", pd.DataFrame(columns=columns), pd.DataFrame()
+
+    discovery["Feature id"] = pd.to_numeric(discovery["Feature id"], errors="coerce")
+    controlled["Feature id"] = pd.to_numeric(controlled["Feature id"], errors="coerce")
+    discovery = discovery.dropna(subset=["Feature id"]).copy()
+    controlled = controlled.dropna(subset=["Feature id"]).copy()
+    discovery["Feature id"] = discovery["Feature id"].astype(int)
+    controlled["Feature id"] = controlled["Feature id"].astype(int)
+
+    needed_discovery = {"Rank", "Candidate score"}
+    needed_controlled = {
+        "Rank",
+        "SAE Δ mean log p/token",
+        "Random mean |Δ|",
+        "Target specificity ratio",
+        "Target empirical tail p",
+        "SAE next-token JS",
+        "Random mean JS",
+        "JS specificity ratio",
+        "JS empirical tail p",
+    }
+    if not needed_discovery.issubset(discovery.columns) or not needed_controlled.issubset(controlled.columns):
+        return "", pd.DataFrame(columns=columns), pd.DataFrame()
+
+    discovery_lookup = discovery.set_index("Feature id", drop=False)
+    target_ranked = controlled.assign(
+        _abs_target=controlled["SAE Δ mean log p/token"].astype(float).abs()
+    ).sort_values(["_abs_target", "Feature id"], ascending=[False, True])
+    target_ranks = {
+        int(row["Feature id"]): rank for rank, (_, row) in enumerate(target_ranked.iterrows(), start=1)
+    }
+    js_ranked = controlled.sort_values(
+        ["JS specificity ratio", "Feature id"], ascending=[False, True]
+    )
+    js_ranks = {
+        int(row["Feature id"]): rank for rank, (_, row) in enumerate(js_ranked.iterrows(), start=1)
+    }
+
+    rows: list[list[object]] = []
+    for _, row in controlled.iterrows():
+        feature_id = int(row["Feature id"])
+        if feature_id not in discovery_lookup.index:
+            continue
+        discovery_row = discovery_lookup.loc[feature_id]
+        if isinstance(discovery_row, pd.DataFrame):
+            discovery_row = discovery_row.iloc[0]
+        discovery_rank = int(float(discovery_row["Rank"]))
+        specificity_rank = int(float(row["Rank"]))
+        rows.append(
+            [
+                feature_id,
+                discovery_rank,
+                specificity_rank,
+                int(target_ranks[feature_id]),
+                int(js_ranks[feature_id]),
+                float(discovery_row["Candidate score"]),
+                abs(float(row["SAE Δ mean log p/token"])),
+                float(row["Random mean |Δ|"]),
+                float(row["Target specificity ratio"]),
+                float(row["Target empirical tail p"]),
+                float(row["SAE next-token JS"]),
+                float(row["Random mean JS"]),
+                float(row["JS specificity ratio"]),
+                float(row["JS empirical tail p"]),
+                discovery_rank - specificity_rank,
+            ]
+        )
+
+    table = pd.DataFrame(rows, columns=columns)
+    if table.empty:
+        return "", table, pd.DataFrame()
+
+    rho_target = _spearman_rank_corr(
+        table["Candidate score"].astype(float).tolist(),
+        table["Target specificity ratio"].astype(float).tolist(),
+    )
+    rho_js = _spearman_rank_corr(
+        table["Candidate score"].astype(float).tolist(),
+        table["JS specificity ratio"].astype(float).tolist(),
+    )
+    top_discovery = table.sort_values(["Discovery rank", "Feature id"]).iloc[0]
+    top_specificity = table.sort_values(["Specificity rank", "Feature id"]).iloc[0]
+    top_js = table.sort_values(["JS-specificity rank", "Feature id"]).iloc[0]
+
+    def fmt_rho(value: float | None) -> str:
+        return "undefined" if value is None else f"{value:+.3f}"
+
+    summary = (
+        f"Controlled comparison covers **{len(table)}** candidates from the discovery/triage workflow.  \n"
+        f"Top discovery candidate: **{int(top_discovery['Feature id'])}** · strongest random-normalized target effect: "
+        f"**{int(top_specificity['Feature id'])}** · strongest random-normalized JS shift: **{int(top_js['Feature id'])}**.  \n"
+        f"Spearman ρ(candidate score, target-specificity ratio): **{fmt_rho(rho_target)}** · "
+        f"ρ(candidate score, JS-specificity ratio): **{fmt_rho(rho_js)}**.  \n\n"
+        "This is stronger than the cheap triage comparison because each candidate is normalized against its own "
+        "norm-matched random ensemble. The candidate count and eight-control ensemble are still small, so the correlations "
+        "and empirical tails are descriptive live diagnostics rather than significance claims."
+    )
+
+    chart = table[
+        ["Feature id", "Candidate score", "Target specificity ratio", "Discovery rank", "Specificity rank", "Target empirical tail p"]
+    ].copy()
+    chart["Feature id"] = chart["Feature id"].astype(str)
+    chart["Series"] = "Controlled candidate"
+    return summary, table, chart
+
+
 def _cue_context_metrics_markdown(result) -> str:
     active = ", ".join(
         f"{cue}: {count}/{len(result.stems)} contexts"
@@ -1349,6 +1561,11 @@ def run_candidate_causal_screen(
         alignment_metrics, alignment_table, alignment_chart = _candidate_alignment_outputs(
             discovery_table, table
         )
+        specificity_shortlist = _controlled_candidate_shortlist(discovery_table, table, limit=3)
+        specificity_update = gr.update(
+            choices=choices,
+            value=specificity_shortlist,
+        )
         return (
             _candidate_screen_metrics_markdown(result),
             table,
@@ -1359,6 +1576,69 @@ def run_candidate_causal_screen(
             alignment_table,
             alignment_chart,
             _tsv(alignment_table),
+            specificity_update,
+        )
+    except Exception as exc:
+        _raise_ui_error(exc)
+
+
+@gpu(duration=40)
+def run_candidate_specificity_screen(
+    prompt: str,
+    layer: int,
+    token_index: int,
+    feature_ids: list[str] | None,
+    target_text: str,
+    discovery_table: pd.DataFrame | None,
+):
+    try:
+        selected = [int(float(value)) for value in (feature_ids or [])]
+        result = RUNTIME.candidate_specificity_screen(
+            text=prompt,
+            layer=int(layer),
+            token_index=int(token_index),
+            feature_ids=selected,
+            target_text=target_text,
+        )
+        columns = [
+            "Rank",
+            "Feature id",
+            "Native activation",
+            "Active at current token",
+            "Perturbation L2",
+            "SAE Δ mean log p/token",
+            "Random signed mean Δ",
+            "Random mean |Δ|",
+            "Random |Δ| std",
+            "Target specificity ratio",
+            "Target empirical tail p",
+            "SAE Δ sequence log p",
+            "SAE next-token JS",
+            "Random mean JS",
+            "Random JS std",
+            "JS specificity ratio",
+            "JS empirical tail p",
+        ]
+        table = pd.DataFrame(result.rows, columns=columns)
+        chart = pd.DataFrame(
+            result.chart_rows,
+            columns=["Feature", "Specificity metric", "Ratio"],
+        )
+        alignment_metrics, alignment_table, alignment_chart = _controlled_alignment_outputs(
+            discovery_table, table
+        )
+        choices = [str(feature_id) for feature_id in result.feature_ids]
+        candidate_update = gr.update(choices=choices, value=choices[0] if choices else None)
+        return (
+            _candidate_specificity_metrics_markdown(result),
+            table,
+            chart,
+            candidate_update,
+            _tsv(table),
+            alignment_metrics,
+            alignment_table,
+            alignment_chart,
+            _tsv(alignment_table),
         )
     except Exception as exc:
         _raise_ui_error(exc)
@@ -1445,7 +1725,7 @@ def set_mode_help(mode: str):
 with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_width=True) as demo:
     gr.HTML(
         '<header class="hero">'
-        '<h1>FeatureLens <span style="font-size:.48em;opacity:.58;font-weight:400">v0.10</span></h1>'
+        '<h1>FeatureLens <span style="font-size:.48em;opacity:.58;font-weight:400">v0.11</span></h1>'
         '<div class="subtitle">Causal Interpretability Workbench</div>'
         '<div class="question">Discover sparse features, test robustness, and separate correlation from causal influence.</div>'
         '</header>'
@@ -2075,6 +2355,108 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench", fill_w
                     height=330,
                 )
 
+        gr.Markdown("#### Controlled candidate specificity")
+        gr.Markdown(
+            "Follow the cheap triage with a small random-controlled comparison. FeatureLens automatically proposes a "
+            "three-feature shortlist that preserves the disagreement between discovery, target-effect, and distribution-shift "
+            "rankings. Each candidate receives its own 8-direction norm-matched random ensemble in one batched forward."
+        )
+        with gr.Row(equal_height=True):
+            candidate_specificity_ids = gr.Dropdown(
+                choices=[],
+                value=[],
+                multiselect=True,
+                allow_custom_value=True,
+                max_choices=3,
+                label="Candidates for controlled comparison",
+                info="Auto-filled after triage; choose up to three features.",
+                scale=3,
+            )
+            candidate_specificity_target = gr.Textbox(
+                label="Controlled target continuation",
+                value="2x",
+                info="Exact continuation scored for every candidate and its random controls.",
+                scale=2,
+            )
+        candidate_specificity_btn = gr.Button(
+            "Run controlled candidate comparison", variant="primary", elem_classes=["action-btn"]
+        )
+        candidate_specificity_metrics = gr.Markdown()
+        with gr.Row(equal_height=False):
+            with gr.Column(scale=3):
+                _table_heading('Controlled candidate specificity')
+                candidate_specificity_table = gr.Dataframe(
+                    interactive=False,
+                    label="Controlled candidate specificity",
+                    show_label=False,
+                    buttons=["fullscreen"],
+                    elem_classes=["result-table"],
+                    wrap=False,
+                    max_height=390,
+                )
+                candidate_specificity_tsv = gr.Textbox(visible="hidden")
+                candidate_specificity_copy = _copy_button()
+            with gr.Column(scale=2):
+                candidate_specificity_plot = gr.BarPlot(
+                    x="Feature",
+                    y="Ratio",
+                    color="Specificity metric",
+                    color_map={
+                        "Target specificity": INK_TEAL,
+                        "JS specificity": INK_UMBER,
+                    },
+                    title="Random-normalized causal specificity",
+                    elem_id="plot-candidate-specificity",
+                    x_title="Feature id",
+                    y_title="SAE effect / random mean effect",
+                    x_label_angle=-35,
+                    buttons=["fullscreen", "export"],
+                    elem_classes=["fl-plot"],
+                    height=330,
+                )
+
+        gr.Markdown("#### Association vs controlled causality")
+        gr.Markdown(
+            "Computed from concept discovery and the random-controlled comparison above. This is the stronger live version "
+            "of the association-vs-causality diagnostic because each candidate is normalized against its own matched-control ensemble.",
+            elem_classes=["candidate-help"],
+        )
+        controlled_alignment_metrics = gr.Markdown()
+        with gr.Row(equal_height=False):
+            with gr.Column(scale=3):
+                _table_heading('Discovery–controlled-causality alignment')
+                controlled_alignment_table = gr.Dataframe(
+                    interactive=False,
+                    label="Discovery–controlled-causality alignment",
+                    show_label=False,
+                    buttons=["fullscreen"],
+                    elem_classes=["result-table"],
+                    wrap=False,
+                    max_height=360,
+                )
+                controlled_alignment_tsv = gr.Textbox(visible="hidden")
+                controlled_alignment_copy = _copy_button()
+            with gr.Column(scale=2):
+                controlled_alignment_plot = gr.ScatterPlot(
+                    x="Candidate score",
+                    y="Target specificity ratio",
+                    color="Series",
+                    color_map={"Controlled candidate": INK_TEAL},
+                    title="Association evidence vs controlled target specificity",
+                    elem_id="plot-controlled-association-causality",
+                    x_title="Discovery candidate score",
+                    y_title="Target specificity ratio",
+                    tooltip=[
+                        "Feature id",
+                        "Discovery rank",
+                        "Specificity rank",
+                        "Target empirical tail p",
+                    ],
+                    buttons=["fullscreen", "export"],
+                    elem_classes=["fl-plot"],
+                    height=330,
+                )
+
         gr.HTML('<div class="section-rule">C. Inspect one feature</div>')
         contrast_location = gr.Markdown(
             "**Activation trace uses the current Workbench prompt.** The controlled concept scan below uses its own balanced prompt set."
@@ -2390,7 +2772,7 @@ Batched experiments include an explicit **zero-edit row**. Causal effects are me
 
 1. SAE reconstruction quality.
 2. Held-out feature/concept prediction.
-3. Concept-guided candidate discovery and batched causal candidate triage.
+3. Concept-guided candidate discovery, batched causal triage, and random-controlled candidate comparison.
 4. Selected-feature concept contrast and token-local activation traces, completion-cue sensitivity, and cue × context specificity.
 5. Local and prompt-wide paraphrase robustness.
 6. Single-feature causal intervention and dose-response.
@@ -2519,8 +2901,37 @@ Association, robustness, geometry, and intervention evidence remain separate cla
             candidate_alignment_table,
             candidate_alignment_plot,
             candidate_alignment_tsv,
+            candidate_specificity_ids,
         ],
     )
+    candidate_specificity_btn.click(
+        run_candidate_specificity_screen,
+        inputs=[
+            prompt,
+            discovery_layer,
+            token_index,
+            candidate_specificity_ids,
+            candidate_specificity_target,
+            discovery_table,
+        ],
+        outputs=[
+            candidate_specificity_metrics,
+            candidate_specificity_table,
+            candidate_specificity_plot,
+            discovery_candidate,
+            candidate_specificity_tsv,
+            controlled_alignment_metrics,
+            controlled_alignment_table,
+            controlled_alignment_plot,
+            controlled_alignment_tsv,
+        ],
+    )
+    candidate_specificity_table.select(
+        select_candidate_row,
+        inputs=[candidate_specificity_table],
+        outputs=[discovery_candidate],
+        queue=False,
+    )
     candidate_screen_table.select(
         select_candidate_row,
         inputs=[candidate_screen_table],
@@ -2576,6 +2987,8 @@ Association, robustness, geometry, and intervention evidence remain separate cla
         (discovery_copy, discovery_tsv),
         (candidate_screen_copy, candidate_screen_tsv),
         (candidate_alignment_copy, candidate_alignment_tsv),
+        (candidate_specificity_copy, candidate_specificity_tsv),
+        (controlled_alignment_copy, controlled_alignment_tsv),
         (cue_copy, cue_tsv),
         (cue_context_copy, cue_context_tsv),
         (para_copy, para_tsv),

docs/METHODOLOGY.md

@@ -462,3 +462,72 @@ so positive values indicate a feature that rises in the target-effect ordering r
 Across the small screened set, FeatureLens computes tie-aware Spearman correlations between candidate score and (a) absolute target effect and (b) next-token JS. These are deliberately labeled **descriptive** because the live shortlist is small and the triage stage does not spend random-control ensembles. A promoted feature still requires the full single-feature causal test before specificity is interpreted.
 
 This synthesis costs no additional model forward: it is computed from the discovery and triage tables already produced by the two-stage workflow.
+
+## v0.11 controlled candidate specificity
+
+The cheap candidate-ablation triage intentionally omits random controls so multiple candidate features can be screened in one small batch. Raw intervention magnitude is useful for triage, but it does not establish that an SAE direction is more behaviorally specific than an arbitrary residual perturbation of the same norm.
+
+v0.11 therefore adds a second-stage controlled screen for at most three candidates. For candidate feature \(i\) with native activation \(z_i\) and decoder direction \(d_i\), the targeted ablation is
+
+\[
+\Delta h_i = -z_i d_i.
+\]
+
+For each candidate, FeatureLens generates `live_random_controls` deterministic random residual directions \(r_{ij}\) such that
+
+\[
+\|r_{ij}\|_2 = \|\Delta h_i\|_2.
+\]
+
+A single batched execution contains one zero-edit reference, every targeted SAE ablation, and every candidate-specific random control. All target and JS effects are therefore measured relative to the same batched null.
+
+Two random-normalized causal quantities are kept separate:
+
+1. **Target specificity ratio**
+
+\[
+\frac{|\Delta \bar{\ell}_{\text{SAE}}|}
+     {\operatorname{mean}_j |\Delta \bar{\ell}_{r_j}|},
+\]
+
+where \(\bar{\ell}\) is the teacher-forced mean log probability per target token.
+
+2. **JS specificity ratio**
+
+\[
+\frac{\operatorname{JS}(p_0, p_{\text{SAE}})}
+     {\operatorname{mean}_j \operatorname{JS}(p_0, p_{r_j})}.
+\]
+
+The first asks whether the SAE edit is unusually influential for the specified continuation. The second asks whether it is unusually disruptive to the local next-token distribution as a whole. They are intentionally not collapsed into a single score.
+
+The live empirical random-control tail uses the finite-ensemble correction
+
+\[
+p = \frac{1 + \#\{|e_{r_j}| \ge |e_{\text{SAE}}|\}}
+         {1 + N_{\text{controls}}}.
+\]
+
+With eight controls, the smallest possible live value is \(1/9\approx0.111\). These values are therefore coarse specificity diagnostics, not conventional significance tests.
+
+### Strategic controlled shortlist
+
+After cheap triage, the default controlled shortlist is chosen to preserve disagreement rather than merely retest the top target-effect rows:
+
+1. strongest discovery candidate among the screened features;
+2. strongest raw target-effect candidate;
+3. strongest next-token-JS candidate if distinct;
+4. fill any remaining slot by target-effect rank.
+
+This makes the controlled follow-up directly test the project's central question: whether strong concept-association evidence predicts random-normalized causal specificity.
+
+### Association vs controlled causality
+
+The zero-GPU synthesis layer joins discovery evidence to the controlled table and keeps four ranks separate:
+
+- discovery rank;
+- raw target-effect rank;
+- random-normalized target-specificity rank;
+- random-normalized JS-specificity rank.
+
+Descriptive Spearman correlations between candidate score and the two specificity ratios are shown only as small-sample diagnostics. The full offline benchmark remains the place for larger candidate sets, more random controls, uncertainty intervals, and formal held-out conclusions.

docs/VALIDATION.md

@@ -1,12 +1,10 @@
-# FeatureLens v0.10 validation
+# FeatureLens v0.11 validation
 
-v0.10 does **not** change paraphrase robustness, layer trajectory, feature-set interventions, dose-response inference, cue × context, or the in-place focus implementation. Do not spend ZeroGPU quota rerunning those paths.
+v0.11 adds one new GPU inference path: **Controlled candidate specificity**. It does **not** change concept discovery, cheap candidate triage, paraphrase robustness, layer trajectory, feature-set interventions, dose response, cue diagnostics, or the validated in-place focus behavior. Do not spend ZeroGPU quota rerunning those unchanged paths.
 
-The only new live behavior is the **association-to-causality synthesis** computed after the existing discovery + candidate-triage workflow.
+## Local release gate
 
-## 1. Local release gate — no HF GPU
-
-From the repository root:
+Run from the repository root:
 
 ```bash
 python3 -m pytest -q && \
@@ -16,139 +14,122 @@ python3 scripts/ui_smoke.py && \
 python3 scripts/release_check.py
 ```
 
-Expected high-level result:
-
-- all tests pass;
-- Ruff reports `All checks passed!`;
-- `FeatureLens UI launch smoke: PASS`;
-- release check ends with `release: v0.10.0`.
-
-Stop if any local gate fails.
-
----
-
-# HF acceptance — two GPU calls total
-
-Use the existing Workbench context:
+Expected release checker tail:
 
-- **Workbench → Prompt:** `The derivative of x squared is`
-- **Workbench → Residual layer:** `14`
-- **Workbench → Prompt token index:** `-1`
-
-You do **not** need to click **Inspect sparse features** first.
-
-## 2. GPU call 1 — causal-ready mathematics discovery
-
-Exact path:
-
-**Feature evidence → A. Concept-guided candidate discovery**
+```text
+FeatureLens release check: PASS
+  discovery prompts: 224
+  causal tasks: 28
+  layers: [4, 14, 26]
+  feature-set sizes: [1, 3, 5]
+  random controls: 8
+  release: v0.11.0
+```
 
-Set exactly:
+## HF acceptance: one GPU call only
 
-- **Target concept:** `mathematics`
-- **Residual layer:** `14`
-- **Prompts per concept:** `4`
-- **Candidate features:** `12`
-- **Candidate ranking:** `Causal-ready at current token`
+The v0.10 discovery and triage paths were already validated. For v0.11, test only the new controlled follow-up.
 
-Click:
+### Exact path
 
-**Discover concept-associated candidates**
+Open **Feature evidence** and scroll to:
 
-Expected regression behavior from v0.9:
+**B. Batched causal candidate triage → Controlled candidate specificity**
 
-- the summary should say `12/12` displayed candidates are active at the selected Workbench token;
-- **Candidate features to screen** should auto-populate with the first five returned feature IDs;
-- no additional UI/focus regression testing is required because that code was not changed in v0.10.
+Use the current Workbench context:
 
-For the current canonical prompt, the v0.9 result began with candidates `16369`, `5712`, `26112`, `25992`, `21670`. Exact floating-point values can vary slightly, but a major ordering change should be reported.
+```text
+Prompt: The derivative of x squared is
+Residual layer: 14
+Prompt token index: -1
+```
 
-## 3. GPU call 2 — triage + association-to-causality synthesis
+In **Candidates for controlled comparison**, manually enter/select exactly:
 
-Exact path:
+```text
+16369
+25992
+21670
+```
 
-**Feature evidence → B. Batched causal candidate triage**
+These are the three strategic candidates established by the validated v0.10 workflow:
 
-Leave the five auto-populated candidates selected.
+- `16369`: discovery leader and next-token-JS leader;
+- `25992`: raw target-effect leader;
+- `21670`: second-largest raw target effect.
 
 Set:
 
-- **Screen target continuation:** `2x`
-
-Click:
-
-**Screen candidate ablations**
-
-The normal **Candidate ablation screen** should appear first.
-
-Immediately below it, without another GPU action, v0.10 should populate:
-
-### `Association vs causal influence`
-
-with:
+```text
+Controlled target continuation: 2x
+```
 
-1. a descriptive summary;
-2. **Discovery–causality alignment** table;
-3. **Association evidence vs target effect** scatter plot.
+Click **Run controlled candidate comparison**.
 
-### Required alignment-table columns
+### Pass conditions
 
-- `Feature id`
-- `Discovery rank`
-- `Target-effect rank`
-- `Distribution-shift rank`
-- `Candidate score`
-- `Selectivity`
-- `Current token activation`
-- `|Δ mean log p/token|`
-- `Next-token JS`
-- `Discovery→target rank shift`
+The run should produce **Controlled candidate specificity** with exactly three feature rows. For every row verify:
 
-### Required summary behavior
+- `Active at current token` is `True`;
+- `Target specificity ratio` is finite and non-negative;
+- `Target empirical tail p` lies in `(0, 1]`;
+- `JS specificity ratio` is finite and non-negative;
+- `JS empirical tail p` lies in `(0, 1]`;
+- random-control means/stds are populated;
+- the summary says each candidate used 8 norm-matched random controls;
+- execution-context null drift is reported separately from the causal effect.
 
-The summary must identify separately:
+The plot **Random-normalized causal specificity** should show two series per feature:
 
-- top discovery candidate;
-- strongest target-effect candidate;
-- strongest next-token distribution-shift candidate;
-- Spearman `ρ(candidate score, |target effect|)`;
-- Spearman `ρ(candidate score, next-token JS)`;
-- an explicit warning that the live correlations are descriptive because the screened set is small and triage has no random-control ensemble.
+```text
+Target specificity
+JS specificity
+```
 
-For the exact v0.9 values you reported, the expected qualitative pattern is:
+Do not assume which feature must win. That is the empirical result this version is intended to measure.
 
-- discovery rank #1: feature `16369`;
-- strongest target effect: feature `25992`;
-- strongest next-token JS shift: feature `16369`;
-- discovery score versus target-effect magnitude: strongly negative descriptive rank correlation;
-- discovery score versus JS: positive but weaker descriptive rank correlation.
+## Optional integration check — no additional GPU call
 
-Do not require exact decimals as a pass condition.
+If the Space still has the v0.10 discovery table populated in the same browser session, the **Association vs controlled causality** section should also populate automatically.
 
-### Rank-shift sanity check
+Verify that **Discovery–controlled-causality alignment** contains the same controlled features and includes:
 
-For the v0.9 ordering:
+- discovery rank;
+- specificity rank;
+- target-effect rank;
+- JS-specificity rank;
+- target specificity ratio;
+- target empirical tail p;
+- JS specificity ratio;
+- JS empirical tail p;
+- discovery→specificity rank shift.
 
-- feature `25992`: discovery rank `4`, target-effect rank `1` → `Discovery→target rank shift = +3`;
-- feature `16369`: discovery rank `1`, target-effect rank `5` → rank shift `-4`.
+The summary may include descriptive Spearman correlations. With only three controlled candidates, these are diagnostic summaries, **not significance claims**.
 
-This is the most important v0.10 regression check because it demonstrates that concept-evidence rank and target-causal rank are not interchangeable.
+If the discovery table is not populated because the Space was freshly rebuilt, do **not** spend two extra GPU calls merely to repopulate it for v0.11 acceptance. The zero-GPU join logic is covered by automated tests.
 
----
+## Do not rerun for v0.11
 
-## 4. What to send back
+Do not rerun these paths unless you independently notice a regression:
 
-Only send:
+- Concept-guided candidate discovery;
+- Batched causal candidate triage;
+- Scale dose-response;
+- Single-feature causal test;
+- Paraphrase identity control;
+- Layer trajectory;
+- 1/3/5 feature-set sweep;
+- cue sensitivity or cue × context;
+- plot/table focus behavior.
 
-1. the v0.10 **Candidate ablation screen** if it changed materially from v0.9;
-2. the new **Discovery–causality alignment** table;
-3. the new association/causality summary with the two Spearman values;
-4. optionally a screenshot of **Association evidence vs target effect** if the plot looks wrong.
+Those implementations were not changed in v0.11.
 
-Do **not** rerun identity paraphrase, layer trajectory, 1/3/5 sweep, dose response, cue × context, or focus behavior for v0.10. Those implementations were not changed.
+## What to send back
 
----
+For the new controlled comparison, send:
 
-# Final hardening later
+1. the three feature rows from **Controlled candidate specificity**;
+2. the summary above the table;
+3. optionally the **Association vs controlled causality** summary/table if it populated without extra GPU work.
 
-The broad adversarial/release suite remains deferred until the live feature set is frozen. After v0.10 acceptance, the next high-value step should be the real offline held-out benchmark rather than another round of unrelated live widgets.
+That is sufficient to decide whether the live feature set should now be frozen and the project moved to the full offline benchmark.

featurelens/runtime.py

@@ -255,6 +255,19 @@ class CandidateCausalScreenResult:
     execution_drift_js: float
 
 
+@dataclass
+class CandidateSpecificityResult:
+    feature_ids: list[int]
+    target_tokens: list[str]
+    rows: list[list[object]]
+    chart_rows: list[list[object]]
+    active_feature_count: int
+    candidate_count: int
+    random_control_count: int
+    execution_drift_mean_logprob: float
+    execution_drift_js: float
+
+
 @dataclass
 class FeatureCueScanResult:
     feature_id: int
@@ -2131,6 +2144,206 @@ class FeatureLensRuntime:
             execution_drift_js=float(execution_drift_js),
         )
 
+    @torch.inference_mode()
+    def candidate_specificity_screen(
+        self,
+        text: str,
+        layer: int,
+        token_index: int,
+        feature_ids: Sequence[int],
+        target_text: str,
+    ) -> CandidateSpecificityResult:
+        """Compare a small candidate set against per-feature norm-matched random ensembles.
+
+        This is the controlled follow-up to :meth:`candidate_causal_screen`. Each SAE
+        ablation gets its own deterministic random-control ensemble with the same L2 norm.
+        All targeted and random conditions share one batched zero-edit reference so the
+        comparison is both GPU-efficient and execution-context consistent.
+        """
+        if not text.strip():
+            raise ValueError('Enter and inspect a Workbench prompt first.')
+        if not target_text.strip():
+            raise ValueError('Enter a target continuation for controlled candidate comparison.')
+        ids = list(dict.fromkeys(int(x) for x in feature_ids))
+        if not ids:
+            raise ValueError('Select at least one candidate feature for controlled comparison.')
+        if len(ids) > 3:
+            raise ValueError('Controlled candidate comparison supports at most three features per run.')
+        if any(feature_id < 0 or feature_id >= self.settings.sae_width for feature_id in ids):
+            raise ValueError(f'Feature ids must be in [0, {self.settings.sae_width - 1}].')
+
+        self.ensure_ready(preload_saes=False)
+        assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
+        prompt_inputs = self._inputs(text)
+        prompt_len = int(prompt_inputs['input_ids'].shape[1])
+        idx = self._resolve_index(int(token_index), prompt_len)
+        target_ids = self._target_ids(target_text)
+        full_inputs = self._append_target(prompt_inputs, target_ids)
+        sae = self.sae_store.get(int(layer))
+
+        capture: dict = {}
+        with self._capture_hook(int(layer), capture):
+            single_baseline_out = self.model(**full_inputs, use_cache=False)
+        single_logits = single_baseline_out.logits[0]
+        _, single_mean, _ = sequence_logprob_summary(
+            single_logits, prompt_length=prompt_len, target_ids=target_ids
+        )
+
+        residual = capture['hidden'][0, idx]
+        encoding = sae.encode(residual)
+        activations = [float(encoding.activation_for(feature_id)) for feature_id in ids]
+
+        zero = torch.zeros_like(residual)
+        all_deltas: list[torch.Tensor] = [zero]
+        metadata: list[tuple[int, str, float, float]] = []
+        for feature_id, activation in zip(ids, activations, strict=True):
+            delta = residual_delta(
+                sae.decoder_direction(feature_id),
+                activation,
+                InterventionSpec('ablate', 0.0),
+            )
+            norm = float(torch.linalg.vector_norm(delta.float()).item())
+            all_deltas.append(delta)
+            metadata.append((feature_id, 'sae', activation, norm))
+            seed = self._control_seed(text, int(layer), str(feature_id), 'candidate_specificity', 0.0)
+            controls = self._random_controls(
+                delta, seed=seed, count=self.settings.live_random_controls
+            )
+            for control_idx, control in enumerate(controls):
+                all_deltas.append(control)
+                metadata.append((feature_id, f'random_{control_idx}', activation, norm))
+
+        delta_batch = torch.stack(all_deltas, dim=0)
+        repeated = self._repeat_inputs(full_inputs, delta_batch.shape[0])
+        with self._batch_delta_hook(int(layer), idx, delta_batch):
+            outputs = self.model(**repeated, use_cache=False)
+
+        baseline_logits = outputs.logits[0]
+        baseline_seq, baseline_mean, _ = sequence_logprob_summary(
+            baseline_logits, prompt_length=prompt_len, target_ids=target_ids
+        )
+        baseline_next = baseline_logits[prompt_len - 1]
+        execution_drift_mean = float(baseline_mean - single_mean)
+        execution_drift_js = js_divergence_from_logits(
+            single_logits[prompt_len - 1], baseline_next
+        )
+
+        grouped: dict[int, dict[str, object]] = {
+            feature_id: {
+                'activation': activation,
+                'norm': 0.0,
+                'sae': None,
+                'random': [],
+            }
+            for feature_id, activation in zip(ids, activations, strict=True)
+        }
+
+        for output_idx, meta in enumerate(metadata, start=1):
+            feature_id, kind, activation, norm = meta
+            logits = outputs.logits[output_idx]
+            seq_logp, mean_logp, _ = sequence_logprob_summary(
+                logits, prompt_length=prompt_len, target_ids=target_ids
+            )
+            item = {
+                'seq_delta': float(seq_logp - baseline_seq),
+                'mean_delta': float(mean_logp - baseline_mean),
+                'js': float(js_divergence_from_logits(baseline_next, logits[prompt_len - 1])),
+            }
+            grouped[feature_id]['activation'] = float(activation)
+            grouped[feature_id]['norm'] = float(norm)
+            if kind == 'sae':
+                grouped[feature_id]['sae'] = item
+            else:
+                random_items = grouped[feature_id]['random']
+                assert isinstance(random_items, list)
+                random_items.append(item)
+
+        scored: list[dict[str, object]] = []
+        for feature_id in ids:
+            group = grouped[feature_id]
+            sae_item = group['sae']
+            random_items = group['random']
+            assert isinstance(sae_item, dict)
+            assert isinstance(random_items, list) and random_items
+            mean_delta = float(sae_item['mean_delta'])
+            js = float(sae_item['js'])
+            random_mean_deltas = [float(item['mean_delta']) for item in random_items]
+            random_js_values = [float(item['js']) for item in random_items]
+            random_signed, random_abs, random_std, target_p = self._random_effect_summary(
+                random_mean_deltas, mean_delta
+            )
+            _, random_js_abs, random_js_std, js_p = self._random_effect_summary(
+                random_js_values, js
+            )
+            scored.append(
+                {
+                    'feature_id': int(feature_id),
+                    'activation': float(group['activation']),
+                    'active': bool(float(group['activation']) > 0),
+                    'norm': float(group['norm']),
+                    'mean_delta': mean_delta,
+                    'seq_delta': float(sae_item['seq_delta']),
+                    'random_signed': float(random_signed),
+                    'random_abs': float(random_abs),
+                    'random_std': float(random_std),
+                    'target_ratio': float(abs(mean_delta) / max(random_abs, 1e-12)),
+                    'target_p': float(target_p),
+                    'js': js,
+                    'random_js': float(random_js_abs),
+                    'random_js_std': float(random_js_std),
+                    'js_ratio': float(js / max(random_js_abs, 1e-12)),
+                    'js_p': float(js_p),
+                }
+            )
+
+        scored.sort(
+            key=lambda item: (float(item['target_ratio']), abs(float(item['mean_delta']))),
+            reverse=True,
+        )
+        rows: list[list[object]] = []
+        chart_rows: list[list[object]] = []
+        for rank, item in enumerate(scored, start=1):
+            feature_id = int(item['feature_id'])
+            rows.append(
+                [
+                    rank,
+                    feature_id,
+                    float(item['activation']),
+                    bool(item['active']),
+                    float(item['norm']),
+                    float(item['mean_delta']),
+                    float(item['random_signed']),
+                    float(item['random_abs']),
+                    float(item['random_std']),
+                    float(item['target_ratio']),
+                    float(item['target_p']),
+                    float(item['seq_delta']),
+                    float(item['js']),
+                    float(item['random_js']),
+                    float(item['random_js_std']),
+                    float(item['js_ratio']),
+                    float(item['js_p']),
+                ]
+            )
+            chart_rows.extend(
+                [
+                    [str(feature_id), 'Target specificity', float(item['target_ratio'])],
+                    [str(feature_id), 'JS specificity', float(item['js_ratio'])],
+                ]
+            )
+
+        return CandidateSpecificityResult(
+            feature_ids=[int(item['feature_id']) for item in scored],
+            target_tokens=[self.tokenizer.decode([int(token_id)]) for token_id in target_ids],
+            rows=rows,
+            chart_rows=chart_rows,
+            active_feature_count=sum(bool(item['active']) for item in scored),
+            candidate_count=len(scored),
+            random_control_count=self.settings.live_random_controls,
+            execution_drift_mean_logprob=execution_drift_mean,
+            execution_drift_js=float(execution_drift_js),
+        )
+
     @staticmethod
     def _cue_prompt(stem: str, cue: str) -> str:
         stem = stem.rstrip()

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "featurelens"
-version = "0.10.0"
+version = "0.11.0"
 description = "Causal sparse-feature interpretability workbench for Qwen3 and Qwen-Scope SAEs"
 requires-python = ">=3.10"
 

research_config.json

@@ -90,7 +90,7 @@
     "contrastive_continuation_preference_test",
     "feature_decoder_geometry"
   ],
-  "concept_candidate_discovery_metric": "balanced exploratory score = selectivity × target activation rate × log1p(target mean); causal-ready mode additionally requires current-token activity and log-scales that activation; raw mean-difference remains available as a scale-sensitive comparison",
+  "concept_candidate_discovery_metric": "balanced exploratory score = selectivity \u00d7 target activation rate \u00d7 log1p(target mean); causal-ready mode additionally requires current-token activity and log-scales that activation; raw mean-difference remains available as a scale-sensitive comparison",
   "completion_cue_scan": "final-token feature activation after controlled suffix/cue substitution",
   "live_features_v0_6": [
     "start_here_plain_language_onboarding",
@@ -139,5 +139,20 @@
     "descriptive_spearman_concordance_summary",
     "target_effect_vs_distribution_shift_rank_separation",
     "no_extra_gpu_candidate_synthesis"
+  ],
+  "candidate_specificity_limit": 3,
+  "candidate_specificity_control": "each candidate gets live_random_controls deterministic norm-matched residual directions in one shared batched zero-edit execution context",
+  "candidate_specificity_metrics": [
+    "target effect versus candidate-specific random mean absolute effect",
+    "next-token JS versus candidate-specific random mean JS",
+    "coarse empirical random-control tail probabilities",
+    "discovery rank versus random-normalized target-specificity rank"
+  ],
+  "live_features_v0_11": [
+    "controlled_multi_candidate_random_specificity_screen",
+    "strategic_discovery_target_js_shortlist",
+    "association_vs_controlled_causality_alignment",
+    "target_specificity_vs_js_specificity_separation",
+    "single_new_gpu_call_hf_acceptance"
   ]
 }

scripts/release_check.py

@@ -84,6 +84,7 @@ def check_config(config: dict) -> None:
         'live_geometry_feature_limit': 8,
         'concept_contrast_pooling': 'max activation across non-padding prompt tokens',
         'candidate_causal_screen_limit': 8,
+        'candidate_specificity_limit': 3,
     }
     for key, value in expected.items():
         if config.get(key) != value:
@@ -179,6 +180,32 @@ def check_config(config: dict) -> None:
             'research_config.json live_features_v0_9 mismatch: ' f'{sorted(actual_live_v09)}'
         )
 
+    required_live_v10 = {
+        'discovery_to_causality_alignment_table',
+        'association_evidence_vs_target_effect_scatter',
+        'descriptive_spearman_concordance_summary',
+        'target_effect_vs_distribution_shift_rank_separation',
+        'no_extra_gpu_candidate_synthesis',
+    }
+    actual_live_v10 = set(config.get('live_features_v0_10', []))
+    if actual_live_v10 != required_live_v10:
+        raise SystemExit(
+            'research_config.json live_features_v0_10 mismatch: ' f'{sorted(actual_live_v10)}'
+        )
+
+    required_live_v11 = {
+        'controlled_multi_candidate_random_specificity_screen',
+        'strategic_discovery_target_js_shortlist',
+        'association_vs_controlled_causality_alignment',
+        'target_specificity_vs_js_specificity_separation',
+        'single_new_gpu_call_hf_acceptance',
+    }
+    actual_live_v11 = set(config.get('live_features_v0_11', []))
+    if actual_live_v11 != required_live_v11:
+        raise SystemExit(
+            'research_config.json live_features_v0_11 mismatch: ' f'{sorted(actual_live_v11)}'
+        )
+
     if 'german_language' not in config.get('concepts', []) or 'french_language' in config.get('concepts', []):
         raise SystemExit('research_config.json must use german_language and must not contain french_language.')
 
@@ -260,16 +287,21 @@ def check_readme() -> None:
         'spearman',
         'rank-shift',
         'no additional gpu',
+        'controlled candidate specificity',
+        'norm-matched random ensemble',
+        'association vs controlled causality',
+        'target-specificity ratio',
+        'js-specificity ratio',
     ]
     missing = [value for value in required_strings if value.lower() not in readme.lower()]
     if missing:
-        raise SystemExit(f'README.md is missing required v0.10 content: {missing}')
+        raise SystemExit(f'README.md is missing required v0.11 content: {missing}')
 
 
 def check_pyproject() -> None:
     text = (ROOT / 'pyproject.toml').read_text(encoding='utf-8')
-    if 'version = "0.10.0"' not in text:
-        raise SystemExit('pyproject.toml must declare version 0.10.0.')
+    if 'version = "0.11.0"' not in text:
+        raise SystemExit('pyproject.toml must declare version 0.11.0.')
 
 
 def main() -> None:
@@ -287,7 +319,7 @@ def main() -> None:
     print(f'  layers: {config["layers"]}')
     print(f'  feature-set sizes: {config["feature_set_sizes"]}')
     print(f'  random controls: {config["live_random_controls"]}')
-    print('  release: v0.10.0')
+    print('  release: v0.11.0')
 
 
 if __name__ == '__main__':

tests/test_live_runtime_helpers.py

@@ -342,3 +342,24 @@ def test_candidate_causal_screen_batches_multiple_ablation_candidates() -> None:
     assert 0 <= result.active_feature_count <= result.candidate_count
     assert all(math.isfinite(float(row[5])) for row in result.rows)
     assert all(float(row[7]) >= 0 for row in result.rows)
+
+
+def test_candidate_specificity_screen_batches_random_controlled_candidates() -> None:
+    runtime = make_runtime()
+    result = runtime.candidate_specificity_screen(
+        text='abc',
+        layer=0,
+        token_index=-1,
+        feature_ids=[0, 1],
+        target_text='d',
+    )
+    assert result.candidate_count == 2
+    assert result.random_control_count == 3
+    assert len(result.rows) == 2
+    assert len(result.chart_rows) == 4
+    assert all(len(row) == 17 for row in result.rows)
+    assert all(row[0] == rank for rank, row in enumerate(result.rows, start=1))
+    assert all(float(row[9]) >= 0 for row in result.rows)  # target specificity
+    assert all(0 < float(row[10]) <= 1 for row in result.rows)  # empirical tail
+    assert all(float(row[15]) >= 0 for row in result.rows)  # JS specificity
+    assert all(0 < float(row[16]) <= 1 for row in result.rows)

tests/test_ui_helpers.py

@@ -181,3 +181,78 @@ def test_candidate_alignment_ui_uses_same_screen_call_and_no_new_gpu_button() ->
     assert app.candidate_alignment_table.show_label is False
     assert app.candidate_alignment_plot.visible is True
     assert 'Association evidence vs target effect' == app.candidate_alignment_plot.title
+
+
+def _v10_discovery_table() -> pd.DataFrame:
+    return pd.DataFrame(
+        [
+            [1, 16369, 4.7348, 15.0, 0.0, 15.0, 1.0, 0.5, 0.0, 50.0, 29.25, True],
+            [2, 5712, 4.1969, 9.1, 1.5, 7.6, 0.72, 1.0, 0.17, 11.4, 11.38, True],
+            [3, 26112, 3.9419, 10.8, 0.0, 10.8, 1.0, 0.5, 0.0, 23.4, 23.42, True],
+            [4, 25992, 3.7342, 10.1, 0.0, 10.1, 1.0, 0.5, 0.0, 21.4, 21.36, True],
+            [5, 21670, 3.6200, 18.9, 2.0, 16.9, 0.81, 0.75, 0.125, 25.3, 6.36, True],
+        ],
+        columns=[
+            'Rank', 'Feature id', 'Candidate score', 'Target mean max', 'Other mean max',
+            'Mean difference', 'Selectivity', 'Target activation rate', 'Other activation rate',
+            'Current prompt max', 'Current token activation', 'Active at current token',
+        ],
+    )
+
+
+def _v10_screen_table() -> pd.DataFrame:
+    return pd.DataFrame(
+        [
+            [1, 25992, 21.3, True, 21.3, -0.1119, -0.2238, 0.001066],
+            [2, 21670, 6.36, True, 6.36, 0.0313, 0.0626, 0.000106],
+            [3, 26112, 23.4, True, 23.4, -0.0211, -0.0423, 0.000417],
+            [4, 5712, 11.38, True, 11.38, 0.0110, 0.0221, 0.000155],
+            [5, 16369, 29.23, True, 29.23, -0.0101, -0.0201, 0.001714],
+        ],
+        columns=[
+            'Rank', 'Feature id', 'Native activation', 'Active at current token', 'Perturbation L2',
+            'Δ mean log p/token', 'Δ sequence log p', 'Next-token JS',
+        ],
+    )
+
+
+def test_controlled_candidate_shortlist_preserves_discovery_and_causal_leaders() -> None:
+    app = _import_app()
+    selected = app._controlled_candidate_shortlist(_v10_discovery_table(), _v10_screen_table(), limit=3)
+    assert selected == ['16369', '25992', '21670']
+
+
+def test_controlled_alignment_uses_random_normalized_specificity() -> None:
+    app = _import_app()
+    controlled = pd.DataFrame(
+        [
+            [1, 25992, 21.3, True, 21.3, -0.1119, -0.01, 0.04, 0.02, 2.80, 0.111, -0.2238, 0.001066, 0.00040, 0.0001, 2.665, 0.111],
+            [2, 16369, 29.2, True, 29.2, -0.0101, 0.00, 0.05, 0.03, 0.20, 0.778, -0.0201, 0.001714, 0.00050, 0.0002, 3.428, 0.111],
+            [3, 21670, 6.36, True, 6.36, 0.0313, 0.00, 0.03, 0.01, 1.04, 0.444, 0.0626, 0.000106, 0.00020, 0.0001, 0.53, 0.778],
+        ],
+        columns=[
+            'Rank', 'Feature id', 'Native activation', 'Active at current token', 'Perturbation L2',
+            'SAE Δ mean log p/token', 'Random signed mean Δ', 'Random mean |Δ|', 'Random |Δ| std',
+            'Target specificity ratio', 'Target empirical tail p', 'SAE Δ sequence log p',
+            'SAE next-token JS', 'Random mean JS', 'Random JS std', 'JS specificity ratio',
+            'JS empirical tail p',
+        ],
+    )
+    summary, table, chart = app._controlled_alignment_outputs(_v10_discovery_table(), controlled)
+    assert len(table) == 3
+    assert int(table.loc[table['Specificity rank'].idxmin(), 'Feature id']) == 25992
+    row_25992 = table.loc[table['Feature id'] == 25992].iloc[0]
+    row_16369 = table.loc[table['Feature id'] == 16369].iloc[0]
+    assert int(row_25992['Discovery→specificity rank shift']) == 3
+    assert int(row_16369['Discovery→specificity rank shift']) == -1
+    assert 'norm-matched random ensemble' in summary
+    assert set(chart['Feature id']) == {'25992', '16369', '21670'}
+
+
+def test_controlled_candidate_ui_limits_live_comparison_to_three_features() -> None:
+    app = _import_app()
+    assert app.candidate_specificity_ids.multiselect is True
+    assert app.candidate_specificity_ids.max_choices == 3
+    assert app.candidate_specificity_target.value == '2x'
+    assert app.candidate_specificity_table.show_label is False
+    assert app.controlled_alignment_table.show_label is False