Release FeatureLens v0.3.0

.github/workflows/ci.yml

@@ -17,7 +17,7 @@ jobs:
           python -m pip install --upgrade pip
           pip install "torch>=2.8,<2.12" "huggingface_hub>=0.34,<2" "numpy>=2,<3" "pytest>=8.3,<10" "ruff>=0.9,<1"
       - name: Ruff
-        run: python -m ruff check featurelens tests scripts experiments/split.py experiments/build_dataset.py
+        run: python -m ruff check app.py featurelens experiments tests scripts
       - name: Unit tests
         run: python -m pytest -q
       - name: Compile

CHANGELOG.md

@@ -1,26 +1,33 @@
 # Changelog
 
-## 0.2.0 — 2026-08-13
+## v0.3.0
 
-### Added
+### Causal measurement
+- Replaced first-token-only target evaluation with exact full-continuation teacher-forced log-probability scoring.
+- Added total sequence and mean-per-token log-probability deltas plus per-target-token decomposition.
+- Retained next-token probability/JS diagnostics and greedy generation as complementary outputs.
 
-- Live norm-matched random residual control for causal interventions.
-- Target-token and JS-divergence causal specificity ratios.
-- Feature-scale dose-response sweep (0× through 3×).
-- Cross-layer representation trajectory for layers 4, 14 and 26.
-- Native Gradio activation, dose-response and trajectory plots.
-- Bootstrap 95% confidence intervals in the generated offline report.
-- Paired sign-flip randomization test for SAE-vs-random causal-effect differences.
-- Hosted/local validation matrix.
+### Distributed feature causality
+- Added joint multi-feature ablation/scaling using summed reconstruction-preserving SAE decoder deltas.
+- Added live top-1/top-3/top-5 joint-ablation sweep with norm-matched random controls.
+- Added offline `experiments/run_feature_sets.py` and report integration.
 
-### Changed
+### Robustness
+- Added a live paraphrase-robustness explorer with TopK Jaccard, sparse cosine, overlap table, and activation comparison.
 
-- Space SDK and dependency pin to Gradio 6.24.0.
-- Explicit client-side rendering (`ssr_mode=False`) to avoid the observed SSR async-user warning.
-- Shorter ZeroGPU callback durations and a smaller default generation budget.
-- More explicit inactive-feature warning and error surfacing.
-- Dark/light-mode-safe UI styling using Gradio theme variables.
+### Efficiency
+- Batched all six single-feature dose-response edits into one model forward after the baseline.
+- Batched targeted/random 1/3/5 feature-set sweep conditions into one model forward after the baseline.
 
-### Removed
+### UI / deployment
+- Replaced the bright blue visual emphasis with muted teal/stone accents and explicit chart palettes.
+- Added visible `Prompt tokens` headings and aligned validation terminology with actual UI labels.
+- Explicitly labels the dose-response panel as a scale intervention: 0× = ablation, 1× = no edit.
+- Kept SSR disabled for the Hugging Face Space.
 
-- Build/cache directories from release artifacts.
+## v0.2.0
+- Added live norm-matched random controls.
+- Added single-feature causal dose-response.
+- Added layer trajectory diagnostics.
+- Added bootstrap confidence intervals and paired sign-flip tests.
+- Hardened Gradio / ZeroGPU deployment.

README.md

@@ -1,8 +1,8 @@
 ---
 title: FeatureLens
 emoji: 🔬
-colorFrom: blue
-colorTo: indigo
+colorFrom: gray
+colorTo: green
 sdk: gradio
 python_version: "3.12.12"
 sdk_version: "6.24.0"
@@ -13,49 +13,68 @@ license: mit
 
 # FeatureLens — Causal Interpretability Workbench
 
-> **v0.2:** live norm-matched controls, causal dose-response sweeps, layer trajectories, statistical uncertainty, and a hardened ZeroGPU deployment.
+> **v0.3:** full-continuation causal scoring, joint sparse feature-set interventions, batched 1/3/5-feature ablation sweeps, paraphrase-robustness inspection, and a calmer low-saturation UI.
 
 **FeatureLens asks one concrete question:**
 
 > Do sparse features that predict a concept also causally influence model behaviour?
 
-It uses **Qwen3-1.7B-Base** with the official **Qwen-Scope residual-stream sparse autoencoders (SAEs)**. The live Hugging Face Space inspects feature activations and performs causal residual-stream interventions; the offline pipeline measures whether predictive features remain predictive on held-out paraphrases and whether manipulating them actually changes downstream behavior.
+It uses **Qwen3-1.7B-Base** with the official **Qwen-Scope residual-stream sparse autoencoders (SAEs)**. The live Hugging Face Space inspects sparse residual features and performs controlled interventions; the offline pipeline measures held-out predictiveness, paraphrase stability, single-feature causality, and whether causal influence is distributed across small sparse feature sets.
 
-This is deliberately **not** a clone of an SAE viewer and does not depend on thesis code or thesis results.
+FeatureLens is deliberately **not** an SAE viewer clone and is independent of thesis code and thesis datasets.
 
-## What is different from a feature viewer?
+## Evidence ladder
 
-FeatureLens separates three claims that are often blurred together:
+FeatureLens keeps several claims separate:
 
-1. **Representation:** does an SAE reconstruct the residual stream reasonably well?
-2. **Prediction:** does a feature distinguish a controlled concept on held-out prompts?
-3. **Causation:** does changing that feature coefficient alter downstream token probabilities or generation more than a norm-matched random perturbation?
+1. **Reconstruction** — does the SAE represent the residual stream reasonably well?
+2. **Prediction** — does a feature distinguish a controlled concept on held-out paraphrase groups?
+3. **Robustness** — does a sparse representation survive a paraphrase?
+4. **Single-feature causality** — does changing one feature alter downstream behaviour?
+5. **Feature-set causality** — does jointly editing a sparse subspace reveal distributed influence?
+6. **Specificity** — is the effect larger than a norm-matched random residual perturbation?
+7. **Dose-response** — does the behavioural effect change coherently as a feature coefficient is varied?
 
-A feature can score highly on (2) and weakly on (3). FeatureLens treats that as a valid result rather than a failure.
+A feature can be predictive but weakly causal. A feature set can also fail to outperform a matched random perturbation. Both are valid experimental outcomes.
 
 ## Live workbench
 
-The Gradio app supports:
+The Gradio Space supports:
 
-- prompt input and token-level inspection;
-- early / middle / late residual layers: **4, 14, 26**;
-- strongest TopK SAE features at a selected token;
-- reconstruction cosine and normalized MSE;
-- feature **ablation**, **scaling**, or **injection**;
-- baseline vs modified greedy generation;
-- first-token probability table and Jensen-Shannon divergence;
-- **live norm-matched random residual control** for every causal edit;
-- target-token causal specificity ratio against that matched control;
-- **dose-response sweep** over 0×, 0.5×, 1×, 1.5×, 2× and 3× feature scaling;
-- **layer trajectory** diagnostics across layers 4, 14 and 26;
-- optional target-continuation probability / log-probability delta;
-- optional concept hints loaded from real offline benchmark artifacts.
+- prompt input with an explicit **Prompt tokens** view and selected-token highlight;
+- residual layers **4, 14, 26**;
+- strongest active TopK SAE features and reconstruction diagnostics;
+- feature **ablation**, **scaling**, and **injection**;
+- baseline vs SAE-edited greedy generation;
+- next-token probability shifts and Jensen-Shannon divergence;
+- **full-continuation teacher-forced scoring**, including per-token log-probability rows;
+- deterministic **norm-matched random controls**;
+- a batched single-feature scale dose-response sweep;
+- joint multi-feature ablation/scaling;
+- a batched **1 / 3 / 5 strongest-feature joint-ablation sweep**;
+- an interactive **paraphrase robustness** comparison;
+- early/middle/late representation trajectories;
+- optional empirical feature hints loaded only from real offline artifacts.
 
-### Reconstruction-preserving causal edit
+### Why full-continuation scoring matters
 
-Qwen-Scope provides encoder and decoder weights. Let the original residual be `h`, the selected sparse feature activation be `z_i`, and the feature decoder direction be `d_i`.
+v0.2 measured the first token of a user-supplied target. That is insufficient for a continuation such as `2x` when it tokenizes into multiple tokens.
 
-FeatureLens patches the **original** residual:
+v0.3 concatenates the exact target token IDs to the prompt and scores every target token teacher-forced. It reports:
+
+- total target sequence log probability;
+- **mean target log probability per token**;
+- SAE-edit deltas;
+- norm-matched random-control deltas;
+- a token-by-token decomposition.
+
+Mean log probability per target token is the primary length-comparable causal metric used by the v0.3 offline report.
+
+Greedy text may remain unchanged even when these probability-level metrics move. That is expected: deterministic generation changes only after an edit moves a different token across the argmax boundary.
+
+## Reconstruction-preserving causal edits
+
+Let the original residual be `h`, sparse activation `z_i`, and decoder direction `d_i`.
 
 ```text
 ablate:   h' = h - z_i d_i
@@ -63,7 +82,52 @@ scale α:  h' = h + (α - 1) z_i d_i
 inject δ: h' = h + δ d_i
 ```
 
-It does **not** replace `h` with the SAE reconstruction. This keeps SAE reconstruction error out of the causal perturbation.
+FeatureLens patches the **original** residual. It does not replace `h` with the complete SAE reconstruction, so SAE reconstruction error is not introduced as a causal confound.
+
+For a feature set `S`, ablation/scaling deltas are summed before patching:
+
+```text
+h' = h + Σ_i∈S Δz_i d_i
+```
+
+`inject` is intentionally excluded from multi-feature sets because one shared additive coefficient is not naturally comparable across unrelated decoder directions.
+
+## Batched causal sweeps
+
+The stronger v0.3 experiments are designed not to multiply ZeroGPU round-trips unnecessarily.
+
+### Single-feature scale dose-response
+
+The UI evaluates:
+
+```text
+0×, 0.5×, 1×, 1.5×, 2×, 3×
+```
+
+where **0× = ablation** and **1× = no intervention**. All six residual edits are stacked and evaluated in one model forward after the baseline.
+
+### Feature-set size sweep
+
+The live and offline experiments jointly ablate the strongest active / concept-selected features at:
+
+```text
+k = 1, 3, 5
+```
+
+Each targeted edit is paired with a random residual perturbation of identical L2 norm. The edited and control conditions are batched together after one baseline forward.
+
+This directly tests a plausible failure mode of single-feature interpretability: a concept may be represented across a small sparse subspace rather than one SAE unit.
+
+## Paraphrase robustness explorer
+
+The **Paraphrase robustness** tab compares the same residual layer for an original prompt and a manually supplied rewording. It reports:
+
+- full TopK feature-set Jaccard;
+- sparse activation cosine without densifying the 32,768-wide SAE vector;
+- shared / original-only / paraphrase-only displayed features;
+- side-by-side activation bars.
+
+High overlap is robustness evidence, not a semantic proof about any individual feature.
 
 ## Offline experiment
 
@@ -71,29 +135,29 @@ The repository ships a controlled benchmark with:
 
 - **224 discovery prompts** across 7 concepts;
 - **112 paraphrase pairs** kept together during splitting;
-- concepts: code, mathematics, positive sentiment, negative sentiment, French, factual entities, uncertainty;
+- code, mathematics, positive sentiment, negative sentiment, French, factual entities, uncertainty;
 - **28 separate causal completion tasks**;
-- one forward pass captures all configured residual layers for each batch;
-- TopK sparse feature activations and dense residuals are saved separately.
+- residual and SAE activation collection at layers 4, 14 and 26.
 
 The evaluation computes:
 
 - SAE reconstruction cosine / NMSE;
-- actual active-feature count / sparsity;
 - held-out feature/concept AUROC and F1;
 - paraphrase TopK Jaccard and sparse-activation cosine;
 - layer-wise multinomial linear probes on dense residual states;
 - selected-feature ablation and 2× amplification;
-- target first-token probability, log-probability, rank and JS divergence;
-- **norm-matched random residual-direction controls**;
-- bootstrap 95% confidence intervals for aggregate metrics;
-- a paired sign-flip randomization test for SAE-vs-random causal-effect differences.
+- exact full-target sequence and mean-per-token log probabilities;
+- next-token probability/rank, JS divergence, and top-1 changes;
+- norm-matched random residual controls;
+- **top-1 / top-3 / top-5 same-layer concept-feature joint ablations**;
+- bootstrap 95% confidence intervals;
+- paired sign-flip tests for SAE-vs-control effect differences.
 
-Feature selection uses the **training split**. Held-out AUROC/F1 are reported afterward. Paraphrases from the same pair never cross the train/test boundary.
+Feature selection uses only the **training split**. Held-out AUROC/F1 are reported afterward. Paraphrases from the same pair never cross the train/test boundary.
 
-## Run the full benchmark
+## Run the benchmark
 
-A CUDA machine is strongly recommended. The default pipeline downloads Qwen3-1.7B-Base plus the three selected Qwen-Scope layer checkpoints.
+A CUDA machine is strongly recommended.
 
 ```bash
 python -m venv .venv
@@ -102,6 +166,17 @@ pip install -r requirements.txt
 python experiments/run_all.py
 ```
 
+The pipeline runs:
+
+```text
+build_dataset
+→ collect_activations
+→ evaluate_features
+→ run_causal
+→ run_feature_sets
+→ make_report
+```
+
 Outputs are materialized under `artifacts/`:
 
 ```text
@@ -115,24 +190,21 @@ artifacts/
 ├── layer_metrics.csv
 ├── stability.csv
 ├── causal_results.csv
+├── feature_set_results.csv
 ├── summary.json
 ├── report.md
 └── figures/
+    ├── feature_auroc.png
+    ├── layer_diagnostics.png
+    ├── causal_effects.png
+    └── feature_set_effects.png
 ```
 
-`report.md` and `summary.json` are generated from the measured results. The code contains no fabricated benchmark numbers.
+`report.md` and `summary.json` are generated from measured results. The repository contains no fabricated benchmark numbers.
 
 ## Hugging Face ZeroGPU deployment
 
-FeatureLens is intentionally a **Gradio SDK Space**, not a Docker Space, because ZeroGPU currently supports Gradio SDK Spaces.
-
-1. Create a new **Gradio** Space.
-2. In the Space hardware settings, select **ZeroGPU**.
-3. Push this repository as-is.
-4. No API key is required; both Qwen repositories are public.
-5. On first cold start, the Space downloads Qwen3-1.7B-Base and only SAE layers **4, 14 and 26**, not all 28 SAE files.
-
-The app uses the `spaces.GPU` decorator when available and falls back to ordinary execution locally. On a Hugging Face Space it eagerly places the model and selected SAEs on the ZeroGPU CUDA-emulation device at startup, matching Hugging Face's recommended loading pattern.
+FeatureLens is a **Gradio SDK Space**. The live demo only needs Qwen3-1.7B-Base plus SAE layers **4, 14, 26** rather than all 28 layer checkpoints.
 
 Useful environment overrides:
 
@@ -145,54 +217,38 @@ FEATURELENS_SAE_DTYPE=float16
 FEATURELENS_MAX_NEW_TOKENS=32
 ```
 
-For offline benchmarking, use `FEATURELENS_SAE_DTYPE=float32` (the scripts already use FP32 SAE weights by default).
+The app uses the `spaces.GPU` decorator when available and falls back to ordinary local execution. SSR is explicitly disabled in `app.py` to avoid the earlier Gradio auth-coroutine warning observed during v0.2 deployment testing.
 
 ## Repository layout
 
 ```text
 FeatureLens/
-├── app.py                         # Gradio / ZeroGPU workbench
+├── app.py
 ├── featurelens/
-│   ├── config.py                  # model + layer configuration
-│   ├── sae.py                     # TopK SAE loading / encode / decode
-│   ├── interventions.py           # causal deltas + random controls
-│   ├── runtime.py                 # hooks, generation, probability deltas
-│   ├── metrics.py                 # reconstruction + divergence metrics
-│   ├── stats.py                   # bootstrap CIs + paired randomization test
-│   └── catalog.py                 # offline result integration
+│   ├── config.py
+│   ├── sae.py
+│   ├── interventions.py
+│   ├── runtime.py
+│   ├── metrics.py
+│   ├── stats.py
+│   ├── selection.py
+│   └── catalog.py
 ├── experiments/
 │   ├── build_dataset.py
 │   ├── collect_activations.py
 │   ├── evaluate_features.py
 │   ├── run_causal.py
+│   ├── run_feature_sets.py
 │   ├── make_report.py
 │   └── run_all.py
 ├── data/
-│   ├── prompts.jsonl
-│   └── causal_tasks.jsonl
 ├── tests/
 ├── scripts/release_check.py
 ├── docs/
 └── research_config.json
 ```
 
-## Methodological limitations
-
-- Qwen-Scope features are TopK sparse directions, not guaranteed monosemantic concepts.
-- The controlled benchmark is intentionally small enough to reproduce on modest research compute; it is not a universal feature ontology.
-- The live demo applies one residual edit at one selected prompt token. This is cleaner for causal attribution than repeatedly steering every generated token, but it may produce smaller behavioral effects.
-- A target string may tokenize into multiple tokens. The live workbench explicitly labels its target metric as the **first-token** probability in that case.
-- A causal effect can depend strongly on prompt, layer, feature scale, and downstream task. The report therefore includes raw per-task rows rather than only aggregate means.
-
-## v0.2 deployment hardening
-
-The Space pins **Gradio 6.24.0** in both the Space metadata and Python dependencies and launches with `ssr_mode=False`. This deliberately avoids the SSR execution path that produced an un-awaited `get_current_user` coroutine warning in the initial deployment while keeping the app fully functional in client-side rendering mode.
-
-The ZeroGPU callbacks also use shorter declared GPU durations than v0.1 and generation defaults to 16 new tokens, reducing queue cost for ordinary demo usage.
-
-See [`docs/VALIDATION.md`](docs/VALIDATION.md) for the pre-push and hosted test matrix.
-
-## Reproducibility and checks
+## Validation
 
 ```bash
 python -m ruff check app.py featurelens experiments tests scripts
@@ -201,11 +257,22 @@ python -m compileall -q app.py featurelens experiments scripts
 python scripts/release_check.py
 ```
 
+See [`docs/VALIDATION.md`](docs/VALIDATION.md) for the exact human HF smoke tests. The names in that document match the actual labels in the v0.3 UI.
+
+## Methodological limitations
+
+- Qwen-Scope features are sparse directions, not guaranteed monosemantic concepts.
+- A controlled seven-concept benchmark is not a universal feature ontology.
+- Similar TopK sets under paraphrasing do not prove identical semantic meaning.
+- Joint feature ablation can create a larger residual perturbation as `k` increases; norm-matched controls are therefore essential.
+- Full target scoring is teacher-forced: it measures how the edit changes probability assigned to a specified continuation, not free-running sequence probability under sampled generation.
+- Causal effects remain prompt-, layer-, token-, scale-, and task-dependent.
+
 ## Resume-ready description
 
 > **FeatureLens — Causal Interpretability Workbench** | PyTorch, Qwen3, Sparse Autoencoders, Mechanistic Interpretability, Gradio  
-> Built an SAE-based interpretability system for Qwen3-1.7B that discovers concept-associated residual features, benchmarks held-out predictiveness and paraphrase stability against linear probes, and causally ablates/amplifies features with norm-matched controls while measuring downstream token-probability and generation shifts.
+> Built an SAE-based interpretability system for Qwen3-1.7B that discovers held-out concept-associated residual features, measures paraphrase stability against dense probes, and causally tests single features and sparse feature sets with full-continuation scoring, dose-response analysis, and norm-matched controls.
 
 ## Acknowledgements
 
-FeatureLens builds on the open Qwen3 model and Qwen-Scope SAE checkpoints from the Qwen team. See the upstream model cards and Qwen-Scope technical report for model and SAE details and licensing requirements.
+FeatureLens builds on the open Qwen3 model and Qwen-Scope residual-stream SAE checkpoints from the Qwen team.

app.py

@@ -7,24 +7,63 @@ from featurelens.config import SETTINGS
 from featurelens.hf_runtime import gpu
 from featurelens.runtime import RUNTIME
 
+# Muted, low-saturation palette chosen to stay readable in light and dark UI modes.
+MUTED_TEAL = "#6F8F8B"
+MUTED_OCHRE = "#B08D57"
+MUTED_RED = "#A66B6B"
+MUTED_PURPLE = "#8B7EA8"
+MUTED_STONE = "#A59E93"
+
 CSS = """
 .gradio-container { max-width: 1320px !important; }
 .hero { padding: 8px 2px 2px; }
 .hero h1 { margin: 0; font-size: 2.35rem; letter-spacing: -0.045em; }
 .hero p { margin: .35rem 0 0; opacity: .72; font-size: 1rem; }
-.research-q { border-left: 4px solid var(--primary-500); padding: 10px 14px; margin: 10px 0 14px; border-radius: 0 10px 10px 0; background: var(--background-fill-secondary); }
+.research-q {
+  border-left: 4px solid #6F8F8B;
+  padding: 10px 14px;
+  margin: 10px 0 14px;
+  border-radius: 0 10px 10px 0;
+  background: var(--background-fill-secondary);
+}
 .badges { display:flex; flex-wrap:wrap; gap:7px; margin:8px 0 3px; }
-.badge { border:1px solid var(--border-color-primary); background:var(--background-fill-secondary); border-radius:999px; padding:4px 9px; font-size:12px; }
-.step { font-size: .78rem; text-transform: uppercase; letter-spacing:.09em; opacity:.62; font-weight:700; margin-top:2px; }
-.token-wrap { display:flex; flex-wrap:wrap; gap:5px; padding:8px 2px; line-height:1.7; }
-.token { background:var(--background-fill-secondary); border:1px solid var(--border-color-primary); border-radius:7px; padding:2px 7px; font-family:ui-monospace,SFMono-Regular,monospace; font-size:12px; }
-.token.selected { border:2px solid var(--primary-500); color:var(--primary-600); font-weight:600; }
+.badge {
+  border:1px solid var(--border-color-primary);
+  background:var(--background-fill-secondary);
+  border-radius:999px;
+  padding:4px 9px;
+  font-size:12px;
+}
+.step {
+  font-size: .78rem;
+  text-transform: uppercase;
+  letter-spacing:.09em;
+  opacity:.66;
+  font-weight:700;
+  margin-top:2px;
+}
+.token-wrap { display:flex; flex-wrap:wrap; gap:5px; padding:6px 2px 10px; line-height:1.7; }
+.token {
+  background:var(--background-fill-secondary);
+  border:1px solid var(--border-color-primary);
+  border-radius:7px;
+  padding:2px 7px;
+  font-family:ui-monospace,SFMono-Regular,monospace;
+  font-size:12px;
+}
+.token.selected { border:2px solid #7FA39F; font-weight:650; }
 .token sup { opacity:.55; margin-right:4px; }
-.small-note { opacity:.65; font-size:12px; }
-.callout { border:1px solid var(--border-color-primary); background:var(--background-fill-secondary); border-radius:12px; padding:10px 12px; }
+.small-note { opacity:.68; font-size:12px; }
+.callout {
+  border:1px solid var(--border-color-primary);
+  background:var(--background-fill-secondary);
+  border-radius:12px;
+  padding:10px 12px;
+}
+.metric-note { opacity:.78; }
 """
 
-THEME = gr.themes.Soft(primary_hue="blue", neutral_hue="slate")
+THEME = gr.themes.Soft(primary_hue="teal", secondary_hue="stone", neutral_hue="stone")
 
 
 def _raise_ui_error(exc: Exception) -> None:
@@ -33,7 +72,8 @@ def _raise_ui_error(exc: Exception) -> None:
 
 def _analysis_metrics_markdown(result) -> str:
     return (
-        f"**Layer {result.layer} · token {result.token_index}**  \n"
+        "#### Analysis metrics\n"
+        f"**Layer {result.layer} · prompt token {result.token_index}**  \n"
         f"Active SAE features: **{int(result.metrics['active_features'])}/{SETTINGS.sae_top_k}**  \n"
         f"Reconstruction cosine: **{result.metrics['cosine']:.4f}** · "
         f"NMSE: **{result.metrics['nmse']:.4f}**  \n"
@@ -42,27 +82,27 @@ def _analysis_metrics_markdown(result) -> str:
 
 
 def _intervention_metrics_markdown(result) -> str:
-    target = "No target continuation supplied — specificity is shown using JS divergence only."
-    if result.baseline_target_prob is not None:
-        warning = ""
-        if result.target_token_count > 1:
-            warning = (
-                f"  \n_Note: target text tokenizes to {result.target_token_count} tokens; "
-                "the displayed causal metric is for its first token._"
-            )
+    target = (
+        "No target continuation supplied. The causal comparison therefore uses only "
+        "next-token Jensen-Shannon divergence."
+    )
+    if result.baseline_sequence_logprob is not None:
+        tokens = " ".join(repr(token) for token in result.target_tokens)
         target = (
-            f"Target first token: `{result.target_token}`  \n"
-            f"Baseline p: **{result.baseline_target_prob:.6f}** · "
-            f"SAE edit p: **{result.modified_target_prob:.6f}** · "
-            f"Random-control p: **{result.random_target_prob:.6f}**  \n"
-            f"SAE Δlog p: **{result.target_logprob_delta:+.4f}** · "
-            f"random Δlog p: **{result.random_target_logprob_delta:+.4f}** · "
-            f"specificity ratio: **{result.target_specificity_ratio:.2f}×**{warning}"
+            f"**Target continuation:** {result.target_token_count} token(s): {tokens}  \n"
+            f"Sequence log p — baseline: **{result.baseline_sequence_logprob:.4f}** · "
+            f"SAE edit: **{result.modified_sequence_logprob:.4f}** · "
+            f"random control: **{result.random_sequence_logprob:.4f}**  \n"
+            f"SAE Δ sequence log p: **{result.sequence_logprob_delta:+.4f}** · "
+            f"random Δ: **{result.random_sequence_logprob_delta:+.4f}**  \n"
+            f"SAE Δ mean log p/token: **{result.mean_logprob_delta:+.4f}** · "
+            f"random Δ: **{result.random_mean_logprob_delta:+.4f}** · "
+            f"specificity ratio: **{result.target_specificity_ratio:.2f}×**"
         )
     inactive = ""
     if abs(result.feature_activation) < 1e-12:
         inactive = (
-            "  \n⚠️ **Selected feature is inactive at this token.** Ablate/scale therefore produces "
+            "  \n⚠️ **Selected feature is inactive at this prompt token.** Ablate/scale produces "
             "a zero feature delta; use `inject` to test the decoder direction directly."
         )
     return (
@@ -71,23 +111,59 @@ def _intervention_metrics_markdown(result) -> str:
         f"Perturbation L2: **{result.perturbation_norm:.4f}**  \n"
         f"Next-token JS: **{result.js_divergence:.6f}** · "
         f"random-control JS: **{result.random_js_divergence:.6f}** · "
-        f"specificity: **{result.js_specificity_ratio:.2f}×**  \n\n"
-        f"{target}{inactive}"
+        f"JS specificity: **{result.js_specificity_ratio:.2f}×**  \n\n"
+        f"{target}{inactive}  \n\n"
+        "_Greedy generations can remain identical even when probability-level causal metrics move; "
+        "an edit must cross an argmax boundary before deterministic text changes._"
     )
 
 
 def _dose_metrics_markdown(result) -> str:
-    note = ""
-    if result.target_token_count > 1:
-        note = (
-            f" Target text spans {result.target_token_count} tokens; the curve measures its first token."
-        )
+    tokens = " ".join(repr(token) for token in result.target_tokens)
     inactive = ""
     if abs(result.feature_activation) < 1e-12:
-        inactive = " **The feature is inactive here, so a multiplicative sweep is flat by construction.**"
+        inactive = " **The feature is inactive here, so multiplicative scaling is flat by construction.**"
     return (
         f"Feature activation at baseline: **{result.feature_activation:.4f}** · "
-        f"target first token: `{result.target_token}`.{note}{inactive}"
+        f"target continuation: {len(result.target_tokens)} token(s): {tokens}.{inactive}  \n\n"
+        "This panel is **always a scale intervention**: 0× = ablation, 1× = no edit, "
+        "2× = double the original coefficient. It does not use the intervention radio above."
+    )
+
+
+def _feature_set_metrics_markdown(result) -> str:
+    tokens = " ".join(repr(token) for token in result.target_tokens)
+    inactive_count = sum(abs(float(row[1])) < 1e-12 for row in result.feature_rows)
+    inactive_note = (
+        f"  \n{inactive_count} selected feature(s) were inactive and therefore contributed zero "
+        "delta under ablation/scale."
+        if inactive_count
+        else ""
+    )
+    return (
+        f"Selected feature set: **{len(result.feature_ids)} features** · "
+        f"perturbation L2: **{result.perturbation_norm:.4f}**  \n"
+        f"Target continuation: {len(result.target_tokens)} token(s): {tokens}  \n"
+        f"SAE Δ mean log p/token: **{result.mean_logprob_delta:+.4f}** · "
+        f"random Δ: **{result.random_mean_logprob_delta:+.4f}** · "
+        f"specificity: **{result.target_specificity_ratio:.2f}×**  \n"
+        f"SAE Δ sequence log p: **{result.sequence_logprob_delta:+.4f}** · "
+        f"random Δ: **{result.random_sequence_logprob_delta:+.4f}**  \n"
+        f"Next-token JS: **{result.js_divergence:.6f}** · "
+        f"random-control JS: **{result.random_js_divergence:.6f}** · "
+        f"JS specificity: **{result.js_specificity_ratio:.2f}×**{inactive_note}"
+    )
+
+
+def _paraphrase_metrics_markdown(result) -> str:
+    return (
+        "#### Robustness metrics\n"
+        f"Full TopK feature-set Jaccard: **{result.topk_jaccard:.3f}**  \n"
+        f"Sparse activation cosine: **{result.sparse_cosine:.3f}**  \n"
+        f"Shared features among displayed top-{result.top_n}: "
+        f"**{result.shared_top_n}/{result.top_n}**  \n\n"
+        "Higher values mean the sparse representation is more stable under the supplied paraphrase; "
+        "they do **not** prove the shared features have identical semantics."
     )
 
 
@@ -99,10 +175,15 @@ def analyze_prompt(prompt: str, layer: int, token_index: int, top_n: int):
         result = RUNTIME.analyze(prompt, int(layer), int(token_index), int(top_n))
         choices = [str(int(row[1])) for row in result.rows]
         feature_update = gr.update(choices=choices, value=choices[0] if choices else None)
+        feature_set_update = gr.update(
+            choices=choices,
+            value=choices[: min(3, len(choices))],
+        )
         chart_df = pd.DataFrame(
             {
                 "Feature": [str(int(row[1])) for row in result.rows],
                 "Activation": [float(row[2]) for row in result.rows],
+                "Series": ["Activation"] * len(result.rows),
             }
         )
         return (
@@ -110,6 +191,7 @@ def analyze_prompt(prompt: str, layer: int, token_index: int, top_n: int):
             result.rows,
             chart_df,
             feature_update,
+            feature_set_update,
             _analysis_metrics_markdown(result),
         )
     except Exception as exc:
@@ -148,12 +230,13 @@ def run_intervention(
             result.modified_text,
             _intervention_metrics_markdown(result),
             result.top_token_rows,
+            result.target_token_rows,
         )
     except Exception as exc:
         _raise_ui_error(exc)
 
 
-@gpu(duration=40)
+@gpu(duration=35)
 def run_dose_response(
     prompt: str,
     layer: int,
@@ -167,7 +250,7 @@ def run_dose_response(
         if feature_id is None or str(feature_id).strip() == "":
             raise ValueError("Choose or enter a feature id.")
         if not target_text.strip():
-            raise ValueError("Enter a target continuation before running a dose-response sweep.")
+            raise ValueError("Enter a target continuation before running the scale dose-response.")
         result = RUNTIME.dose_response(
             text=prompt,
             layer=int(layer),
@@ -179,13 +262,15 @@ def run_dose_response(
             "Multiplier",
             "Δ feature coefficient",
             "Perturbation L2",
-            "Baseline p(target)",
-            "Modified p(target)",
-            "Δ log p(target)",
-            "JS divergence",
+            "Baseline mean log p/token",
+            "Modified mean log p/token",
+            "Δ mean log p/token",
+            "Δ sequence log p",
+            "Next-token JS",
         ]
         table = pd.DataFrame(result.rows, columns=columns)
-        plot = table[["Multiplier", "Δ log p(target)"]].copy()
+        plot = table[["Multiplier", "Δ mean log p/token"]].copy()
+        plot["Series"] = "SAE feature"
         return table, plot, _dose_metrics_markdown(result)
     except Exception as exc:
         _raise_ui_error(exc)
@@ -218,6 +303,116 @@ def run_layer_sweep(prompt: str, token_index: int):
         _raise_ui_error(exc)
 
 
+@gpu(duration=35)
+def run_feature_set(
+    prompt: str,
+    layer: int,
+    token_index: int,
+    feature_ids: list[str] | None,
+    mode: str,
+    coefficient: float,
+    target_text: str,
+):
+    try:
+        if not prompt.strip():
+            raise ValueError("Enter a prompt in the Workbench first.")
+        selected = [int(float(value)) for value in (feature_ids or [])]
+        if not selected:
+            raise ValueError("Select at least one feature in 'Feature set'.")
+        if not target_text.strip():
+            raise ValueError("Enter a target continuation for the feature-set causal test.")
+        result = RUNTIME.intervene_feature_set(
+            text=prompt,
+            layer=int(layer),
+            token_index=int(token_index),
+            feature_ids=selected,
+            mode=mode,
+            coefficient=float(coefficient),
+            target_text=target_text,
+        )
+        return result.feature_rows, _feature_set_metrics_markdown(result), result.target_token_rows
+    except Exception as exc:
+        _raise_ui_error(exc)
+
+
+@gpu(duration=35)
+def run_feature_set_sweep(
+    prompt: str,
+    layer: int,
+    token_index: int,
+    target_text: str,
+):
+    try:
+        if not prompt.strip():
+            raise ValueError("Enter a prompt in the Workbench first.")
+        if not target_text.strip():
+            raise ValueError("Enter a target continuation before running the set-size sweep.")
+        result = RUNTIME.feature_set_size_sweep(
+            text=prompt,
+            layer=int(layer),
+            token_index=int(token_index),
+            target_text=target_text,
+        )
+        columns = [
+            "Set size k",
+            "Feature ids",
+            "Perturbation L2",
+            "Baseline mean log p/token",
+            "SAE mean log p/token",
+            "SAE Δ mean log p/token",
+            "Random Δ mean log p/token",
+            "Specificity ratio",
+            "SAE Δ sequence log p",
+            "SAE next-token JS",
+            "Random next-token JS",
+        ]
+        table = pd.DataFrame(result.rows, columns=columns)
+        plot_rows = []
+        for _, row in table.iterrows():
+            plot_rows.append([row["Set size k"], "Top-k SAE ablation", row["SAE Δ mean log p/token"]])
+            plot_rows.append([row["Set size k"], "Norm-matched random", row["Random Δ mean log p/token"]])
+        plot = pd.DataFrame(plot_rows, columns=["Set size k", "Condition", "Δ mean log p/token"])
+        tokens = " ".join(repr(token) for token in result.target_tokens)
+        note = (
+            f"Target continuation: {len(result.target_tokens)} token(s): {tokens}. "
+            "For each k, FeatureLens jointly **ablates the k strongest active SAE features** at the "
+            "selected prompt token and compares that joint edit with a norm-matched random residual control."
+        )
+        return table, plot, note
+    except Exception as exc:
+        _raise_ui_error(exc)
+
+
+@gpu(duration=30)
+def run_paraphrase_compare(
+    original_prompt: str,
+    paraphrase_prompt: str,
+    layer: int,
+    token_index_a: int,
+    token_index_b: int,
+    top_n: int,
+):
+    try:
+        result = RUNTIME.compare_paraphrases(
+            text_a=original_prompt,
+            text_b=paraphrase_prompt,
+            layer=int(layer),
+            token_index_a=int(token_index_a),
+            token_index_b=int(token_index_b),
+            top_n=int(top_n),
+        )
+        chart = pd.DataFrame(result.chart_rows, columns=["Feature", "Prompt", "Activation"])
+        return (
+            RUNTIME.token_html(result.tokens_a, result.token_index_a),
+            RUNTIME.token_html(result.tokens_b, result.token_index_b),
+            _paraphrase_metrics_markdown(result),
+            result.rows,
+            chart,
+        )
+    except Exception as exc:
+        _raise_ui_error(exc)
+
+
 def mode_help(mode: str):
     if mode == "ablate":
         return gr.update(
@@ -241,20 +436,37 @@ def mode_help(mode: str):
     )
 
 
+def set_mode_help(mode: str):
+    if mode == "ablate":
+        return gr.update(
+            value=0.0,
+            interactive=False,
+            label="Set coefficient (unused for ablation)",
+            info="Jointly sets every selected active feature coefficient to zero.",
+        )
+    return gr.update(
+        value=2.0,
+        interactive=True,
+        label="Shared feature multiplier",
+        info="Applies the same multiplier to each selected feature before summing decoder deltas.",
+    )
+
+
 with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench") as demo:
     gr.HTML(
-        '<div class="hero"><h1>FeatureLens <span style="font-size:.42em;opacity:.55">v0.2</span></h1>'
+        '<div class="hero"><h1>FeatureLens <span style="font-size:.42em;opacity:.55">v0.3</span></h1>'
         '<p>Causal sparse-feature interpretability for Qwen3-1.7B — inspect → intervene → control → measure.</p></div>'
         '<div class="research-q"><b>Research question:</b> Do sparse features that predict a concept also '
         'causally influence the model’s behaviour?</div>'
         '<div class="badges">'
         '<span class="badge">Qwen3-1.7B-Base</span><span class="badge">Qwen-Scope SAE</span>'
         '<span class="badge">32,768 features</span><span class="badge">TopK=50</span>'
+        '<span class="badge">full-sequence scoring</span><span class="badge">feature sets</span>'
         '<span class="badge">norm-matched controls</span><span class="badge">ZeroGPU</span></div>'
     )
 
     with gr.Tab("Workbench"):
-        gr.HTML('<div class="step">Step 1 · Choose a prompt and residual location</div>')
+        gr.HTML('<div class="step">Step 1 · Inspect one prompt location</div>')
         with gr.Row(equal_height=False):
             with gr.Column(scale=5):
                 prompt = gr.Textbox(
@@ -284,49 +496,54 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench") as dem
                     value=-1,
                     precision=0,
                     label="Prompt token index",
-                    info="-1 = final prompt token. Inspect once to see all token positions.",
+                    info="-1 = final prompt token. Inspect once to see all token positions below.",
                 )
-                top_n = gr.Slider(5, 20, value=12, step=1, label="Top features")
+                top_n = gr.Slider(5, 20, value=12, step=1, label="Displayed active features")
                 analyze_btn = gr.Button("Inspect sparse features", variant="primary")
 
-        token_view = gr.HTML('<div class="small-note">Token positions appear here after inspection.</div>')
+        gr.Markdown("#### Prompt tokens\nThe selected prompt token is outlined more strongly.")
+        token_view = gr.HTML('<div class="small-note">Prompt tokens appear here after clicking <b>Inspect sparse features</b>.</div>')
         analysis_metrics = gr.Markdown()
         with gr.Row(equal_height=False):
             feature_table = gr.Dataframe(
                 headers=["Rank", "Feature id", "Activation", "Offline concept hint"],
                 datatype=["number", "number", "number", "str"],
                 interactive=False,
-                label="Strongest TopK features",
+                label="Strongest active SAE features",
                 wrap=True,
                 scale=3,
             )
             feature_plot = gr.BarPlot(
                 x="Feature",
                 y="Activation",
+                color="Series",
+                color_map={"Activation": MUTED_TEAL},
                 title="Activation profile",
                 x_title="Feature id",
                 y_title="Activation",
+                x_label_angle=-35,
+                height=330,
                 scale=2,
             )
 
-        gr.HTML('<div class="step">Step 2 · Intervene and compare against a matched control</div>')
+        gr.HTML('<div class="step">Step 2 · Run one single-feature causal test</div>')
         gr.Markdown(
-            "FeatureLens changes only the selected SAE coefficient and adds that decoder-vector delta "
-            "to the **original** residual. The live negative control applies a deterministic random "
-            "residual perturbation with the **same L2 norm**."
+            "FeatureLens edits the **original residual**, then compares the SAE edit with a deterministic "
+            "random residual perturbation of the **same L2 norm**. If a target continuation is supplied, "
+            "v0.3 scores the **entire continuation teacher-forced**, not only its first token."
         )
         with gr.Row(equal_height=False):
             with gr.Column(scale=2):
                 feature_id = gr.Dropdown(
                     choices=[],
                     allow_custom_value=True,
-                    label="Feature id",
+                    label="Single feature id",
                     info="Inspection populates the strongest active features; custom IDs are also allowed.",
                 )
                 mode = gr.Radio(
                     choices=["ablate", "scale", "inject"],
                     value="ablate",
-                    label="Intervention",
+                    label="Single-feature intervention",
                 )
                 coefficient = gr.Number(
                     value=0.0,
@@ -334,73 +551,220 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench") as dem
                     label="Coefficient (unused for ablation)",
                 )
                 target_text = gr.Textbox(
-                    label="Optional target continuation",
+                    label="Target continuation (optional)",
                     placeholder="e.g. 2x",
-                    info="Supplying this enables target-token Δlog p and causal-specificity metrics.",
+                    info=(
+                        "Exact text to score after the prompt. v0.3 reports full-sequence and per-token "
+                        "log probabilities. Include a leading space if that is part of the continuation."
+                    ),
                 )
                 max_new = gr.Slider(
                     4,
                     SETTINGS.max_new_tokens,
-                    value=min(16, SETTINGS.max_new_tokens),
+                    value=min(12, SETTINGS.max_new_tokens),
                     step=1,
-                    label="Max new tokens",
+                    label="Greedy generation length",
                 )
-                intervene_btn = gr.Button("Run causal test", variant="primary")
+                intervene_btn = gr.Button("Run single-feature causal test", variant="primary")
                 intervention_metrics = gr.Markdown()
             with gr.Column(scale=3):
                 with gr.Row():
-                    baseline_out = gr.Textbox(label="Baseline generation", lines=7, interactive=False)
-                    modified_out = gr.Textbox(label="SAE-intervened generation", lines=7, interactive=False)
+                    baseline_out = gr.Textbox(label="Baseline greedy generation", lines=6, interactive=False)
+                    modified_out = gr.Textbox(label="SAE-edited greedy generation", lines=6, interactive=False)
                 token_prob_table = gr.Dataframe(
                     headers=["Token", "Baseline p", "SAE-edit p", "Δ probability"],
                     datatype=["str", "number", "number", "number"],
                     interactive=False,
                     label="Next-token distribution shift",
                 )
+                target_token_table = gr.Dataframe(
+                    headers=[
+                        "Target position",
+                        "Target token",
+                        "Baseline log p",
+                        "SAE-edit log p",
+                        "Random log p",
+                        "SAE Δ log p",
+                        "Random Δ log p",
+                    ],
+                    datatype=["number", "str", "number", "number", "number", "number", "number"],
+                    interactive=False,
+                    label="Target continuation token-by-token score",
+                )
 
-        gr.HTML('<div class="step">Step 3 · Test dose–response</div>')
-        with gr.Accordion("Causal dose–response sweep", open=False):
+        gr.HTML('<div class="step">Step 3 · Check single-feature dose-response</div>')
+        with gr.Accordion("Single-feature scale dose-response", open=False):
             gr.Markdown(
-                "A single coefficient can be cherry-picked. This sweep scales the same active feature "
-                "through **0×, 0.5×, 1×, 1.5×, 2× and 3×** and measures target-token Δlog p. "
-                "A coherent monotonic response is stronger causal evidence than one isolated edit."
+                "This panel is **always a scale intervention**, regardless of the radio choice above. "
+                "It evaluates **0×, 0.5×, 1×, 1.5×, 2× and 3×** in one batched forward pass after the "
+                "baseline. Here **0× = ablation** and **1× = no intervention**."
             )
-            dose_btn = gr.Button("Run dose–response sweep")
+            dose_btn = gr.Button("Run scale dose-response")
             dose_metrics = gr.Markdown()
             with gr.Row():
-                dose_table = gr.Dataframe(interactive=False, label="Dose-response measurements", scale=3)
+                dose_table = gr.Dataframe(interactive=False, label="Scale dose-response measurements", scale=3)
                 dose_plot = gr.LinePlot(
                     x="Multiplier",
-                    y="Δ log p(target)",
-                    title="Causal dose–response",
+                    y="Δ mean log p/token",
+                    color="Series",
+                    color_map={"SAE feature": MUTED_TEAL},
+                    title="Scale dose-response",
                     x_title="Feature multiplier",
-                    y_title="Δ log p(target)",
+                    y_title="Δ mean log p/token",
+                    height=330,
                     scale=2,
                 )
 
-        analyze_btn.click(
-            analyze_prompt,
-            inputs=[prompt, layer, token_index, top_n],
-            outputs=[token_view, feature_table, feature_plot, feature_id, analysis_metrics],
+    with gr.Tab("Feature sets"):
+        gr.Markdown(
+            "### Test distributed sparse representations\n"
+            "This tab reuses the **Prompt**, **Residual layer**, and **Prompt token index** from the Workbench. "
+            "Click **Inspect sparse features** there first; its displayed feature IDs populate the selector below."
+        )
+        feature_set_ids = gr.Dropdown(
+            choices=[],
+            value=[],
+            multiselect=True,
+            allow_custom_value=True,
+            max_choices=12,
+            label="Feature set",
+            info="Choose several active features from the Workbench inspection, or enter custom IDs.",
         )
-        mode.change(mode_help, inputs=[mode], outputs=[coefficient])
-        intervene_btn.click(
-            run_intervention,
-            inputs=[prompt, layer, token_index, feature_id, mode, coefficient, target_text, max_new],
-            outputs=[baseline_out, modified_out, intervention_metrics, token_prob_table],
+        with gr.Row():
+            set_mode = gr.Radio(
+                choices=["ablate", "scale"],
+                value="ablate",
+                label="Feature-set intervention",
+                info="Inject is omitted because one shared additive coefficient is not comparable across multiple directions.",
+            )
+            set_coefficient = gr.Number(
+                value=0.0,
+                interactive=False,
+                label="Set coefficient (unused for ablation)",
+            )
+            set_target = gr.Textbox(
+                label="Target continuation",
+                value="2x",
+                info="Required. The complete continuation is scored teacher-forced.",
+            )
+        set_btn = gr.Button("Run joint feature-set causal test", variant="primary")
+        set_metrics = gr.Markdown()
+        with gr.Row():
+            set_feature_table = gr.Dataframe(
+                headers=["Feature id", "Original activation", "Δ coefficient", "Offline concept hint"],
+                datatype=["number", "number", "number", "str"],
+                interactive=False,
+                label="Joint intervention features",
+                scale=2,
+            )
+            set_target_table = gr.Dataframe(
+                headers=[
+                    "Target position",
+                    "Target token",
+                    "Baseline log p",
+                    "SAE-edit log p",
+                    "Random log p",
+                    "SAE Δ log p",
+                    "Random Δ log p",
+                ],
+                datatype=["number", "str", "number", "number", "number", "number", "number"],
+                interactive=False,
+                label="Target continuation token-by-token score",
+                scale=3,
+            )
+
+        gr.Markdown("### Top-k joint-ablation sweep")
+        gr.Markdown(
+            "Instead of choosing a single feature, this experiment automatically ablates the **1, 3, and 5 "
+            "strongest active features** at the current Workbench location. All six edited/control conditions "
+            "are evaluated in one batched forward pass after the baseline."
         )
-        dose_btn.click(
-            run_dose_response,
-            inputs=[prompt, layer, token_index, feature_id, target_text],
-            outputs=[dose_table, dose_plot, dose_metrics],
+        set_sweep_target = gr.Textbox(
+            label="Target continuation for set-size sweep",
+            value="2x",
         )
+        set_sweep_btn = gr.Button("Run 1/3/5-feature ablation sweep")
+        set_sweep_note = gr.Markdown()
+        with gr.Row():
+            set_sweep_table = gr.Dataframe(interactive=False, label="Feature-set size measurements", scale=3)
+            set_sweep_plot = gr.LinePlot(
+                x="Set size k",
+                y="Δ mean log p/token",
+                color="Condition",
+                color_map={
+                    "Top-k SAE ablation": MUTED_TEAL,
+                    "Norm-matched random": MUTED_STONE,
+                },
+                title="Effect vs feature-set size",
+                x_title="Number of jointly ablated features",
+                y_title="Δ mean log p/token",
+                height=330,
+                scale=2,
+            )
+
+    with gr.Tab("Paraphrase robustness"):
+        gr.Markdown(
+            "### Does the sparse representation survive a rewording?\n"
+            "Compare the TopK SAE representation of an original prompt with a manually supplied paraphrase. "
+            "This mirrors the offline paraphrase-stability evaluation without pretending that feature IDs "
+            "have known semantics."
+        )
+        with gr.Row():
+            para_a = gr.Textbox(
+                label="Original prompt",
+                lines=4,
+                value="The derivative of x squared is",
+            )
+            para_b = gr.Textbox(
+                label="Paraphrase",
+                lines=4,
+                value="Differentiate x squared with respect to x:",
+            )
+        with gr.Row():
+            para_layer = gr.Dropdown(
+                choices=list(SETTINGS.layers),
+                value=SETTINGS.layers[1] if len(SETTINGS.layers) > 1 else SETTINGS.layers[0],
+                label="Residual layer",
+            )
+            para_idx_a = gr.Number(value=-1, precision=0, label="Original prompt token index")
+            para_idx_b = gr.Number(value=-1, precision=0, label="Paraphrase token index")
+            para_top_n = gr.Slider(5, 20, value=12, step=1, label="Displayed active features")
+        para_btn = gr.Button("Compare paraphrase representations", variant="primary")
+        with gr.Row():
+            with gr.Column():
+                gr.Markdown("#### Original prompt tokens")
+                para_tokens_a = gr.HTML()
+            with gr.Column():
+                gr.Markdown("#### Paraphrase tokens")
+                para_tokens_b = gr.HTML()
+        para_metrics = gr.Markdown()
+        with gr.Row():
+            para_table = gr.Dataframe(
+                headers=["Feature id", "Original activation", "Paraphrase activation", "Status", "Offline concept hint"],
+                datatype=["number", "number", "number", "str", "str"],
+                interactive=False,
+                label="Top-feature overlap",
+                scale=3,
+            )
+            para_plot = gr.BarPlot(
+                x="Feature",
+                y="Activation",
+                color="Prompt",
+                color_map={"Original": MUTED_TEAL, "Paraphrase": MUTED_PURPLE},
+                title="Original vs paraphrase activation",
+                x_title="Feature id",
+                y_title="Activation",
+                x_label_angle=-35,
+                height=330,
+                scale=2,
+            )
 
     with gr.Tab("Layer trajectory"):
         gr.Markdown(
-            "### Follow the representation across early, middle and late residual streams\n"
+            "### Follow representation structure across early, middle and late residual streams\n"
             "This is **not** a cross-layer feature-ID comparison — SAE dictionaries are layer-specific. "
-            "Instead, it compares reconstruction quality and sparsity/concentration statistics at the "
-            "same prompt token across layers 4, 14 and 26."
+            "It compares reconstruction quality and sparsity/concentration statistics at the same prompt "
+            "token across layers 4, 14 and 26."
         )
         with gr.Row():
             trajectory_prompt = gr.Textbox(
@@ -417,6 +781,7 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench") as dem
                 scale=1,
             )
         trajectory_btn = gr.Button("Compare layers", variant="primary")
+        gr.Markdown("#### Prompt tokens")
         trajectory_tokens = gr.HTML()
         with gr.Row():
             trajectory_table = gr.Dataframe(interactive=False, label="Layer diagnostics", scale=3)
@@ -424,24 +789,26 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench") as dem
                 x="Layer",
                 y="Value",
                 color="Metric",
+                color_map={
+                    "Reconstruction cosine": MUTED_TEAL,
+                    "Top-5 mass": MUTED_OCHRE,
+                    "Activation entropy": MUTED_RED,
+                },
                 title="Representation trajectory",
                 x_title="Layer",
                 y_title="Normalized value",
+                height=330,
                 scale=2,
             )
-        trajectory_btn.click(
-            run_layer_sweep,
-            inputs=[trajectory_prompt, trajectory_token],
-            outputs=[trajectory_tokens, trajectory_table, trajectory_plot],
-        )
 
     with gr.Tab("Offline benchmark"):
         gr.Markdown(RUNTIME.catalog.benchmark_markdown())
         gr.Markdown(
             "The offline pipeline evaluates held-out feature/concept AUROC + F1, reconstruction quality, "
-            "paraphrase stability, dense residual linear probes, causal ablation/amplification and "
-            "norm-matched random-direction controls. Results are loaded from `artifacts/`; the app never "
-            "ships invented benchmark numbers."
+            "paraphrase stability, dense residual linear probes, single-feature causal edits, and v0.3 "
+            "top-k feature-set ablations against norm-matched controls. Target outcomes use **full continuation "
+            "teacher-forced log probability**, with mean log p/token used for length-comparable aggregate tests. "
+            "Results are loaded from `artifacts/`; the app never ships invented benchmark numbers."
         )
 
     with gr.Tab("Method"):
@@ -449,24 +816,36 @@ with gr.Blocks(title="FeatureLens — Causal Interpretability Workbench") as dem
             r"""
 ### Reconstruction-preserving intervention
 
-For residual vector $h$, selected sparse coefficient $z_i$, decoder direction $d_i$, and scale $\alpha$:
+For residual vector $h$, sparse coefficient $z_i$, decoder direction $d_i$, and scale $\alpha$:
 
 - **Ablate:** $h' = h - z_i d_i$
 - **Scale:** $h' = h + (\alpha - 1) z_i d_i$
 - **Inject:** $h' = h + \delta d_i$
 
-The app edits the original residual rather than replacing it with the full SAE reconstruction, so SAE
-reconstruction error is not introduced as a causal confound.
+For a feature set $S$, FeatureLens sums the individual ablation/scale deltas before patching the original residual:
+
+$$h' = h + \sum_{i \in S} \Delta z_i d_i.$$
+
+The app never replaces $h$ with the complete SAE reconstruction, so SAE reconstruction error is not introduced
+as a causal confound.
+
+### Full-continuation scoring
+
+For a user-supplied target continuation, v0.3 concatenates its token IDs to the prompt and computes teacher-forced
+log probabilities for **every target token**. The primary length-comparable live/offline statistic is the change in
+**mean log probability per target token**. The next-token distribution is still reported separately.
 
 ### Evidence ladder
 
 1. **Reconstruction:** does the SAE represent the residual reasonably well?
 2. **Prediction:** does a feature predict a controlled concept on held-out paraphrase groups?
-3. **Intervention:** does changing it alter downstream behaviour?
-4. **Specificity:** is that effect larger than a norm-matched random residual perturbation?
-5. **Dose-response:** does effect size change coherently as the feature coefficient is varied?
+3. **Robustness:** does the sparse representation remain stable across a paraphrase?
+4. **Single-feature intervention:** does changing one feature alter downstream behaviour?
+5. **Feature-set intervention:** does jointly editing a sparse subspace reveal distributed causal influence?
+6. **Specificity:** are those effects larger than norm-matched random residual perturbations?
+7. **Dose-response:** does effect size change coherently as a feature coefficient is varied?
 
-A high AUROC alone remains correlational evidence.
+A high AUROC or high paraphrase overlap alone remains correlational evidence.
 """
         )
 
@@ -475,8 +854,60 @@ A high AUROC alone remains correlational evidence.
         "FeatureLens is independent of thesis code and thesis datasets.</small>"
     )
 
+    # Event wiring is kept together so cross-tab state is explicit.
+    analyze_btn.click(
+        analyze_prompt,
+        inputs=[prompt, layer, token_index, top_n],
+        outputs=[
+            token_view,
+            feature_table,
+            feature_plot,
+            feature_id,
+            feature_set_ids,
+            analysis_metrics,
+        ],
+    )
+    mode.change(mode_help, inputs=[mode], outputs=[coefficient])
+    intervene_btn.click(
+        run_intervention,
+        inputs=[prompt, layer, token_index, feature_id, mode, coefficient, target_text, max_new],
+        outputs=[
+            baseline_out,
+            modified_out,
+            intervention_metrics,
+            token_prob_table,
+            target_token_table,
+        ],
+    )
+    dose_btn.click(
+        run_dose_response,
+        inputs=[prompt, layer, token_index, feature_id, target_text],
+        outputs=[dose_table, dose_plot, dose_metrics],
+    )
+    set_mode.change(set_mode_help, inputs=[set_mode], outputs=[set_coefficient])
+    set_btn.click(
+        run_feature_set,
+        inputs=[prompt, layer, token_index, feature_set_ids, set_mode, set_coefficient, set_target],
+        outputs=[set_feature_table, set_metrics, set_target_table],
+    )
+    set_sweep_btn.click(
+        run_feature_set_sweep,
+        inputs=[prompt, layer, token_index, set_sweep_target],
+        outputs=[set_sweep_table, set_sweep_plot, set_sweep_note],
+    )
+    para_btn.click(
+        run_paraphrase_compare,
+        inputs=[para_a, para_b, para_layer, para_idx_a, para_idx_b, para_top_n],
+        outputs=[para_tokens_a, para_tokens_b, para_metrics, para_table, para_plot],
+    )
+    trajectory_btn.click(
+        run_layer_sweep,
+        inputs=[trajectory_prompt, trajectory_token],
+        outputs=[trajectory_tokens, trajectory_table, trajectory_plot],
+    )
+
 if __name__ == "__main__":
-    # Explicit CSR avoids an HF/Gradio SSR auth-coroutine warning seen in 6.23.x-era Space builds.
+    # Explicit CSR avoids the Gradio SSR auth-coroutine warning seen in earlier Space builds.
     demo.queue(default_concurrency_limit=1, max_size=8).launch(
         css=CSS,
         theme=THEME,

artifacts/README.md

@@ -1,6 +1,6 @@
 # Generated artifacts
 
-This directory intentionally ships without invented results.
+This directory intentionally ships without invented empirical results.
 
 Run:
 
@@ -8,6 +8,16 @@ Run:
 python experiments/run_all.py
 ```
 
-to create activation caches, `feature_catalog.csv`, `layer_metrics.csv`, `stability.csv`, `causal_results.csv`, `summary.json`, `report.md`, and figures.
+to create:
 
-Large activation arrays are ignored by Git. Commit the small CSV/JSON/report/figure outputs if you want the live Space to show benchmark-derived feature hints and the benchmark summary.
+- activation caches;
+- `feature_catalog.csv`;
+- `layer_metrics.csv`;
+- `stability.csv`;
+- `causal_results.csv`;
+- `feature_set_results.csv`;
+- `summary.json`;
+- `report.md`;
+- report figures.
+
+Large residual/activation arrays are ignored by Git. Commit only small CSV/JSON/report/figure outputs if you want the live Space to display benchmark-derived feature hints and measured conclusions.

docs/HF_DEPLOY.md

@@ -1,24 +1,54 @@
 # Hugging Face deployment
 
-FeatureLens targets a Gradio SDK Space with ZeroGPU hardware.
+FeatureLens targets a **Gradio SDK Space** with ZeroGPU hardware.
 
-## Why Gradio rather than Docker?
+## Runtime shape
 
-ZeroGPU currently supports Gradio SDK Spaces. The workbench therefore uses `app.py` plus `requirements.txt` and avoids a Dockerfile.
+On Hugging Face, `FEATURELENS_EAGER_LOAD` defaults to `1`. The runtime loads:
 
-## Cold start
+- `Qwen/Qwen3-1.7B-Base`;
+- Qwen-Scope SAE layers **4, 14, 26** only.
 
-On Hugging Face, `FEATURELENS_EAGER_LOAD` defaults to `1`. The runtime loads Qwen3-1.7B-Base and only the configured Qwen-Scope layer files. The default layer set is 4, 14 and 26.
+The full 28-layer SAE repository is not required by the live app.
 
-The model and SAEs are placed on `cuda` during startup. ZeroGPU provides CUDA emulation outside GPU-decorated callbacks and a real GPU during `spaces.GPU` execution.
+`app.py` explicitly launches with `ssr_mode=False`. This avoids the SSR/auth path that produced the earlier `get_current_user was never awaited` warning during v0.2 deployment testing.
 
-## GPU-decorated functions
+## GPU-decorated actions
 
-- feature inspection: 45-second maximum allocation;
-- baseline-vs-modified generation: 60-second maximum allocation.
+Current v0.3 callback allocations:
 
-Keeping prompt length and generation length bounded protects the free daily quota and improves queue behavior.
+- **Inspect sparse features** — 30 s;
+- **Run single-feature causal test** — 45 s;
+- **Run scale dose-response** — 35 s;
+- **Run joint feature-set causal test** — 35 s;
+- **Run 1/3/5-feature ablation sweep** — 35 s;
+- **Compare paraphrase representations** — 30 s;
+- **Compare layers** — 35 s.
 
-## No persistent benchmark jobs in the Space
+These durations are allocation ceilings, not expected runtimes.
 
-The full experiment is intentionally offline. A free ZeroGPU quota is appropriate for interactive inspection, not a hundreds-of-forward-passes benchmark. Run `python experiments/run_all.py` elsewhere once, commit the small report/catalog artifacts, and the Space will display them automatically.
+## Batched interventions
+
+v0.3 deliberately batches experiment conditions inside one GPU callback:
+
+- all six dose-response multipliers share one edited forward after the baseline;
+- targeted/random conditions for k=1/3/5 feature-set ablation share one edited forward after the baseline;
+- full-target SAE and random-control scoring is batched when possible.
+
+This keeps the stronger diagnostics compatible with a quota-limited interactive Space.
+
+## Greedy generation vs target scoring
+
+The single-feature causal test still runs baseline and edited greedy generation because the visible text comparison is useful for a demo. Probability-level evidence uses teacher-forced full-continuation scoring, because deterministic text can remain unchanged despite meaningful logit shifts.
+
+The feature-set and set-size panels intentionally avoid extra free-running generations and focus on causal probability metrics.
+
+## Offline benchmark
+
+Do **not** run the full research benchmark in the public Space. Run:
+
+```bash
+python experiments/run_all.py
+```
+
+on separate CUDA compute, then commit only the small report/catalog/CSV/figure artifacts you want the Space to display. Large activation arrays remain gitignored.

docs/METHODOLOGY.md

@@ -2,38 +2,100 @@
 
 ## Primary hypothesis
 
-A sparse feature that is predictive of a semantic category is not automatically a causal mechanism for behavior. FeatureLens evaluates association and intervention separately.
+A sparse feature that predicts a semantic category is not automatically a causal mechanism for behavior. FeatureLens therefore evaluates reconstruction, association, robustness, and causal intervention separately.
 
 ## Discovery data
 
-Seven controlled concepts are represented by 16 independently authored prompt pairs per concept. Each pair contains two paraphrases. The split is grouped by pair id so lexical near-duplicates never cross train and test.
+Seven controlled concepts are represented by 16 prompt pairs per concept. Each pair contains two paraphrases. The split is grouped by `pair_id`, so lexical near-duplicates never cross train and held-out test.
 
-The final token residual is used for the offline concept benchmark because it has access to the complete prompt prefix and keeps the activation tensor small enough to reproduce easily. The live workbench is token-selectable and is not restricted to the final token.
+The offline discovery benchmark uses the final prompt-token residual because it summarizes the complete prompt prefix and keeps cached tensors small. The live workbench remains token-selectable.
 
-## Sparse feature evaluation
+## Sparse feature selection
 
-For each configured layer, FeatureLens stores the TopK SAE code for each sample. Candidate features must fire at least three times on the training split. Features are ranked using training AUROC and an activation-rate contrast tie-break. AUROC and F1 shown in the report are then calculated on the held-out split.
+For each configured residual layer, FeatureLens stores the TopK SAE code. Candidate features must fire often enough on the training split. They are ranked using training AUROC with activation-rate contrast as a tie-break.
 
-This makes the feature catalog a selection procedure rather than a post-hoc leaderboard over the test set.
+Held-out AUROC and F1 are computed **after** selection. The test split is not used to pick the winning feature.
 
 ## Dense baseline
 
-A multinomial logistic-regression probe is fit to the dense residual vectors at the same layers. This asks whether the representation carries concept information even if no single SAE feature isolates it cleanly.
+A multinomial logistic-regression probe is fit to dense residual vectors at the same layers. This asks whether concept information exists in the representation even when no individual sparse feature isolates it cleanly.
 
-## Causal intervention
+## Paraphrase robustness
 
-If the SAE code of the selected prompt residual is `z` and the selected feature is `i`, FeatureLens changes only coefficient `z_i`. The resulting decoder delta is added back to the original residual. Full SAE reconstruction is never substituted for the original activation.
+Sparse representation stability is measured with:
 
-Ablation uses `Δz_i = -z_i`; 2× amplification uses `Δz_i = z_i`.
+- TopK support Jaccard;
+- sparse activation cosine.
+
+The live **Paraphrase robustness** explorer computes the same style of diagnostics for a manually supplied prompt pair. High overlap is robustness evidence only; it does not establish semantic identity of individual feature IDs.
+
+## Reconstruction-preserving causal edit
+
+For residual `h`, selected SAE activation `z_i`, decoder direction `d_i`, and multiplier `α`:
+
+```text
+ablate:   h' = h - z_i d_i
+scale:    h' = h + (α - 1) z_i d_i
+inject:   h' = h + δ d_i
+```
+
+The delta is added to the **original residual**. FeatureLens never replaces the residual with the full SAE reconstruction, avoiding reconstruction error as a causal confound.
+
+## Full-continuation target scoring
+
+v0.3 scores the complete user/task target rather than only its first token.
+
+Prompt token IDs are concatenated with the exact target token IDs. A teacher-forced forward pass supplies a log probability for every target token. FeatureLens stores:
+
+- total target sequence log probability;
+- mean log probability per target token;
+- per-token log probabilities;
+- SAE-edit deltas;
+- matched-control deltas.
+
+The primary aggregate causal metric is **Δ mean target log probability per token** because it is comparable across targets with different token lengths.
+
+Next-token probability/rank and JS divergence remain secondary diagnostics.
 
 ## Negative control
 
-For every SAE residual perturbation, a deterministic random direction with identical L2 norm is generated and patched at the same layer/token. This does not prove specificity by itself, but it gives a much stronger baseline than reporting an intervention effect without a perturbation control.
+For every targeted residual perturbation, FeatureLens constructs a deterministic random residual direction with the same L2 norm and patches it at the same layer and prompt token.
+
+The paired targeted-vs-random comparison asks whether the chosen SAE direction matters more than an arbitrary perturbation of equal magnitude.
+
+## Single-feature dose-response
+
+The live dose-response experiment is explicitly a **scale sweep**:
+
+```text
+0×, 0.5×, 1×, 1.5×, 2×, 3×
+```
+
+`0×` is ablation and `1×` is the exact no-edit control. All six residual deltas are evaluated as one batch after the baseline.
+
+A monotonic curve would strengthen causal evidence, but monotonicity is not assumed. Non-monotonic responses are retained as results.
+
+## Feature-set intervention
+
+A concept may be distributed across multiple sparse features. For a same-layer feature set `S`, FeatureLens sums individual ablation/scale deltas:
+
+```text
+h' = h + Σ_i∈S Δz_i d_i
+```
+
+Features from different SAE layers are never summed into one residual intervention.
+
+The live app supports custom same-layer feature sets. The offline pipeline additionally selects a concept's best layer using training-only feature scores and tests its top **1 / 3 / 5** distinct features.
+
+## Feature-set size control
+
+For each k in `1, 3, 5`, targeted joint ablation is compared with a norm-matched random residual perturbation. Increasing k is **not** assumed to increase causal specificity: a larger feature set can simply create a larger perturbation, which is why matched controls and paired statistics remain necessary.
 
-## Causal outcomes
+## Statistical interpretation
 
-The causal task set is separate from feature discovery. The main behavioral score is the change in log-probability of the first token of a target completion. The pipeline also stores probability delta, target rank change, JS divergence of the next-token distribution, and whether the top-1 token changed.
+The generated report calculates:
 
-## Interpretation
+- bootstrap 95% confidence intervals;
+- paired sign-flip randomization tests for targeted-vs-random absolute effects.
 
-The auto-generated report uses conservative heuristics to summarize measured results, but preserves all raw rows. A high predictive AUROC with a small causal effect is explicitly reported as evidence that correlation did not translate into strong downstream control.
+A large point-estimate ratio alone is not treated as strong causal-specificity evidence if paired uncertainty does not support it. The report preserves raw rows regardless of the narrative classification.

docs/VALIDATION.md

@@ -1,76 +1,416 @@
-# FeatureLens validation matrix
+# FeatureLens v0.3 validation guide
 
-FeatureLens has two distinct validation targets: software correctness and scientific validity. Passing one does not imply the other.
+This document intentionally uses the **exact labels visible in the v0.3 Gradio UI**.
 
-## Before every push
+The goal is to distinguish three things:
+
+1. software correctness;
+2. Hugging Face / ZeroGPU deployment correctness;
+3. scientific sanity checks.
+
+Do not interpret one successful prompt as a scientific result. The live tests below only verify that the instrumentation behaves coherently.
+
+---
+
+## A. Local software gate
+
+From the repository root:
 
 ```bash
-python -m ruff check app.py featurelens experiments tests scripts
 python -m pytest -q
 python -m compileall -q app.py featurelens experiments scripts
 python scripts/release_check.py
 ```
 
-Also run `git status` and confirm that no `__pycache__/`, `pycache/`, `.pytest_cache/`, model weights, or `artifacts/activations/` files are staged.
+Expected:
+
+- all tests pass;
+- `compileall` exits without output/error;
+- release check ends with `FeatureLens release check: PASS` and `release: v0.3.0`.
 
-## Local UI smoke test
+Optional lint gate:
 
 ```bash
-python app.py
+python -m ruff check app.py featurelens experiments tests scripts
+```
+
+---
+
+## B. Hugging Face startup test
+
+### Test B1 — container startup
+
+Open the Space container logs after a fresh rebuild.
+
+Expected launch lines:
+
+```text
+* Running on local URL:  http://0.0.0.0:7860
+```
+
+The earlier warning below should **not** reappear:
+
+```text
+coroutine 'App.create_app.<locals>.get_current_user' was never awaited
+```
+
+The launch output should also not contain `with SSR ⚡` because `app.py` uses `ssr_mode=False`.
+
+Pass condition: Space reaches `Running` and the UI loads.
+
+---
+
+## C. Workbench inspection
+
+### Test C1 — inspect a mathematics prompt
+
+Open **Workbench**.
+
+Set:
+
+- **Prompt**: `The derivative of x squared is`
+- **Residual layer**: `14`
+- **Prompt token index**: `-1`
+- **Displayed active features**: `12`
+
+Click **Inspect sparse features**.
+
+Verify all of the following:
+
+1. **Prompt tokens** appears directly above token-like boxes. One token has a stronger outline; this is the selected prompt token.
+2. **Analysis metrics** appears and includes:
+   - active SAE features;
+   - reconstruction cosine;
+   - NMSE;
+   - Top-5 activation mass.
+3. **Strongest active SAE features** contains 12 rows unless fewer than 12 are available.
+4. **Activation profile** contains the same displayed feature IDs as the table.
+5. **Single feature id** is populated with those feature IDs.
+6. Open **Feature sets** and confirm **Feature set** is populated, with up to the first 3 displayed features selected by default.
+
+Sanity checks:
+
+- reconstruction cosine must be finite and normally lie in `[-1, 1]`;
+- NMSE must be finite and non-negative;
+- feature IDs must lie in `[0, 32767]`;
+- activations should be non-negative for the TopK ReLU SAE.
+
+---
+
+## D. Single-feature causal test
+
+Use the same Workbench prompt/location from Test C1.
+
+Choose the strongest value in **Single feature id**.
+
+### Test D1 — ablation with a multi-token target
+
+Set:
+
+- **Single-feature intervention**: `ablate`
+- **Target continuation (optional)**: `2x`
+- **Greedy generation length**: `8`
+
+Click **Run single-feature causal test**.
+
+Verify:
+
+1. **Baseline greedy generation** is populated.
+2. **SAE-edited greedy generation** is populated.
+3. **Next-token distribution shift** is populated.
+4. The metrics report:
+   - original feature activation;
+   - Δ coefficient;
+   - perturbation L2;
+   - next-token JS;
+   - random-control JS;
+   - full target continuation token count;
+   - baseline / SAE / random sequence log p;
+   - SAE and random Δ sequence log p;
+   - SAE and random Δ mean log p/token;
+   - specificity ratio.
+5. **Target continuation token-by-token score** has one row for every token in `2x`.
+
+Important expected behaviour:
+
+- ablation should report `Δ coefficient = -original activation` for an active feature;
+- targeted and random perturbation L2 norms should be matched internally;
+- greedy generations **may be identical** even when log-probability and JS metrics differ. This is not a failure.
+
+### Test D2 — scale
+
+Set:
+
+- **Single-feature intervention**: `scale`
+- **Feature multiplier**: `2`
+- keep **Target continuation (optional)**: `2x`
+
+Click **Run single-feature causal test**.
+
+Expected:
+
+- for an active feature, `Δ coefficient ≈ +original activation`;
+- full-continuation metrics are present;
+- output need not differ at the text level.
+
+### Test D3 — inject
+
+Set:
+
+- **Single-feature intervention**: `inject`
+- **Additive feature coefficient**: `5`
+- keep target `2x`.
+
+Expected:
+
+- `Δ coefficient = +5` regardless of the original feature activation;
+- perturbation L2 is finite;
+- full-continuation metrics are present.
+
+---
+
+## E. Single-feature scale dose-response
+
+Keep the Test C1 prompt/location, strongest **Single feature id**, and target `2x`.
+
+Open **Single-feature scale dose-response**.
+
+Click **Run scale dose-response**.
+
+This test has no ambiguous intervention mode: the panel is **always a scale sweep**.
+
+Expected rows in **Scale dose-response measurements**:
+
+```text
+0.0
+0.5
+1.0
+1.5
+2.0
+3.0
 ```
 
-Verify that the app opens without an SSR banner and without `get_current_user ... was never awaited` warnings. The app intentionally launches with `ssr_mode=False`.
-
-On a CUDA machine, test one example through each path:
-
-1. Inspect features on layer 14, final token.
-2. Ablate the strongest active feature.
-3. Supply a target continuation and confirm SAE-edit and random-control metrics both appear.
-4. Run the dose-response sweep and confirm the 1× row has approximately zero intervention effect.
-5. Run the layer trajectory and confirm rows for 4, 14 and 26.
-
-## Hugging Face Space smoke test
-
-After a push/rebuild:
-
-1. Confirm Space hardware is ZeroGPU and status reaches **Running**.
-2. Check startup logs for model/SAE download or load failures.
-3. Confirm the launch line does **not** say `with SSR`.
-4. Run `The derivative of x squared is` at layer 14, token `-1`.
-5. Confirm the feature dropdown is populated after inspection.
-6. Run an ablation with `target continuation = 2x`.
-7. Confirm baseline and modified generations render and random-control metrics appear.
-8. Run a dose-response sweep once; confirm the curve renders.
-9. Run the layer trajectory once; confirm all three layers render.
-10. Refresh the page and repeat a short inspection to catch state/cold-start regressions.
-
-## Scientific acceptance checks
-
-Do not publish headline findings until the offline benchmark is run on the real Qwen3/Qwen-Scope weights.
-
-Required checks:
-
-- paraphrase groups do not cross train/test splits;
-- feature selection uses training metrics only;
-- held-out AUROC/F1 are reported separately;
-- reconstruction metrics are reported per layer;
-- causal tasks are separate from discovery prompts;
-- SAE edits and random controls are paired by task/intervention;
-- perturbation norms match within numerical tolerance;
-- bootstrap confidence intervals are included;
-- paired sign-flip p-value is reported as uncertainty evidence, not as proof of a mechanistic claim;
-- raw causal rows remain available for inspection;
-- null or mixed results are retained in the generated report.
-
-## Recommended manual adversarial tests
-
-Use prompts designed to expose failure modes rather than only attractive examples:
-
-- a mathematics prompt with a code-like token;
-- English content containing one French entity name;
-- negation such as `This is not a positive review`;
-- an uncertainty prompt that becomes certain in its final clause;
-- a custom inactive feature ID with `ablate` versus `inject`;
-- a multi-token target continuation to verify the first-token warning;
-- token index `0`, `-1`, and an invalid out-of-range index;
-- feature ID `0`, `32767`, and an invalid `32768`.
+Interpretation of the multiplier:
+
+- `0×` = ablation;
+- `0.5×` = halve the original activation;
+- `1×` = no intervention;
+- `1.5×` = increase by 50%;
+- `2×` = double;
+- `3×` = triple.
+
+Critical no-op sanity check for the `1×` row:
+
+- `Δ feature coefficient ≈ 0`;
+- `Perturbation L2 ≈ 0`;
+- `Δ mean log p/token ≈ 0`;
+- `Δ sequence log p ≈ 0`;
+- `Next-token JS ≈ 0`.
+
+The curve need not be monotonic. A non-monotonic curve is a scientific observation, not a software error, provided the `1×` no-op row is correct.
+
+---
+
+## F. Joint feature-set causal test
+
+First run Test C1 so **Feature set** is populated.
+
+Open **Feature sets**.
+
+### Test F1 — top three joint ablation
+
+Use the default 3 selected values in **Feature set**.
+
+Set:
+
+- **Feature-set intervention**: `ablate`
+- **Target continuation**: `2x`
+
+Click **Run joint feature-set causal test**.
+
+Verify:
+
+1. **Joint intervention features** contains the selected feature IDs, original activations, and each Δ coefficient.
+2. Under ablation, each active feature's Δ coefficient is the negative of its original activation.
+3. The metrics report:
+   - selected set size;
+   - total joint perturbation L2;
+   - full target sequence / mean-per-token effects;
+   - matched random-control effects;
+   - specificity ratios.
+4. **Target continuation token-by-token score** contains all target tokens.
+
+### Test F2 — joint scale
+
+Set:
+
+- **Feature-set intervention**: `scale`
+- **Shared feature multiplier**: `2`
+
+Run again.
+
+Expected: every selected active feature receives its own `+original activation` coefficient delta before the decoder deltas are summed.
+
+---
+
+## G. 1/3/5-feature ablation sweep
+
+Still in **Feature sets**.
+
+Set **Target continuation for set-size sweep** to `2x`.
+
+Click **Run 1/3/5-feature ablation sweep**.
+
+Expected **Feature-set size measurements** rows:
+
+```text
+1
+3
+5
+```
+
+For each row verify:
+
+- the feature ID list contains exactly `k` IDs;
+- perturbation L2 is finite;
+- SAE and random Δ mean log p/token are present;
+- specificity ratio is finite unless the random effect is numerically zero;
+- SAE and random next-token JS are present.
+
+This sweep always performs **joint ablation of the strongest active features** at the current Workbench prompt location. It does not use **Feature-set intervention** or **Shared feature multiplier**.
+
+Scientific sanity check: effect magnitude does **not** have to increase with k. If k=5 is weaker or less specific than k=1, retain that result.
+
+---
+
+## H. Paraphrase robustness
+
+Open **Paraphrase robustness**.
+
+Set:
+
+- **Original prompt**: `The derivative of x squared is`
+- **Paraphrase**: `Differentiate x squared with respect to x:`
+- **Residual layer**: `14`
+- **Original prompt token index**: `-1`
+- **Paraphrase token index**: `-1`
+- **Displayed active features**: `12`
+
+Click **Compare paraphrase representations**.
+
+Verify:
+
+1. **Original prompt tokens** and **Paraphrase tokens** both render.
+2. **Robustness metrics** reports:
+   - full TopK feature-set Jaccard;
+   - sparse activation cosine;
+   - shared displayed top features.
+3. **Top-feature overlap** labels features as `shared`, `original only`, or `paraphrase only`.
+4. **Original vs paraphrase activation** renders both conditions with distinct muted colors.
+
+Bounds:
+
+- Jaccard must lie in `[0, 1]`;
+- sparse cosine should lie approximately in `[0, 1]` because SAE TopK activations are non-negative.
+
+Repeat with a deliberately weak/non-paraphrase second prompt. The metrics should be allowed to decrease; there is no hard-coded expected threshold.
+
+---
+
+## I. Layer trajectory
+
+Open **Layer trajectory**.
+
+Set:
+
+- **Prompt**: `The derivative of x squared is`
+- **Prompt token index**: `-1`
+
+Click **Compare layers**.
+
+Expected:
+
+- **Prompt tokens** renders;
+- **Layer diagnostics** contains rows `4`, `14`, `26`;
+- **Representation trajectory** contains:
+  - Reconstruction cosine;
+  - Top-5 mass;
+  - Activation entropy.
+
+Do not compare feature ID numbers across layers. Each layer uses a separate SAE dictionary.
+
+---
+
+## J. Adversarial / edge-case tests
+
+These are intended to find bugs, not produce attractive screenshots.
+
+### J1 — invalid token index
+
+Set **Prompt token index** to an index larger than the prompt length and click **Inspect sparse features**.
+
+Expected: a clear UI error explaining the token index is outside the prompt length.
+
+### J2 — inactive custom feature
+
+After inspection, enter a custom **Single feature id** that is not active at the selected token and choose `ablate`.
+
+Expected: FeatureLens warns that ablation/scaling produces a zero feature delta for an inactive feature.
+
+Then switch to `inject` with coefficient `5`.
+
+Expected: a non-zero decoder-direction perturbation is still possible.
+
+### J3 — negation prompt
+
+Inspect:
+
+```text
+This is not a positive review.
+```
+
+Do not assume a positive-sentiment feature should dominate merely because the word `positive` appears.
+
+### J4 — mixed language
+
+Inspect:
+
+```text
+The answer est probablement correct, but I am not certain.
+```
+
+Use this only as a robustness/adversarial check. Do not retrofit semantic labels from one example.
+
+### J5 — target whitespace
+
+Compare target continuations `2x` and ` 2x`.
+
+They may tokenize differently. v0.3 intentionally scores the exact text entered in **Target continuation (optional)**.
+
+---
+
+## K. Scientific benchmark acceptance
+
+After the live Space is stable, run:
+
+```bash
+python experiments/run_all.py
+```
+
+Expected new v0.3 artifact:
+
+```text
+artifacts/feature_set_results.csv
+```
+
+The generated report should use `target_mean_logprob_delta` when the v0.3 causal results are present and should include `figures/feature_set_effects.png`.
+
+Before using numbers on a resume or portfolio page, manually inspect:
+
+- held-out feature selection discipline;
+- paraphrase split integrity;
+- single-feature SAE vs random paired rows;
+- feature-set SAE vs random paired rows;
+- inactive-feature rate;
+- confidence intervals / sign-flip tests;
+- whether any headline claim is actually supported by the saved raw rows.

experiments/make_report.py

@@ -24,9 +24,7 @@ def parse_args() -> argparse.Namespace:
 
 def _selected_features(catalog: pd.DataFrame) -> pd.DataFrame:
     scored = catalog.copy()
-    scored['activation_contrast'] = (
-        scored['activation_rate_pos'] - scored['activation_rate_neg']
-    )
+    scored['activation_contrast'] = scored['activation_rate_pos'] - scored['activation_rate_neg']
     ordered = scored.sort_values(
         ['concept', 'train_auroc', 'activation_contrast'],
         ascending=[True, False, False],
@@ -34,7 +32,20 @@ def _selected_features(catalog: pd.DataFrame) -> pd.DataFrame:
     return ordered.groupby('concept', as_index=False).first()
 
 
-def _save_plots(artifact_dir: Path, selected: pd.DataFrame, layers: pd.DataFrame, causal: pd.DataFrame) -> None:
+def _effect_column(frame: pd.DataFrame) -> str:
+    """Prefer the v0.3 length-normalized full-continuation metric, with v0.2 fallback."""
+    if 'target_mean_logprob_delta' in frame.columns:
+        return 'target_mean_logprob_delta'
+    return 'target_logprob_delta'
+
+
+def _save_plots(
+    artifact_dir: Path,
+    selected: pd.DataFrame,
+    layers: pd.DataFrame,
+    causal: pd.DataFrame,
+    feature_sets: pd.DataFrame | None,
+) -> None:
     fig_dir = artifact_dir / 'figures'
     fig_dir.mkdir(parents=True, exist_ok=True)
 
@@ -51,8 +62,18 @@ def _save_plots(artifact_dir: Path, selected: pd.DataFrame, layers: pd.DataFrame
 
     figure = plt.figure(figsize=(7.0, 4.2))
     ax = figure.add_subplot(111)
-    ax.plot(layers['layer'], layers['linear_probe_macro_auroc'], marker='o', label='Linear probe AUROC')
-    ax.plot(layers['layer'], layers['reconstruction_cosine'], marker='o', label='SAE reconstruction cosine')
+    ax.plot(
+        layers['layer'],
+        layers['linear_probe_macro_auroc'],
+        marker='o',
+        label='Linear probe AUROC',
+    )
+    ax.plot(
+        layers['layer'],
+        layers['reconstruction_cosine'],
+        marker='o',
+        label='SAE reconstruction cosine',
+    )
     ax.set_xlabel('Layer')
     ax.set_ylim(0, 1.05)
     ax.set_title('Layer-wise representation diagnostics')
@@ -61,22 +82,81 @@ def _save_plots(artifact_dir: Path, selected: pd.DataFrame, layers: pd.DataFrame
     figure.savefig(fig_dir / 'layer_diagnostics.png', dpi=160)
     plt.close(figure)
 
+    causal_metric = _effect_column(causal)
     grouped = (
-        causal.groupby(['intervention', 'condition'])['target_logprob_delta']
+        causal.groupby(['intervention', 'condition'])[causal_metric]
         .apply(lambda values: float(np.mean(np.abs(values))))
-        .reset_index(name='mean_abs_delta_logp')
+        .reset_index(name='mean_abs_effect')
     )
-    pivot = grouped.pivot(index='intervention', columns='condition', values='mean_abs_delta_logp')
+    pivot = grouped.pivot(index='intervention', columns='condition', values='mean_abs_effect')
     figure = plt.figure(figsize=(7.0, 4.2))
     ax = figure.add_subplot(111)
     pivot.plot(kind='bar', ax=ax)
-    ax.set_ylabel('Mean |Δ log p(target)|')
-    ax.set_title('SAE interventions vs norm-matched random controls')
+    ax.set_ylabel('Mean |Δ mean log p/token|' if causal_metric == 'target_mean_logprob_delta' else 'Mean |Δ log p(target)|')
+    ax.set_title('Single-feature SAE edits vs norm-matched controls')
     ax.tick_params(axis='x', rotation=0)
     figure.tight_layout()
     figure.savefig(fig_dir / 'causal_effects.png', dpi=160)
     plt.close(figure)
 
+    if feature_sets is not None and not feature_sets.empty:
+        set_metric = _effect_column(feature_sets)
+        grouped_sets = (
+            feature_sets.groupby(['set_size', 'condition'])[set_metric]
+            .apply(lambda values: float(np.mean(np.abs(values))))
+            .reset_index(name='mean_abs_effect')
+        )
+        set_pivot = grouped_sets.pivot(index='set_size', columns='condition', values='mean_abs_effect')
+        figure = plt.figure(figsize=(7.0, 4.2))
+        ax = figure.add_subplot(111)
+        set_pivot.plot(kind='line', marker='o', ax=ax)
+        ax.set_xlabel('Jointly ablated feature count')
+        ax.set_ylabel('Mean |Δ mean log p/token|' if set_metric == 'target_mean_logprob_delta' else 'Mean |Δ log p(target)|')
+        ax.set_title('Distributed feature-set causal effect')
+        figure.tight_layout()
+        figure.savefig(fig_dir / 'feature_set_effects.png', dpi=160)
+        plt.close(figure)
+
+
+def _paired_stats(
+    frame: pd.DataFrame,
+    *,
+    index: list[str],
+    sae_condition: str,
+    random_condition: str,
+    seed: int,
+) -> dict[str, float | list[float]]:
+    metric = _effect_column(frame)
+    paired = frame.pivot_table(
+        index=index,
+        columns='condition',
+        values=metric,
+        aggfunc='first',
+    ).dropna(subset=[sae_condition, random_condition])
+    sae_abs = np.abs(paired[sae_condition].to_numpy(dtype=float))
+    random_abs = np.abs(paired[random_condition].to_numpy(dtype=float))
+    if sae_abs.size == 0:
+        return {
+            'sae_abs': float('nan'),
+            'random_abs': float('nan'),
+            'ratio': float('nan'),
+            'paired_advantage': float('nan'),
+            'ci': [float('nan'), float('nan')],
+            'pvalue': float('nan'),
+            'n_pairs': 0,
+        }
+    diff = sae_abs - random_abs
+    low, high = paired_bootstrap_difference_ci(sae_abs, random_abs, seed=seed)
+    return {
+        'sae_abs': float(np.mean(sae_abs)),
+        'random_abs': float(np.mean(random_abs)),
+        'ratio': float(np.mean(sae_abs) / max(float(np.mean(random_abs)), 1e-12)),
+        'paired_advantage': float(np.mean(diff)),
+        'ci': [float(low), float(high)],
+        'pvalue': float(paired_sign_flip_pvalue(sae_abs, random_abs, seed=seed + 1)),
+        'n_pairs': int(sae_abs.size),
+    }
+
 
 def main() -> None:
     args = parse_args()
@@ -84,8 +164,10 @@ def main() -> None:
     layers = pd.read_csv(args.artifact_dir / 'layer_metrics.csv')
     stability = pd.read_csv(args.artifact_dir / 'stability.csv')
     causal = pd.read_csv(args.artifact_dir / 'causal_results.csv')
+    feature_set_path = args.artifact_dir / 'feature_set_results.csv'
+    feature_sets = pd.read_csv(feature_set_path) if feature_set_path.exists() else None
     selected = _selected_features(catalog)
-    _save_plots(args.artifact_dir, selected, layers, causal)
+    _save_plots(args.artifact_dir, selected, layers, causal, feature_sets)
 
     mean_auc = float(selected['auroc'].mean())
     median_auc = float(selected['auroc'].median())
@@ -94,68 +176,120 @@ def main() -> None:
     mean_jaccard = float(stability['topk_jaccard'].mean())
     mean_sparse_cos = float(stability['sparse_cosine'].mean())
 
-    sae = causal[causal['condition'] == 'sae_feature']
-    random = causal[causal['condition'] == 'random_norm_matched']
-    sae_abs = float(np.mean(np.abs(sae['target_logprob_delta'])))
-    random_abs = float(np.mean(np.abs(random['target_logprob_delta'])))
-    ratio = sae_abs / max(random_abs, 1e-12)
-
-    paired = causal.pivot_table(
+    single = _paired_stats(
+        causal,
         index=['task_id', 'intervention'],
-        columns='condition',
-        values='target_logprob_delta',
-        aggfunc='first',
-    ).dropna(subset=['sae_feature', 'random_norm_matched'])
-    paired_sae = np.abs(paired['sae_feature'].to_numpy(dtype=float))
-    paired_random = np.abs(paired['random_norm_matched'].to_numpy(dtype=float))
-    diff_mean = float(np.mean(paired_sae - paired_random))
-    diff_ci_low, diff_ci_high = paired_bootstrap_difference_ci(
-        paired_sae, paired_random, seed=43
+        sae_condition='sae_feature',
+        random_condition='random_norm_matched',
+        seed=43,
     )
-    sign_flip_p = paired_sign_flip_pvalue(paired_sae, paired_random, seed=44)
-
+    sae = causal[causal['condition'] == 'sae_feature']
     active_rate = float(np.mean(sae['feature_activation'] > 0))
     top1_change = float(sae['top1_changed'].mean())
 
+    set_summary: dict[int, dict[str, float | list[float]]] = {}
+    largest_set: dict[str, float | list[float]] | None = None
+    largest_k: int | None = None
+    if feature_sets is not None and not feature_sets.empty:
+        for size in sorted(int(x) for x in feature_sets['set_size'].unique()):
+            subset = feature_sets[feature_sets['set_size'] == size]
+            set_summary[size] = _paired_stats(
+                subset,
+                index=['task_id', 'set_size'],
+                sae_condition='sae_feature_set',
+                random_condition='random_norm_matched',
+                seed=100 + size,
+            )
+        largest_k = max(set_summary)
+        largest_set = set_summary[largest_k]
+
+    sae_abs = float(single['sae_abs'])
+    random_abs = float(single['random_abs'])
+    ratio = float(single['ratio'])
+
+    single_ci_low = float(single['ci'][0])
+    single_p = float(single['pvalue'])
+    single_specific = ratio >= 1.5 and single_ci_low > 0.0 and single_p < 0.05
+
     if mean_auc >= 0.8 and sae_abs < 0.08:
         interpretation = (
-            'The selected sparse features were strongly predictive on held-out prompts, but their '
-            'interventions produced only modest downstream target-probability changes. This supports '
-            'the project’s central warning that representation-level correlation need not imply strong '
-            'causal control.'
+            'The selected sparse features were strongly predictive on held-out prompts, but single-feature '
+            'interventions produced only modest downstream changes. FeatureLens therefore treats the '
+            'representation-level signal as correlational rather than automatically causal.'
+        )
+    elif mean_auc >= 0.8 and sae_abs >= 0.08 and single_specific:
+        interpretation = (
+            'The selected sparse features were strongly predictive and single-feature interventions produced '
+            'larger target-continuation shifts than norm-matched random residual perturbations. The paired '
+            'bootstrap interval excludes zero and the sign-flip test passes the configured 0.05 threshold, '
+            'supporting a causal-specificity claim for at least some predictive features.'
         )
     elif mean_auc >= 0.8 and sae_abs >= 0.08 and ratio >= 1.5:
         interpretation = (
-            'The selected sparse features were strongly predictive and their interventions produced '
-            'larger target-probability shifts than norm-matched random residual perturbations, providing '
-            'evidence that at least some predictive features also have downstream causal influence.'
+            'The selected sparse features were strongly predictive and their point-estimate intervention '
+            'effects exceeded norm-matched random controls, but the paired uncertainty test does not support '
+            'a strong causal-specificity claim at the 0.05 threshold. The result is reported as suggestive '
+            'rather than conclusive.'
         )
     elif mean_auc < 0.65:
         interpretation = (
-            'Feature/concept predictiveness was limited on held-out prompts, so strong causal claims '
-            'would be premature. The main result is diagnostic: the chosen concepts or feature-selection '
-            'procedure should be refined before interpreting intervention effects.'
+            'Feature/concept predictiveness was limited on held-out prompts, so strong causal claims would '
+            'be premature. The main result is diagnostic: concept design or feature selection should be '
+            'refined before interpreting intervention effects.'
         )
     else:
         interpretation = (
-            'The results show mixed predictive and causal evidence. FeatureLens therefore reports the '
-            'association and intervention measurements separately rather than collapsing them into a '
-            'single interpretability score.'
+            'The results show mixed predictive and causal evidence. FeatureLens reports association, '
+            'robustness, and intervention measurements separately rather than collapsing them into one score.'
         )
 
+    if largest_set is not None and largest_k is not None:
+        set_advantage = float(largest_set['paired_advantage'])
+        set_ci_low = float(largest_set['ci'][0])
+        set_p = float(largest_set['pvalue'])
+        set_specific = set_ci_low > 0.0 and set_p < 0.05
+        if set_advantage > float(single['paired_advantage']) + 0.02 and set_specific:
+            interpretation += (
+                f' Joint ablation of the top {largest_k} same-layer concept features produced a larger '
+                'paired advantage over random controls than the single-feature edits, with paired uncertainty '
+                'supporting the difference. This is consistent with causal influence being distributed across '
+                'a sparse feature set rather than concentrated in one unit.'
+            )
+        elif set_advantage > float(single['paired_advantage']) + 0.02:
+            interpretation += (
+                f' The top-{largest_k} joint-ablation point estimate exceeded the single-feature advantage, '
+                'but its paired uncertainty test does not support a strong distributed-causality claim at the '
+                '0.05 threshold. The pattern is therefore treated as suggestive only.'
+            )
+        elif abs(set_advantage) <= 0.02:
+            interpretation += (
+                f' Expanding the intervention to the top {largest_k} same-layer features did not materially '
+                'increase specificity over random controls, which argues against assuming that a broader '
+                'concept-associated sparse subspace is automatically more causal.'
+            )
+
+    effect_label = 'mean log p/token' if _effect_column(causal) == 'target_mean_logprob_delta' else 'target log-probability'
     headline = (
-        f'Selected SAE features averaged {mean_auc:.3f} held-out AUROC; SAE interventions changed '
-        f'target log-probability by {sae_abs:.3f} on average versus {random_abs:.3f} for norm-matched '
-        'random residual controls.'
+        f'Selected SAE features averaged {mean_auc:.3f} held-out AUROC; single-feature SAE interventions '
+        f'changed {effect_label} by {sae_abs:.3f} in absolute value on average versus {random_abs:.3f} '
+        'for norm-matched random residual controls.'
     )
     highlights = [
         f'Median selected-feature held-out AUROC: {median_auc:.3f}; mean AUROC 95% bootstrap CI [{auc_ci_low:.3f}, {auc_ci_high:.3f}].',
         f'Best residual linear-probe layer: {int(best_layer_row["layer"])} with macro AUROC {best_layer_row["linear_probe_macro_auroc"]:.3f}.',
         f'Mean paraphrase TopK Jaccard: {mean_jaccard:.3f}; sparse activation cosine: {mean_sparse_cos:.3f}.',
-        f'Selected feature active on {active_rate:.1%} of causal prompts; modified top-1 token on {top1_change:.1%}.',
-        f'Mean absolute causal effect / random-control effect ratio: {ratio:.2f}×.',
-        f'Paired mean |Δlog p| advantage over random: {diff_mean:+.3f}, 95% bootstrap CI [{diff_ci_low:+.3f}, {diff_ci_high:+.3f}], sign-flip p={sign_flip_p:.4f}.',
+        f'Selected feature active on {active_rate:.1%} of causal prompts; modified next-token top-1 on {top1_change:.1%}.',
+        f'Single-feature mean absolute causal effect / random-control effect ratio: {ratio:.2f}×.',
+        f'Single-feature paired mean |effect| advantage over random: {float(single["paired_advantage"]):+.3f}, 95% bootstrap CI [{float(single["ci"][0]):+.3f}, {float(single["ci"][1]):+.3f}], sign-flip p={float(single["pvalue"]):.4f}.',
     ]
+    if largest_set is not None and largest_k is not None:
+        highlights.append(
+            f'Top-{largest_k} joint ablation: SAE/random mean absolute effect ratio {float(largest_set["ratio"]):.2f}×; '
+            f'paired advantage {float(largest_set["paired_advantage"]):+.3f}, 95% CI '
+            f'[{float(largest_set["ci"][0]):+.3f}, {float(largest_set["ci"][1]):+.3f}], '
+            f'sign-flip p={float(largest_set["pvalue"]):.4f}.'
+        )
+
     summary = {
         'headline': headline,
         'highlights': highlights,
@@ -168,14 +302,16 @@ def main() -> None:
             'best_linear_probe_macro_auroc': float(best_layer_row['linear_probe_macro_auroc']),
             'mean_paraphrase_topk_jaccard': mean_jaccard,
             'mean_paraphrase_sparse_cosine': mean_sparse_cos,
-            'mean_abs_sae_target_logprob_delta': sae_abs,
-            'mean_abs_random_target_logprob_delta': random_abs,
+            'single_feature_effect_metric': _effect_column(causal),
+            'mean_abs_sae_effect': sae_abs,
+            'mean_abs_random_effect': random_abs,
             'causal_to_random_effect_ratio': ratio,
-            'paired_mean_abs_effect_advantage': diff_mean,
-            'paired_mean_abs_effect_advantage_bootstrap_ci_95': [diff_ci_low, diff_ci_high],
-            'paired_sign_flip_pvalue': sign_flip_p,
+            'paired_mean_abs_effect_advantage': float(single['paired_advantage']),
+            'paired_mean_abs_effect_advantage_bootstrap_ci_95': single['ci'],
+            'paired_sign_flip_pvalue': float(single['pvalue']),
             'causal_prompt_feature_active_rate': active_rate,
             'sae_top1_change_rate': top1_change,
+            'feature_set_results': {str(k): value for k, value in set_summary.items()},
         },
     }
     (args.artifact_dir / 'summary.json').write_text(json.dumps(summary, indent=2), encoding='utf-8')
@@ -200,15 +336,17 @@ def main() -> None:
         '## Experimental design',
         '',
         '- Model: Qwen3-1.7B-Base.',
-        '- SAEs: Qwen-Scope residual-stream TopK SAEs at the configured early/middle/late layers.',
+        '- SAEs: Qwen-Scope residual-stream TopK SAEs at configured early/middle/late layers.',
         '- Discovery set: controlled concept prompts with paired paraphrases.',
-        '- Split discipline: paraphrase groups are kept entirely in train or held-out test.',
+        '- Split discipline: paraphrase groups stay entirely in train or held-out test.',
         '- Feature selection: training-split AUROC and activation contrast; held-out AUROC/F1 are reported separately.',
         '- Linear baseline: multinomial logistic regression on the dense residual stream.',
-        '- Causal edit: reconstruction-preserving decoder-direction delta patched into the original residual.',
-        '- Negative control: deterministic random residual direction matched to the SAE perturbation L2 norm.',
-        '- Uncertainty: bootstrap 95% confidence intervals and a paired sign-flip randomization test for SAE-vs-control effect differences.',
-        '- Behavioural metric: first-token target probability/log-probability, rank, JS divergence, and top-1 changes.',
+        '- Single-feature causal edit: reconstruction-preserving decoder-direction delta patched into the original residual.',
+        '- Feature-set causal edit: joint ablation of top same-layer concept features, evaluated at k=1/3/5 by default.',
+        '- Negative control: deterministic random residual direction matched to each SAE perturbation L2 norm.',
+        '- Target metric: exact full target continuation scored teacher-forced; mean log probability per target token is the primary length-comparable effect.',
+        '- Secondary diagnostics: first-token probability/rank, next-token JS divergence, and top-1 changes.',
+        '- Uncertainty: bootstrap 95% confidence intervals and paired sign-flip randomization tests.',
         '',
         '## Figures',
         '',
@@ -216,17 +354,23 @@ def main() -> None:
         '',
         '![Layer diagnostics](figures/layer_diagnostics.png)',
         '',
-        '![Causal effects](figures/causal_effects.png)',
-        '',
-        '## Interpretation guardrails',
-        '',
-        'A high feature/concept AUROC is treated as correlational evidence only. Causal evidence requires a downstream change under intervention and is interpreted relative to the norm-matched random control. The narrative above is generated from saved metrics; no result values are hard-coded.',
-        '',
-        '## Reproducibility',
-        '',
-        'Run `python experiments/run_all.py` from the repository root. Raw activation matrices, splits, selected features, causal rows, figures, and this report are all materialized under `artifacts/`.',
-        '',
+        '![Single-feature causal effects](figures/causal_effects.png)',
     ]
+    if feature_sets is not None and not feature_sets.empty:
+        lines.extend(['', '![Feature-set causal effects](figures/feature_set_effects.png)'])
+    lines.extend(
+        [
+            '',
+            '## Interpretation guardrails',
+            '',
+            'A high feature/concept AUROC or high paraphrase overlap is correlational evidence only. Causal evidence requires a downstream change under intervention and is interpreted relative to a norm-matched random control. Feature-set effects are not assumed stronger a priori; they are separately measured. The narrative above is generated from saved metrics, with no hard-coded result values.',
+            '',
+            '## Reproducibility',
+            '',
+            'Run `python experiments/run_all.py` from the repository root. Raw activation matrices, splits, selected features, single-feature rows, feature-set rows, figures, and this report are materialized under `artifacts/`.',
+            '',
+        ]
+    )
     (args.artifact_dir / 'report.md').write_text('\n'.join(lines), encoding='utf-8')
     print(headline)
     print(f'Wrote {args.artifact_dir / "report.md"}')

experiments/run_all.py

@@ -19,6 +19,7 @@ def main() -> None:
     run('collect_activations.py')
     run('evaluate_features.py')
     run('run_causal.py')
+    run('run_feature_sets.py')
     run('make_report.py')
     print('\nFeatureLens experiment pipeline complete. See artifacts/report.md')
 

experiments/run_causal.py

@@ -11,7 +11,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 from experiments.common import ARTIFACT_DIR, DATA_DIR, load_jsonl, set_seed
 from featurelens.config import SETTINGS
 from featurelens.interventions import InterventionSpec, normalized_random_control, residual_delta
-from featurelens.metrics import js_divergence_from_logits, safe_log_probability
+from featurelens.metrics import js_divergence_from_logits, sequence_logprob_summary
 from featurelens.sae import SAEStore
 
 
@@ -25,7 +25,6 @@ def parse_args() -> argparse.Namespace:
 
 
 def load_selected_features(path: Path) -> dict[str, dict]:
-    rows = []
     with path.open(newline='', encoding='utf-8') as handle:
         rows = list(csv.DictReader(handle))
     selected: dict[str, dict] = {}
@@ -54,6 +53,18 @@ def replace_hidden(output, hidden):
     return (hidden, *output[1:]) if isinstance(output, tuple) else hidden
 
 
+def append_target(inputs: dict[str, torch.Tensor], target_ids: list[int]) -> dict[str, torch.Tensor]:
+    prompt_ids = inputs['input_ids']
+    target = torch.tensor(target_ids, dtype=prompt_ids.dtype, device=prompt_ids.device).unsqueeze(0)
+    full_ids = torch.cat([prompt_ids, target], dim=1)
+    attention = inputs.get('attention_mask', torch.ones_like(prompt_ids))
+    target_mask = torch.ones((1, len(target_ids)), dtype=attention.dtype, device=attention.device)
+    return {
+        'input_ids': full_ids,
+        'attention_mask': torch.cat([attention, target_mask], dim=1),
+    }
+
+
 @torch.inference_mode()
 def main() -> None:
     args = parse_args()
@@ -91,8 +102,14 @@ def main() -> None:
         layer = int(choice['layer'])
         feature_id = int(choice['feature_id'])
         sae = sae_store.get(layer)
-        inputs = tokenizer(task['prompt'], return_tensors='pt', truncation=True, max_length=192)
-        inputs = {key: value.to(device) for key, value in inputs.items()}
+        prompt_inputs = tokenizer(task['prompt'], return_tensors='pt', truncation=True, max_length=192)
+        prompt_inputs = {key: value.to(device) for key, value in prompt_inputs.items()}
+        prompt_len = int(prompt_inputs['input_ids'].shape[1])
+        target_ids = tokenizer(task['target'], add_special_tokens=False)['input_ids']
+        if not target_ids:
+            raise RuntimeError(f"Target tokenization empty for task {task['id']}")
+        target_ids = [int(x) for x in target_ids]
+        full_inputs = append_target(prompt_inputs, target_ids)
         capture: dict = {}
 
         def capture_hook(_module, _inp, output):
@@ -100,87 +117,125 @@ def main() -> None:
                 capture['hidden'] = hidden_from_output(output).detach()
 
         handle = model.model.layers[layer].register_forward_hook(capture_hook)
-        baseline_out = model(**inputs, use_cache=False)
+        baseline_out = model(**full_inputs, use_cache=False)
         handle.remove()
-        baseline_logits = baseline_out.logits[0, -1]
-        residual = capture['hidden'][0, -1]
+
+        baseline_logits = baseline_out.logits[0]
+        baseline_next = baseline_logits[prompt_len - 1]
+        baseline_seq, baseline_mean, _ = sequence_logprob_summary(
+            baseline_logits,
+            prompt_length=prompt_len,
+            target_ids=target_ids,
+        )
+        residual = capture['hidden'][0, prompt_len - 1]
         encoding = sae.encode(residual)
         original_activation = encoding.activation_for(feature_id)
-
-        target_ids = tokenizer(task['target'], add_special_tokens=False)['input_ids']
-        if not target_ids:
-            raise RuntimeError(f"Target tokenization empty for task {task['id']}")
-        target_id = int(target_ids[0])
-        baseline_prob = float(torch.softmax(baseline_logits.float(), dim=-1)[target_id].item())
-        baseline_rank = int((baseline_logits > baseline_logits[target_id]).sum().item()) + 1
-        baseline_top1 = int(torch.argmax(baseline_logits).item())
+        target_id = target_ids[0]
+        baseline_prob = float(torch.softmax(baseline_next.float(), dim=-1)[target_id].item())
+        baseline_rank = int((baseline_next > baseline_next[target_id]).sum().item()) + 1
+        baseline_top1 = int(torch.argmax(baseline_next).item())
 
         specs = [
             ('ablate', InterventionSpec('ablate', 0.0)),
             ('amplify_2x', InterventionSpec('scale', 2.0)),
         ]
-        for intervention_name, spec in specs:
+        condition_meta: list[tuple[str, str, InterventionSpec, torch.Tensor, float]] = []
+        for spec_idx, (intervention_name, spec) in enumerate(specs):
             delta = residual_delta(sae.decoder_direction(feature_id), original_activation, spec)
-            control_delta = normalized_random_control(delta, seed=args.seed + task_idx * 17 + len(results))
-
-            for condition, applied_delta in (
-                ('sae_feature', delta),
-                ('random_norm_matched', control_delta),
-            ):
-                applied = {'done': False}
-
-                def edit_hook(_module, _inp, output, *, edit=applied_delta, state=applied):
-                    if state['done']:
-                        return output
-                    hidden = hidden_from_output(output)
-                    modified = hidden.clone()
-                    modified[:, -1, :] = modified[:, -1, :] + edit.to(hidden.device, hidden.dtype)
-                    state['done'] = True
-                    return replace_hidden(output, modified)
-
-                hook = model.model.layers[layer].register_forward_hook(edit_hook)
-                modified_out = model(**inputs, use_cache=False)
-                hook.remove()
-                modified_logits = modified_out.logits[0, -1]
-                modified_prob = float(
-                    torch.softmax(modified_logits.float(), dim=-1)[target_id].item()
-                )
-                modified_rank = int((modified_logits > modified_logits[target_id]).sum().item()) + 1
-                modified_top1 = int(torch.argmax(modified_logits).item())
-                results.append(
-                    {
-                        'task_id': task['id'],
-                        'concept': concept,
-                        'prompt': task['prompt'],
-                        'target_text': task['target'],
-                        'target_first_token': tokenizer.decode([target_id]),
-                        'target_token_count': len(target_ids),
-                        'layer': layer,
-                        'feature_id': feature_id,
-                        'feature_train_auroc': choice['train_auroc'],
-                        'feature_test_auroc': choice['test_auroc'],
-                        'feature_test_f1': choice['test_f1'],
-                        'feature_activation': original_activation,
-                        'intervention': intervention_name,
-                        'condition': condition,
-                        'delta_activation': spec.delta_activation(original_activation)
-                        if condition == 'sae_feature'
-                        else math.nan,
-                        'perturbation_l2': float(torch.linalg.vector_norm(applied_delta.float()).item()),
-                        'baseline_target_prob': baseline_prob,
-                        'modified_target_prob': modified_prob,
-                        'target_prob_delta': modified_prob - baseline_prob,
-                        'target_logprob_delta': safe_log_probability(modified_prob)
-                        - safe_log_probability(baseline_prob),
-                        'baseline_target_rank': baseline_rank,
-                        'modified_target_rank': modified_rank,
-                        'target_rank_delta': modified_rank - baseline_rank,
-                        'js_divergence': js_divergence_from_logits(
-                            baseline_logits, modified_logits
-                        ),
-                        'top1_changed': int(modified_top1 != baseline_top1),
-                    }
-                )
+            control_delta = normalized_random_control(
+                delta,
+                seed=args.seed + task_idx * 101 + spec_idx,
+            )
+            condition_meta.extend(
+                [
+                    (
+                        intervention_name,
+                        'sae_feature',
+                        spec,
+                        delta,
+                        float(spec.delta_activation(original_activation)),
+                    ),
+                    (
+                        intervention_name,
+                        'random_norm_matched',
+                        spec,
+                        control_delta,
+                        math.nan,
+                    ),
+                ]
+            )
+
+        repeated = {key: value.repeat(len(condition_meta), 1) for key, value in full_inputs.items()}
+        deltas = torch.stack([item[3] for item in condition_meta], dim=0)
+        applied = {'done': False}
+
+        def batch_edit_hook(_module, _inp, output):
+            if applied['done']:
+                return output
+            hidden = hidden_from_output(output)
+            modified = hidden.clone()
+            modified[:, prompt_len - 1, :] = (
+                modified[:, prompt_len - 1, :]
+                + deltas.to(hidden.device, hidden.dtype)
+            )
+            applied['done'] = True
+            return replace_hidden(output, modified)
+
+        hook = model.model.layers[layer].register_forward_hook(batch_edit_hook)
+        edited_out = model(**repeated, use_cache=False)
+        hook.remove()
+
+        for row_idx, (intervention_name, condition, spec, applied_delta, delta_activation) in enumerate(condition_meta):
+            modified_logits = edited_out.logits[row_idx]
+            modified_next = modified_logits[prompt_len - 1]
+            modified_prob = float(torch.softmax(modified_next.float(), dim=-1)[target_id].item())
+            modified_rank = int((modified_next > modified_next[target_id]).sum().item()) + 1
+            modified_top1 = int(torch.argmax(modified_next).item())
+            modified_seq, modified_mean, _ = sequence_logprob_summary(
+                modified_logits,
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
+            results.append(
+                {
+                    'task_id': task['id'],
+                    'concept': concept,
+                    'prompt': task['prompt'],
+                    'target_text': task['target'],
+                    'target_first_token': tokenizer.decode([target_id]),
+                    'target_token_count': len(target_ids),
+                    'layer': layer,
+                    'feature_id': feature_id,
+                    'feature_train_auroc': choice['train_auroc'],
+                    'feature_test_auroc': choice['test_auroc'],
+                    'feature_test_f1': choice['test_f1'],
+                    'feature_activation': original_activation,
+                    'intervention': intervention_name,
+                    'condition': condition,
+                    'delta_activation': delta_activation,
+                    'perturbation_l2': float(torch.linalg.vector_norm(applied_delta.float()).item()),
+                    # First-token metrics retained for backwards compatibility and local diagnostics.
+                    'baseline_target_prob': baseline_prob,
+                    'modified_target_prob': modified_prob,
+                    'target_prob_delta': modified_prob - baseline_prob,
+                    'target_logprob_delta': float(
+                        torch.log_softmax(modified_next.float(), dim=-1)[target_id].item()
+                        - torch.log_softmax(baseline_next.float(), dim=-1)[target_id].item()
+                    ),
+                    'baseline_target_rank': baseline_rank,
+                    'modified_target_rank': modified_rank,
+                    'target_rank_delta': modified_rank - baseline_rank,
+                    # v0.3 primary target metric: exact full continuation, teacher-forced.
+                    'baseline_target_sequence_logprob': baseline_seq,
+                    'modified_target_sequence_logprob': modified_seq,
+                    'target_sequence_logprob_delta': modified_seq - baseline_seq,
+                    'baseline_target_mean_logprob': baseline_mean,
+                    'modified_target_mean_logprob': modified_mean,
+                    'target_mean_logprob_delta': modified_mean - baseline_mean,
+                    'js_divergence': js_divergence_from_logits(baseline_next, modified_next),
+                    'top1_changed': int(modified_top1 != baseline_top1),
+                }
+            )
         print(f"Causal task {task_idx + 1}/{len(tasks)}: {concept}", flush=True)
 
     args.output.parent.mkdir(parents=True, exist_ok=True)

experiments/run_feature_sets.py

@@ -0,0 +1,201 @@
+from __future__ import annotations
+
+import argparse
+import csv
+from pathlib import Path
+
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+from experiments.common import ARTIFACT_DIR, DATA_DIR, load_jsonl, set_seed
+from featurelens.config import SETTINGS
+from featurelens.interventions import InterventionSpec, joint_residual_delta, normalized_random_control
+from featurelens.metrics import js_divergence_from_logits, sequence_logprob_summary
+from featurelens.sae import SAEStore
+from featurelens.selection import load_feature_sets
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description='Run top-k joint SAE feature-set ablations.')
+    parser.add_argument('--tasks', type=Path, default=DATA_DIR / 'causal_tasks.jsonl')
+    parser.add_argument('--catalog', type=Path, default=ARTIFACT_DIR / 'feature_catalog.csv')
+    parser.add_argument('--output', type=Path, default=ARTIFACT_DIR / 'feature_set_results.csv')
+    parser.add_argument('--sizes', type=int, nargs='+', default=[1, 3, 5])
+    parser.add_argument('--seed', type=int, default=42)
+    return parser.parse_args()
+
+
+def hidden_from_output(output):
+    return output[0] if isinstance(output, tuple) else output
+
+
+def replace_hidden(output, hidden):
+    return (hidden, *output[1:]) if isinstance(output, tuple) else hidden
+
+
+def append_target(inputs: dict[str, torch.Tensor], target_ids: list[int]) -> dict[str, torch.Tensor]:
+    prompt_ids = inputs['input_ids']
+    target = torch.tensor(target_ids, dtype=prompt_ids.dtype, device=prompt_ids.device).unsqueeze(0)
+    attention = inputs.get('attention_mask', torch.ones_like(prompt_ids))
+    target_mask = torch.ones((1, len(target_ids)), dtype=attention.dtype, device=attention.device)
+    return {
+        'input_ids': torch.cat([prompt_ids, target], dim=1),
+        'attention_mask': torch.cat([attention, target_mask], dim=1),
+    }
+
+
+
+@torch.inference_mode()
+def main() -> None:
+    args = parse_args()
+    set_seed(args.seed)
+    sizes = sorted({int(size) for size in args.sizes if int(size) > 0})
+    if not sizes:
+        raise ValueError('At least one positive feature-set size is required.')
+
+    tasks = load_jsonl(args.tasks)
+    selected = load_feature_sets(args.catalog, max(sizes))
+    missing = sorted({task['concept'] for task in tasks}.difference(selected))
+    if missing:
+        raise RuntimeError(f'No feature sets for concepts: {missing}')
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    model_dtype = torch.float16 if device.type == 'cuda' else torch.float32
+    tokenizer = AutoTokenizer.from_pretrained(SETTINGS.model_id)
+    if tokenizer.pad_token_id is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    model = AutoModelForCausalLM.from_pretrained(
+        SETTINGS.model_id,
+        torch_dtype=model_dtype,
+        low_cpu_mem_usage=True,
+    ).to(device)
+    model.eval()
+    layers = sorted({int(item['layer']) for item in selected.values()})
+    sae_store = SAEStore(
+        SETTINGS.sae_repo_id,
+        layers=layers,
+        device=device,
+        dtype=torch.float32,
+        top_k=SETTINGS.sae_top_k,
+    )
+
+    results: list[dict] = []
+    for task_idx, task in enumerate(tasks):
+        concept = task['concept']
+        layer = int(selected[concept]['layer'])
+        candidate_ids = [int(x) for x in selected[concept]['feature_ids']]
+        sae = sae_store.get(layer)
+        prompt_inputs = tokenizer(task['prompt'], return_tensors='pt', truncation=True, max_length=192)
+        prompt_inputs = {key: value.to(device) for key, value in prompt_inputs.items()}
+        prompt_len = int(prompt_inputs['input_ids'].shape[1])
+        target_ids = tokenizer(task['target'], add_special_tokens=False)['input_ids']
+        if not target_ids:
+            raise RuntimeError(f"Target tokenization empty for task {task['id']}")
+        target_ids = [int(x) for x in target_ids]
+        full_inputs = append_target(prompt_inputs, target_ids)
+
+        capture: dict = {}
+
+        def capture_hook(_module, _inp, output):
+            if 'hidden' not in capture:
+                capture['hidden'] = hidden_from_output(output).detach()
+
+        handle = model.model.layers[layer].register_forward_hook(capture_hook)
+        baseline_out = model(**full_inputs, use_cache=False)
+        handle.remove()
+        baseline_logits = baseline_out.logits[0]
+        baseline_next = baseline_logits[prompt_len - 1]
+        baseline_seq, baseline_mean, _ = sequence_logprob_summary(
+            baseline_logits,
+            prompt_length=prompt_len,
+            target_ids=target_ids,
+        )
+        residual = capture['hidden'][0, prompt_len - 1]
+        encoding = sae.encode(residual)
+
+        valid_sizes = [size for size in sizes if size <= len(candidate_ids)]
+        condition_meta: list[tuple[int, str, list[int], torch.Tensor]] = []
+        for size in valid_sizes:
+            feature_ids = candidate_ids[:size]
+            activations = [encoding.activation_for(feature_id) for feature_id in feature_ids]
+            directions = torch.stack([sae.decoder_direction(feature_id) for feature_id in feature_ids])
+            delta, _ = joint_residual_delta(
+                directions,
+                activations,
+                InterventionSpec('ablate', 0.0),
+            )
+            control = normalized_random_control(
+                delta,
+                seed=args.seed + task_idx * 101 + size,
+            )
+            condition_meta.extend(
+                [
+                    (size, 'sae_feature_set', feature_ids, delta),
+                    (size, 'random_norm_matched', feature_ids, control),
+                ]
+            )
+
+        repeated = {key: value.repeat(len(condition_meta), 1) for key, value in full_inputs.items()}
+        deltas = torch.stack([item[3] for item in condition_meta])
+        applied = {'done': False}
+
+        def edit_hook(_module, _inp, output):
+            if applied['done']:
+                return output
+            hidden = hidden_from_output(output)
+            modified = hidden.clone()
+            modified[:, prompt_len - 1, :] = (
+                modified[:, prompt_len - 1, :] + deltas.to(hidden.device, hidden.dtype)
+            )
+            applied['done'] = True
+            return replace_hidden(output, modified)
+
+        hook = model.model.layers[layer].register_forward_hook(edit_hook)
+        edited_out = model(**repeated, use_cache=False)
+        hook.remove()
+
+        for row_idx, (size, condition, feature_ids, applied_delta) in enumerate(condition_meta):
+            logits = edited_out.logits[row_idx]
+            seq_logp, mean_logp, _ = sequence_logprob_summary(
+                logits,
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
+            active_count = sum(encoding.activation_for(feature_id) > 0 for feature_id in feature_ids)
+            results.append(
+                {
+                    'task_id': task['id'],
+                    'concept': concept,
+                    'prompt': task['prompt'],
+                    'target_text': task['target'],
+                    'target_token_count': len(target_ids),
+                    'layer': layer,
+                    'set_size': int(size),
+                    'feature_ids': ','.join(str(x) for x in feature_ids),
+                    'active_selected_features': int(active_count),
+                    'condition': condition,
+                    'perturbation_l2': float(torch.linalg.vector_norm(applied_delta.float()).item()),
+                    'baseline_target_sequence_logprob': baseline_seq,
+                    'modified_target_sequence_logprob': seq_logp,
+                    'target_sequence_logprob_delta': seq_logp - baseline_seq,
+                    'baseline_target_mean_logprob': baseline_mean,
+                    'modified_target_mean_logprob': mean_logp,
+                    'target_mean_logprob_delta': mean_logp - baseline_mean,
+                    'js_divergence': js_divergence_from_logits(
+                        baseline_next,
+                        logits[prompt_len - 1],
+                    ),
+                }
+            )
+        print(f"Feature-set task {task_idx + 1}/{len(tasks)}: {concept}", flush=True)
+
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    with args.output.open('w', newline='', encoding='utf-8') as handle:
+        writer = csv.DictWriter(handle, fieldnames=list(results[0].keys()))
+        writer.writeheader()
+        writer.writerows(results)
+    print(f'Wrote {len(results)} feature-set rows to {args.output}')
+
+
+if __name__ == '__main__':
+    main()

featurelens/interventions.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 from dataclasses import dataclass
+from collections.abc import Sequence
 
 import torch
 
@@ -32,6 +33,47 @@ def residual_delta(
     return decoder_direction * spec.delta_activation(original_activation)
 
 
+def joint_residual_delta(
+    decoder_directions: torch.Tensor,
+    original_activations: Sequence[float] | torch.Tensor,
+    spec: InterventionSpec,
+) -> tuple[torch.Tensor, list[float]]:
+    """
+    Sum reconstruction-preserving deltas for a set of SAE features.
+
+    ``decoder_directions`` must have shape ``[n_features, d_model]``. The same
+    ablation/scale intervention is applied to every selected feature. ``inject``
+    is intentionally rejected for feature sets because a shared additive
+    coefficient has ambiguous semantics across unrelated decoder directions.
+    """
+    mode = spec.mode.lower().strip()
+    if mode not in {'ablate', 'scale'}:
+        raise ValueError("Feature-set interventions support only 'ablate' or 'scale'.")
+
+    directions = decoder_directions
+    if directions.ndim != 2:
+        raise ValueError('decoder_directions must have shape [n_features, d_model].')
+
+    if isinstance(original_activations, torch.Tensor):
+        activations = original_activations.detach().float().reshape(-1).tolist()
+    else:
+        activations = [float(x) for x in original_activations]
+
+    if len(activations) != directions.shape[0]:
+        raise ValueError('Number of activations must match decoder directions.')
+    if not activations:
+        raise ValueError('Select at least one feature for a feature-set intervention.')
+
+    coefficient_deltas = [spec.delta_activation(value) for value in activations]
+    coeff = torch.tensor(
+        coefficient_deltas,
+        device=directions.device,
+        dtype=directions.dtype,
+    )
+    delta = torch.sum(directions * coeff[:, None], dim=0)
+    return delta, coefficient_deltas
+
+
 def normalized_random_control(delta: torch.Tensor, seed: int) -> torch.Tensor:
     """Generate a deterministic random residual perturbation with identical L2 norm."""
     norm = torch.linalg.vector_norm(delta.float())

featurelens/metrics.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import math
+from collections.abc import Sequence
 
 import numpy as np
 import torch
@@ -34,10 +35,85 @@ def safe_log_probability(probability: float) -> float:
     return math.log(max(float(probability), 1e-12))
 
 
-def sparse_jaccard(indices_a: np.ndarray, values_a: np.ndarray, indices_b: np.ndarray, values_b: np.ndarray) -> float:
+def target_token_logprobs(
+    logits: torch.Tensor,
+    *,
+    prompt_length: int,
+    target_ids: Sequence[int] | torch.Tensor,
+) -> torch.Tensor:
+    """
+    Return teacher-forced log probabilities for an exact target continuation.
+
+    ``logits`` must be ``[sequence, vocab]`` for the concatenated prompt + target
+    sequence. The token at target position ``j`` is predicted by the logit row
+    immediately before that token.
+    """
+    if logits.ndim != 2:
+        raise ValueError('logits must have shape [sequence, vocab].')
+    ids = torch.as_tensor(target_ids, device=logits.device, dtype=torch.long).reshape(-1)
+    if ids.numel() == 0:
+        raise ValueError('target_ids must contain at least one token.')
+    start = int(prompt_length) - 1
+    stop = start + int(ids.numel())
+    if start < 0 or stop > logits.shape[0]:
+        raise ValueError('Prompt/target lengths are incompatible with logits sequence length.')
+    rows = logits[start:stop].float()
+    return torch.log_softmax(rows, dim=-1).gather(1, ids[:, None]).squeeze(1)
+
+
+def sequence_logprob_summary(
+    logits: torch.Tensor,
+    *,
+    prompt_length: int,
+    target_ids: Sequence[int] | torch.Tensor,
+) -> tuple[float, float, list[float]]:
+    """Return total log p, mean log p/token, and token-level log probabilities."""
+    token_values = target_token_logprobs(
+        logits,
+        prompt_length=prompt_length,
+        target_ids=target_ids,
+    )
+    total = float(token_values.sum().item())
+    mean = float(token_values.mean().item())
+    return total, mean, [float(x) for x in token_values.detach().cpu().tolist()]
+
+
+def sparse_jaccard(
+    indices_a: np.ndarray,
+    values_a: np.ndarray,
+    indices_b: np.ndarray,
+    values_b: np.ndarray,
+) -> float:
     a = set(indices_a[np.asarray(values_a) > 0].tolist())
     b = set(indices_b[np.asarray(values_b) > 0].tolist())
     union = a | b
     if not union:
         return 1.0
     return len(a & b) / len(union)
+
+
+def sparse_topk_cosine(
+    indices_a: Sequence[int] | torch.Tensor,
+    values_a: Sequence[float] | torch.Tensor,
+    indices_b: Sequence[int] | torch.Tensor,
+    values_b: Sequence[float] | torch.Tensor,
+) -> float:
+    """Cosine similarity between two sparse TopK vectors without densifying SAE width."""
+    idx_a = torch.as_tensor(indices_a, dtype=torch.long).reshape(-1).cpu().tolist()
+    val_a = torch.as_tensor(values_a, dtype=torch.float64).reshape(-1).cpu().tolist()
+    idx_b = torch.as_tensor(indices_b, dtype=torch.long).reshape(-1).cpu().tolist()
+    val_b = torch.as_tensor(values_b, dtype=torch.float64).reshape(-1).cpu().tolist()
+
+    a = {int(i): float(v) for i, v in zip(idx_a, val_a, strict=True) if float(v) > 0}
+    b = {int(i): float(v) for i, v in zip(idx_b, val_b, strict=True) if float(v) > 0}
+    if not a and not b:
+        return 1.0
+    if not a or not b:
+        return 0.0
+
+    dot = sum(value * b.get(feature_id, 0.0) for feature_id, value in a.items())
+    norm_a = math.sqrt(sum(value * value for value in a.values()))
+    norm_b = math.sqrt(sum(value * value for value in b.values()))
+    if norm_a == 0.0 or norm_b == 0.0:
+        return 0.0
+    return float(dot / (norm_a * norm_b))

featurelens/runtime.py

@@ -15,10 +15,16 @@ from .catalog import FeatureCatalog
 from .config import SETTINGS, Settings
 from .interventions import (
     InterventionSpec,
+    joint_residual_delta,
     normalized_random_control,
     residual_delta,
 )
-from .metrics import js_divergence_from_logits, reconstruction_metrics, safe_log_probability
+from .metrics import (
+    js_divergence_from_logits,
+    reconstruction_metrics,
+    sequence_logprob_summary,
+    sparse_topk_cosine,
+)
 from .sae import SAEStore, SparseEncoding
 
 
@@ -64,14 +70,20 @@ class InterventionResult:
     random_js_divergence: float
     js_specificity_ratio: float
     target_text: str
-    target_token: str
     target_token_count: int
+    target_tokens: list[str]
     baseline_target_prob: float | None
     modified_target_prob: float | None
     random_target_prob: float | None
-    target_logprob_delta: float | None
-    random_target_logprob_delta: float | None
+    baseline_sequence_logprob: float | None
+    modified_sequence_logprob: float | None
+    random_sequence_logprob: float | None
+    sequence_logprob_delta: float | None
+    random_sequence_logprob_delta: float | None
+    mean_logprob_delta: float | None
+    random_mean_logprob_delta: float | None
     target_specificity_ratio: float | None
+    target_token_rows: list[list[object]]
     top_token_rows: list[list[object]]
 
 
@@ -85,9 +97,48 @@ class LayerSweepResult:
 @dataclass
 class DoseResponseResult:
     feature_activation: float
-    target_token: str
-    target_token_count: int
+    target_tokens: list[str]
+    rows: list[list[object]]
+
+
+@dataclass
+class FeatureSetResult:
+    feature_ids: list[int]
+    feature_rows: list[list[object]]
+    perturbation_norm: float
+    js_divergence: float
+    random_js_divergence: float
+    js_specificity_ratio: float
+    baseline_sequence_logprob: float
+    modified_sequence_logprob: float
+    random_sequence_logprob: float
+    sequence_logprob_delta: float
+    random_sequence_logprob_delta: float
+    mean_logprob_delta: float
+    random_mean_logprob_delta: float
+    target_specificity_ratio: float
+    target_tokens: list[str]
+    target_token_rows: list[list[object]]
+
+
+@dataclass
+class FeatureSetSweepResult:
+    target_tokens: list[str]
+    rows: list[list[object]]
+
+
+@dataclass
+class ParaphraseResult:
+    tokens_a: list[str]
+    token_index_a: int
+    tokens_b: list[str]
+    token_index_b: int
+    topk_jaccard: float
+    sparse_cosine: float
+    shared_top_n: int
+    top_n: int
     rows: list[list[object]]
+    chart_rows: list[list[object]]
 
 
 class FeatureLensRuntime:
@@ -153,6 +204,40 @@ class FeatureLensRuntime:
         )
         return {key: value.to(self.device) for key, value in batch.items()}
 
+    def _target_ids(self, target_text: str) -> list[int]:
+        assert self.tokenizer is not None
+        ids = self.tokenizer(target_text, add_special_tokens=False)['input_ids']
+        if not ids:
+            raise ValueError('Target continuation tokenized to an empty sequence.')
+        return [int(x) for x in ids]
+
+    def _append_target(
+        self,
+        prompt_inputs: dict[str, torch.Tensor],
+        target_ids: Sequence[int],
+    ) -> dict[str, torch.Tensor]:
+        prompt_ids = prompt_inputs['input_ids']
+        target = torch.tensor(
+            list(target_ids),
+            dtype=prompt_ids.dtype,
+            device=prompt_ids.device,
+        ).unsqueeze(0)
+        full_ids = torch.cat([prompt_ids, target], dim=1)
+        if 'attention_mask' in prompt_inputs:
+            target_mask = torch.ones(
+                (prompt_ids.shape[0], len(target_ids)),
+                dtype=prompt_inputs['attention_mask'].dtype,
+                device=prompt_ids.device,
+            )
+            attention = torch.cat([prompt_inputs['attention_mask'], target_mask], dim=1)
+        else:
+            attention = torch.ones_like(full_ids)
+        return {'input_ids': full_ids, 'attention_mask': attention}
+
+    @staticmethod
+    def _repeat_inputs(inputs: dict[str, torch.Tensor], repeats: int) -> dict[str, torch.Tensor]:
+        return {key: value.repeat(int(repeats), 1) for key, value in inputs.items()}
+
     @staticmethod
     def _hidden_from_output(output):
         return output[0] if isinstance(output, tuple) else output
@@ -209,11 +294,7 @@ class FeatureLensRuntime:
             if hidden.ndim != 3:
                 return output
             seq_len = hidden.shape[1]
-            idx = int(token_index)
-            if idx < 0:
-                idx = seq_len + idx
-            if idx < 0 or idx >= seq_len:
-                raise IndexError(f'Token index {token_index} outside prompt length {seq_len}.')
+            idx = self._resolve_index(int(token_index), seq_len)
             modified = hidden.clone()
             modified[:, idx, :] = modified[:, idx, :] + delta.to(hidden.device, hidden.dtype)
             applied['done'] = True
@@ -225,6 +306,39 @@ class FeatureLensRuntime:
         finally:
             handle.remove()
 
+    @contextmanager
+    def _batch_delta_hook(
+        self,
+        layer: int,
+        token_index: int,
+        deltas: torch.Tensor,
+    ) -> Iterator[None]:
+        """Apply one residual delta per batch row in a single model forward."""
+        assert self.model is not None
+        if deltas.ndim != 2:
+            raise ValueError('deltas must have shape [batch, d_model].')
+        applied = {'done': False}
+
+        def hook(_module, _inputs, output):
+            if applied['done']:
+                return output
+            hidden = self._hidden_from_output(output)
+            if hidden.ndim != 3:
+                return output
+            if hidden.shape[0] != deltas.shape[0]:
+                raise ValueError('Delta batch size does not match model batch size.')
+            idx = self._resolve_index(int(token_index), hidden.shape[1])
+            modified = hidden.clone()
+            modified[:, idx, :] = modified[:, idx, :] + deltas.to(hidden.device, hidden.dtype)
+            applied['done'] = True
+            return self._replace_hidden_in_output(output, modified)
+
+        handle = self.model.model.layers[int(layer)].register_forward_hook(hook)
+        try:
+            yield
+        finally:
+            handle.remove()
+
     @staticmethod
     def _resolve_index(token_index: int, seq_len: int) -> int:
         idx = int(token_index)
@@ -235,10 +349,22 @@ class FeatureLensRuntime:
         return idx
 
     @staticmethod
-    def _control_seed(text: str, layer: int, feature_id: int, mode: str, coefficient: float) -> int:
-        payload = f'{text}\0{layer}\0{feature_id}\0{mode}\0{coefficient:.8g}'.encode('utf-8')
+    def _control_seed(text: str, layer: int, key: str, mode: str, coefficient: float) -> int:
+        payload = f'{text}\0{layer}\0{key}\0{mode}\0{coefficient:.8g}'.encode('utf-8')
         return int.from_bytes(hashlib.sha256(payload).digest()[:4], 'big', signed=False)
 
+    @staticmethod
+    def _encoding_map(encoding: SparseEncoding) -> dict[int, float]:
+        return {
+            int(feature_id): float(value)
+            for feature_id, value in zip(
+                encoding.indices.detach().cpu().tolist(),
+                encoding.values.detach().float().cpu().tolist(),
+                strict=True,
+            )
+            if float(value) > 0
+        }
+
     @torch.inference_mode()
     def analyze(self, text: str, layer: int, token_index: int = -1, top_n: int = 12) -> AnalysisResult:
         self.ensure_ready(preload_saes=False)
@@ -306,7 +432,9 @@ class FeatureLensRuntime:
                 entropy = 0.0
             else:
                 probs = positive / total
-                entropy = float((-(probs * torch.log(probs)).sum() / math.log(positive.numel())).item())
+                entropy = float(
+                    (-(probs * torch.log(probs)).sum() / math.log(positive.numel())).item()
+                )
             top5_fraction = (
                 float(values[: min(5, values.numel())].sum().item() / total.item())
                 if float(total.item()) > 0
@@ -350,6 +478,37 @@ class FeatureLensRuntime:
         rows.sort(key=lambda row: max(row[1], row[2]), reverse=True)
         return rows[: min(len(rows), 12)]
 
+    def _target_rows(
+        self,
+        target_ids: Sequence[int],
+        baseline_token_logps: Sequence[float],
+        modified_token_logps: Sequence[float],
+        random_token_logps: Sequence[float],
+    ) -> list[list[object]]:
+        assert self.tokenizer is not None
+        rows = []
+        for idx, (token_id, bp, mp, rp) in enumerate(
+            zip(
+                target_ids,
+                baseline_token_logps,
+                modified_token_logps,
+                random_token_logps,
+                strict=True,
+            )
+        ):
+            rows.append(
+                [
+                    idx,
+                    repr(self.tokenizer.decode([int(token_id)])),
+                    float(bp),
+                    float(mp),
+                    float(rp),
+                    float(mp - bp),
+                    float(rp - bp),
+                ]
+            )
+        return rows
+
     @torch.inference_mode()
     def intervene(
         self,
@@ -364,70 +523,130 @@ class FeatureLensRuntime:
     ) -> InterventionResult:
         self.ensure_ready(preload_saes=False)
         assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
-        inputs = self._inputs(text)
-        prompt_len = int(inputs['input_ids'].shape[1])
+        prompt_inputs = self._inputs(text)
+        prompt_len = int(prompt_inputs['input_ids'].shape[1])
         idx = self._resolve_index(int(token_index), prompt_len)
-
         sae = self.sae_store.get(int(layer))
+
+        target_ids: list[int] = []
+        baseline_seq = baseline_mean = None
+        baseline_token_logps: list[float] = []
+        baseline_next_logits: torch.Tensor | None = None
         capture: dict = {}
-        generation_kwargs = {
-            'max_new_tokens': min(int(max_new_tokens), self.settings.max_new_tokens),
-            'do_sample': False,
-            'return_dict_in_generate': True,
-            'output_scores': True,
-            'pad_token_id': self.tokenizer.eos_token_id,
-        }
-        with self._capture_hook(int(layer), capture):
-            baseline = self.model.generate(**inputs, **generation_kwargs)
+
+        if target_text.strip():
+            target_ids = self._target_ids(target_text)
+            full_inputs = self._append_target(prompt_inputs, target_ids)
+            with self._capture_hook(int(layer), capture):
+                baseline_full = self.model(**full_inputs, use_cache=False)
+            baseline_next_logits = baseline_full.logits[0, prompt_len - 1]
+            baseline_seq, baseline_mean, baseline_token_logps = sequence_logprob_summary(
+                baseline_full.logits[0],
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
+        else:
+            with self._capture_hook(int(layer), capture):
+                baseline_prompt = self.model(**prompt_inputs, use_cache=False)
+            baseline_next_logits = baseline_prompt.logits[0, -1]
+
         residual = capture['hidden'][0, idx]
         encoding = sae.encode(residual)
         original_activation = encoding.activation_for(int(feature_id))
         spec = InterventionSpec(mode=mode, coefficient=float(coefficient))
         delta = residual_delta(sae.decoder_direction(int(feature_id)), original_activation, spec)
-
-        with self._delta_hook(int(layer), idx, delta):
-            modified = self.model.generate(**inputs, **generation_kwargs)
-
-        # Live negative control: one extra forward pass, not another full generation.
         control_delta = normalized_random_control(
             delta,
-            seed=self._control_seed(text, int(layer), int(feature_id), mode, float(coefficient)),
+            seed=self._control_seed(
+                text,
+                int(layer),
+                str(int(feature_id)),
+                mode,
+                float(coefficient),
+            ),
         )
-        with self._delta_hook(int(layer), idx, control_delta):
-            random_out = self.model(**inputs, use_cache=False)
-        random_logits = random_out.logits[0, -1]
 
-        baseline_ids = baseline.sequences[0, prompt_len:]
-        modified_ids = modified.sequences[0, prompt_len:]
+        modified_next_logits: torch.Tensor
+        random_next_logits: torch.Tensor
+        modified_seq = modified_mean = random_seq = random_mean = None
+        modified_token_logps: list[float] = []
+        random_token_logps: list[float] = []
+
+        if target_ids:
+            full_inputs = self._append_target(prompt_inputs, target_ids)
+            repeated = self._repeat_inputs(full_inputs, 2)
+            deltas = torch.stack([delta, control_delta], dim=0)
+            with self._batch_delta_hook(int(layer), idx, deltas):
+                edited_full = self.model(**repeated, use_cache=False)
+            modified_logits_all = edited_full.logits[0]
+            random_logits_all = edited_full.logits[1]
+            modified_next_logits = modified_logits_all[prompt_len - 1]
+            random_next_logits = random_logits_all[prompt_len - 1]
+            modified_seq, modified_mean, modified_token_logps = sequence_logprob_summary(
+                modified_logits_all,
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
+            random_seq, random_mean, random_token_logps = sequence_logprob_summary(
+                random_logits_all,
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
+        else:
+            with self._delta_hook(int(layer), idx, delta):
+                modified_prompt = self.model(**prompt_inputs, use_cache=False)
+            with self._delta_hook(int(layer), idx, control_delta):
+                random_prompt = self.model(**prompt_inputs, use_cache=False)
+            modified_next_logits = modified_prompt.logits[0, -1]
+            random_next_logits = random_prompt.logits[0, -1]
+
+        generation_kwargs = {
+            'max_new_tokens': min(int(max_new_tokens), self.settings.max_new_tokens),
+            'do_sample': False,
+            'return_dict_in_generate': True,
+            'output_scores': False,
+            'pad_token_id': self.tokenizer.eos_token_id,
+        }
+        baseline_generation = self.model.generate(**prompt_inputs, **generation_kwargs)
+        with self._delta_hook(int(layer), idx, delta):
+            modified_generation = self.model.generate(**prompt_inputs, **generation_kwargs)
+
+        baseline_ids = baseline_generation.sequences[0, prompt_len:]
+        modified_ids = modified_generation.sequences[0, prompt_len:]
         baseline_text = self.tokenizer.decode(baseline_ids, skip_special_tokens=True)
         modified_text = self.tokenizer.decode(modified_ids, skip_special_tokens=True)
 
-        if not baseline.scores or not modified.scores:
-            raise RuntimeError('Generation returned no score tensors.')
-        baseline_logits = baseline.scores[0][0]
-        modified_logits = modified.scores[0][0]
-        js = js_divergence_from_logits(baseline_logits, modified_logits)
-        random_js = js_divergence_from_logits(baseline_logits, random_logits)
+        assert baseline_next_logits is not None
+        js = js_divergence_from_logits(baseline_next_logits, modified_next_logits)
+        random_js = js_divergence_from_logits(baseline_next_logits, random_next_logits)
         js_ratio = abs(js) / max(abs(random_js), 1e-12)
 
-        target_token = ''
-        target_count = 0
-        bp = mp = rp = log_delta = random_log_delta = specificity = None
-        if target_text.strip():
-            target_ids = self.tokenizer(target_text, add_special_tokens=False)['input_ids']
-            target_count = len(target_ids)
-            if target_ids:
-                target_id = int(target_ids[0])
-                target_token = self.tokenizer.decode([target_id])
-                p = torch.softmax(baseline_logits.float(), dim=-1)
-                q = torch.softmax(modified_logits.float(), dim=-1)
-                r = torch.softmax(random_logits.float(), dim=-1)
-                bp = float(p[target_id].item())
-                mp = float(q[target_id].item())
-                rp = float(r[target_id].item())
-                log_delta = safe_log_probability(mp) - safe_log_probability(bp)
-                random_log_delta = safe_log_probability(rp) - safe_log_probability(bp)
-                specificity = abs(log_delta) / max(abs(random_log_delta), 1e-12)
+        bp = mp = rp = None
+        sequence_delta = random_sequence_delta = mean_delta = random_mean_delta = specificity = None
+        target_rows: list[list[object]] = []
+        target_tokens: list[str] = []
+        if target_ids:
+            p = torch.softmax(baseline_next_logits.float(), dim=-1)
+            q = torch.softmax(modified_next_logits.float(), dim=-1)
+            r = torch.softmax(random_next_logits.float(), dim=-1)
+            first_id = int(target_ids[0])
+            bp = float(p[first_id].item())
+            mp = float(q[first_id].item())
+            rp = float(r[first_id].item())
+            assert baseline_seq is not None and modified_seq is not None and random_seq is not None
+            assert baseline_mean is not None and modified_mean is not None and random_mean is not None
+            sequence_delta = float(modified_seq - baseline_seq)
+            random_sequence_delta = float(random_seq - baseline_seq)
+            mean_delta = float(modified_mean - baseline_mean)
+            random_mean_delta = float(random_mean - baseline_mean)
+            specificity = abs(mean_delta) / max(abs(random_mean_delta), 1e-12)
+            target_tokens = [self.tokenizer.decode([int(token_id)]) for token_id in target_ids]
+            target_rows = self._target_rows(
+                target_ids,
+                baseline_token_logps,
+                modified_token_logps,
+                random_token_logps,
+            )
 
         return InterventionResult(
             baseline_text=baseline_text,
@@ -439,16 +658,25 @@ class FeatureLensRuntime:
             random_js_divergence=float(random_js),
             js_specificity_ratio=float(js_ratio),
             target_text=target_text,
-            target_token=target_token,
-            target_token_count=target_count,
+            target_token_count=len(target_ids),
+            target_tokens=target_tokens,
             baseline_target_prob=bp,
             modified_target_prob=mp,
             random_target_prob=rp,
-            target_logprob_delta=log_delta,
-            random_target_logprob_delta=random_log_delta,
+            baseline_sequence_logprob=baseline_seq,
+            modified_sequence_logprob=modified_seq,
+            random_sequence_logprob=random_seq,
+            sequence_logprob_delta=sequence_delta,
+            random_sequence_logprob_delta=random_sequence_delta,
+            mean_logprob_delta=mean_delta,
+            random_mean_logprob_delta=random_mean_delta,
             target_specificity_ratio=specificity,
+            target_token_rows=target_rows,
             top_token_rows=self._top_token_rows(
-                self.tokenizer, baseline_logits, modified_logits, k=8
+                self.tokenizer,
+                baseline_next_logits,
+                modified_next_logits,
+                k=8,
             ),
         )
 
@@ -466,55 +694,368 @@ class FeatureLensRuntime:
             raise ValueError('Dose-response requires a target continuation.')
         self.ensure_ready(preload_saes=False)
         assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
-        inputs = self._inputs(text)
-        prompt_len = int(inputs['input_ids'].shape[1])
+        prompt_inputs = self._inputs(text)
+        prompt_len = int(prompt_inputs['input_ids'].shape[1])
         idx = self._resolve_index(int(token_index), prompt_len)
+        target_ids = self._target_ids(target_text)
+        full_inputs = self._append_target(prompt_inputs, target_ids)
         sae = self.sae_store.get(int(layer))
 
         capture: dict = {}
         with self._capture_hook(int(layer), capture):
-            baseline_out = self.model(**inputs, use_cache=False)
-        baseline_logits = baseline_out.logits[0, -1]
+            baseline_out = self.model(**full_inputs, use_cache=False)
+        baseline_logits = baseline_out.logits[0]
+        baseline_next = baseline_logits[prompt_len - 1]
+        baseline_seq, baseline_mean, _ = sequence_logprob_summary(
+            baseline_logits,
+            prompt_length=prompt_len,
+            target_ids=target_ids,
+        )
         residual = capture['hidden'][0, idx]
         encoding = sae.encode(residual)
         original_activation = encoding.activation_for(int(feature_id))
         direction = sae.decoder_direction(int(feature_id))
 
-        target_ids = self.tokenizer(target_text, add_special_tokens=False)['input_ids']
-        if not target_ids:
-            raise ValueError('Target continuation tokenized to an empty sequence.')
-        target_id = int(target_ids[0])
-        target_token = self.tokenizer.decode([target_id])
-        baseline_prob = float(torch.softmax(baseline_logits.float(), dim=-1)[target_id].item())
-
-        rows: list[list[object]] = []
+        deltas = []
+        delta_coefficients = []
+        norms = []
         for multiplier in multipliers:
             spec = InterventionSpec('scale', float(multiplier))
             delta = residual_delta(direction, original_activation, spec)
-            if float(torch.linalg.vector_norm(delta.float()).item()) == 0.0:
-                modified_logits = baseline_logits
-            else:
-                with self._delta_hook(int(layer), idx, delta):
-                    output = self.model(**inputs, use_cache=False)
-                modified_logits = output.logits[0, -1]
-            modified_prob = float(torch.softmax(modified_logits.float(), dim=-1)[target_id].item())
-            log_delta = safe_log_probability(modified_prob) - safe_log_probability(baseline_prob)
+            deltas.append(delta)
+            delta_coefficients.append(float(spec.delta_activation(original_activation)))
+            norms.append(float(torch.linalg.vector_norm(delta.float()).item()))
+
+        repeated = self._repeat_inputs(full_inputs, len(deltas))
+        with self._batch_delta_hook(int(layer), idx, torch.stack(deltas, dim=0)):
+            outputs = self.model(**repeated, use_cache=False)
+
+        rows: list[list[object]] = []
+        for row_idx, multiplier in enumerate(multipliers):
+            modified_logits = outputs.logits[row_idx]
+            modified_seq, modified_mean, _ = sequence_logprob_summary(
+                modified_logits,
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
             rows.append(
                 [
                     float(multiplier),
-                    float(spec.delta_activation(original_activation)),
-                    float(torch.linalg.vector_norm(delta.float()).item()),
-                    baseline_prob,
-                    modified_prob,
-                    float(log_delta),
-                    float(js_divergence_from_logits(baseline_logits, modified_logits)),
+                    delta_coefficients[row_idx],
+                    norms[row_idx],
+                    float(baseline_mean),
+                    float(modified_mean),
+                    float(modified_mean - baseline_mean),
+                    float(modified_seq - baseline_seq),
+                    float(
+                        js_divergence_from_logits(
+                            baseline_next,
+                            modified_logits[prompt_len - 1],
+                        )
+                    ),
                 ]
             )
         return DoseResponseResult(
             feature_activation=float(original_activation),
-            target_token=target_token,
-            target_token_count=len(target_ids),
+            target_tokens=[self.tokenizer.decode([int(token_id)]) for token_id in target_ids],
+            rows=rows,
+        )
+
+    @torch.inference_mode()
+    def intervene_feature_set(
+        self,
+        text: str,
+        layer: int,
+        token_index: int,
+        feature_ids: Sequence[int],
+        mode: str,
+        coefficient: float,
+        target_text: str,
+    ) -> FeatureSetResult:
+        if not target_text.strip():
+            raise ValueError('Feature-set causal testing requires a target continuation.')
+        ids = list(dict.fromkeys(int(x) for x in feature_ids))
+        if not ids:
+            raise ValueError('Select at least one feature.')
+        if len(ids) > 12:
+            raise ValueError('Select at most 12 features for a live feature-set intervention.')
+        if mode not in {'ablate', 'scale'}:
+            raise ValueError("Feature-set mode must be 'ablate' or 'scale'.")
+
+        self.ensure_ready(preload_saes=False)
+        assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
+        prompt_inputs = self._inputs(text)
+        prompt_len = int(prompt_inputs['input_ids'].shape[1])
+        idx = self._resolve_index(int(token_index), prompt_len)
+        target_ids = self._target_ids(target_text)
+        full_inputs = self._append_target(prompt_inputs, target_ids)
+        sae = self.sae_store.get(int(layer))
+
+        capture: dict = {}
+        with self._capture_hook(int(layer), capture):
+            baseline_out = self.model(**full_inputs, use_cache=False)
+        residual = capture['hidden'][0, idx]
+        encoding = sae.encode(residual)
+        activations = [encoding.activation_for(feature_id) for feature_id in ids]
+        directions = torch.stack([sae.decoder_direction(feature_id) for feature_id in ids], dim=0)
+        spec = InterventionSpec(mode, float(coefficient))
+        delta, coefficient_deltas = joint_residual_delta(directions, activations, spec)
+        control = normalized_random_control(
+            delta,
+            seed=self._control_seed(
+                text,
+                int(layer),
+                ','.join(str(x) for x in ids),
+                mode,
+                float(coefficient),
+            ),
+        )
+
+        repeated = self._repeat_inputs(full_inputs, 2)
+        with self._batch_delta_hook(int(layer), idx, torch.stack([delta, control], dim=0)):
+            outputs = self.model(**repeated, use_cache=False)
+
+        baseline_logits = baseline_out.logits[0]
+        modified_logits = outputs.logits[0]
+        random_logits = outputs.logits[1]
+        baseline_seq, baseline_mean, baseline_tokens = sequence_logprob_summary(
+            baseline_logits,
+            prompt_length=prompt_len,
+            target_ids=target_ids,
+        )
+        modified_seq, modified_mean, modified_tokens = sequence_logprob_summary(
+            modified_logits,
+            prompt_length=prompt_len,
+            target_ids=target_ids,
+        )
+        random_seq, random_mean, random_tokens = sequence_logprob_summary(
+            random_logits,
+            prompt_length=prompt_len,
+            target_ids=target_ids,
+        )
+        mean_delta = float(modified_mean - baseline_mean)
+        random_mean_delta = float(random_mean - baseline_mean)
+        next_idx = prompt_len - 1
+        js = js_divergence_from_logits(baseline_logits[next_idx], modified_logits[next_idx])
+        random_js = js_divergence_from_logits(baseline_logits[next_idx], random_logits[next_idx])
+
+        feature_rows = [
+            [
+                feature_id,
+                float(activation),
+                float(delta_coefficient),
+                self.catalog.hint(int(layer), feature_id),
+            ]
+            for feature_id, activation, delta_coefficient in zip(
+                ids,
+                activations,
+                coefficient_deltas,
+                strict=True,
+            )
+        ]
+        return FeatureSetResult(
+            feature_ids=ids,
+            feature_rows=feature_rows,
+            perturbation_norm=float(torch.linalg.vector_norm(delta.float()).item()),
+            js_divergence=float(js),
+            random_js_divergence=float(random_js),
+            js_specificity_ratio=float(abs(js) / max(abs(random_js), 1e-12)),
+            baseline_sequence_logprob=float(baseline_seq),
+            modified_sequence_logprob=float(modified_seq),
+            random_sequence_logprob=float(random_seq),
+            sequence_logprob_delta=float(modified_seq - baseline_seq),
+            random_sequence_logprob_delta=float(random_seq - baseline_seq),
+            mean_logprob_delta=mean_delta,
+            random_mean_logprob_delta=random_mean_delta,
+            target_specificity_ratio=float(abs(mean_delta) / max(abs(random_mean_delta), 1e-12)),
+            target_tokens=[self.tokenizer.decode([int(token_id)]) for token_id in target_ids],
+            target_token_rows=self._target_rows(
+                target_ids,
+                baseline_tokens,
+                modified_tokens,
+                random_tokens,
+            ),
+        )
+
+    @torch.inference_mode()
+    def feature_set_size_sweep(
+        self,
+        text: str,
+        layer: int,
+        token_index: int,
+        target_text: str,
+        sizes: Sequence[int] = (1, 3, 5),
+    ) -> FeatureSetSweepResult:
+        """Jointly ablate the strongest k active features for k in ``sizes``."""
+        if not target_text.strip():
+            raise ValueError('Feature-set size sweep requires a target continuation.')
+        self.ensure_ready(preload_saes=False)
+        assert self.model is not None and self.tokenizer is not None and self.sae_store is not None
+        prompt_inputs = self._inputs(text)
+        prompt_len = int(prompt_inputs['input_ids'].shape[1])
+        idx = self._resolve_index(int(token_index), prompt_len)
+        target_ids = self._target_ids(target_text)
+        full_inputs = self._append_target(prompt_inputs, target_ids)
+        sae = self.sae_store.get(int(layer))
+
+        capture: dict = {}
+        with self._capture_hook(int(layer), capture):
+            baseline_out = self.model(**full_inputs, use_cache=False)
+        residual = capture['hidden'][0, idx]
+        encoding = sae.encode(residual)
+        active_ids = [
+            int(feature_id)
+            for feature_id, value in zip(
+                encoding.indices.detach().cpu().tolist(),
+                encoding.values.detach().float().cpu().tolist(),
+                strict=True,
+            )
+            if float(value) > 0
+        ]
+        valid_sizes = [int(size) for size in sizes if int(size) > 0 and int(size) <= len(active_ids)]
+        if not valid_sizes:
+            raise ValueError('Not enough active features for the requested set sizes.')
+
+        deltas: list[torch.Tensor] = []
+        controls: list[torch.Tensor] = []
+        feature_lists: list[list[int]] = []
+        norms: list[float] = []
+        for size in valid_sizes:
+            selected = active_ids[:size]
+            activations = [encoding.activation_for(feature_id) for feature_id in selected]
+            directions = torch.stack([sae.decoder_direction(feature_id) for feature_id in selected])
+            delta, _ = joint_residual_delta(
+                directions,
+                activations,
+                InterventionSpec('ablate', 0.0),
+            )
+            control = normalized_random_control(
+                delta,
+                seed=self._control_seed(
+                    text,
+                    int(layer),
+                    f'top-{size}',
+                    'ablate_set',
+                    0.0,
+                ),
+            )
+            deltas.append(delta)
+            controls.append(control)
+            feature_lists.append(selected)
+            norms.append(float(torch.linalg.vector_norm(delta.float()).item()))
+
+        all_deltas = torch.stack([item for pair in zip(deltas, controls, strict=True) for item in pair])
+        repeated = self._repeat_inputs(full_inputs, all_deltas.shape[0])
+        with self._batch_delta_hook(int(layer), idx, all_deltas):
+            outputs = self.model(**repeated, use_cache=False)
+
+        baseline_logits = baseline_out.logits[0]
+        baseline_seq, baseline_mean, _ = sequence_logprob_summary(
+            baseline_logits,
+            prompt_length=prompt_len,
+            target_ids=target_ids,
+        )
+        baseline_next = baseline_logits[prompt_len - 1]
+        rows: list[list[object]] = []
+        for sweep_idx, size in enumerate(valid_sizes):
+            sae_logits = outputs.logits[2 * sweep_idx]
+            random_logits = outputs.logits[2 * sweep_idx + 1]
+            sae_seq, sae_mean, _ = sequence_logprob_summary(
+                sae_logits,
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
+            random_seq, random_mean, _ = sequence_logprob_summary(
+                random_logits,
+                prompt_length=prompt_len,
+                target_ids=target_ids,
+            )
+            sae_delta = float(sae_mean - baseline_mean)
+            random_delta = float(random_mean - baseline_mean)
+            sae_js = js_divergence_from_logits(baseline_next, sae_logits[prompt_len - 1])
+            random_js = js_divergence_from_logits(baseline_next, random_logits[prompt_len - 1])
+            rows.append(
+                [
+                    int(size),
+                    ', '.join(str(x) for x in feature_lists[sweep_idx]),
+                    norms[sweep_idx],
+                    float(baseline_mean),
+                    float(sae_mean),
+                    sae_delta,
+                    random_delta,
+                    float(abs(sae_delta) / max(abs(random_delta), 1e-12)),
+                    float(sae_seq - baseline_seq),
+                    float(sae_js),
+                    float(random_js),
+                ]
+            )
+        return FeatureSetSweepResult(
+            target_tokens=[self.tokenizer.decode([int(token_id)]) for token_id in target_ids],
+            rows=rows,
+        )
+
+    @torch.inference_mode()
+    def compare_paraphrases(
+        self,
+        text_a: str,
+        text_b: str,
+        layer: int,
+        token_index_a: int = -1,
+        token_index_b: int = -1,
+        top_n: int = 12,
+    ) -> ParaphraseResult:
+        if not text_a.strip() or not text_b.strip():
+            raise ValueError('Enter both the original prompt and a paraphrase.')
+        a = self.analyze(text_a, int(layer), int(token_index_a), max(int(top_n), 12))
+        b = self.analyze(text_b, int(layer), int(token_index_b), max(int(top_n), 12))
+
+        map_a = self._encoding_map(a.features)
+        map_b = self._encoding_map(b.features)
+        set_a = set(map_a)
+        set_b = set(map_b)
+        union = set_a | set_b
+        jaccard = len(set_a & set_b) / len(union) if union else 1.0
+        cosine = sparse_topk_cosine(
+            a.features.indices,
+            a.features.values,
+            b.features.indices,
+            b.features.values,
+        )
+
+        top_ids_a = [int(row[1]) for row in a.rows[: int(top_n)]]
+        top_ids_b = [int(row[1]) for row in b.rows[: int(top_n)]]
+        top_union = list(dict.fromkeys(top_ids_a + top_ids_b))
+        shared_top_n = len(set(top_ids_a) & set(top_ids_b))
+        rows: list[list[object]] = []
+        chart_rows: list[list[object]] = []
+        for feature_id in top_union:
+            va = float(map_a.get(feature_id, 0.0))
+            vb = float(map_b.get(feature_id, 0.0))
+            status = 'shared' if va > 0 and vb > 0 else ('original only' if va > 0 else 'paraphrase only')
+            rows.append(
+                [
+                    feature_id,
+                    va,
+                    vb,
+                    status,
+                    self.catalog.hint(int(layer), feature_id),
+                ]
+            )
+            chart_rows.append([str(feature_id), 'Original', va])
+            chart_rows.append([str(feature_id), 'Paraphrase', vb])
+
+        rows.sort(key=lambda row: max(float(row[1]), float(row[2])), reverse=True)
+        return ParaphraseResult(
+            tokens_a=a.tokens,
+            token_index_a=a.token_index,
+            tokens_b=b.tokens,
+            token_index_b=b.token_index,
+            topk_jaccard=float(jaccard),
+            sparse_cosine=float(cosine),
+            shared_top_n=int(shared_top_n),
+            top_n=int(top_n),
             rows=rows,
+            chart_rows=chart_rows,
         )
 
 

featurelens/selection.py

@@ -0,0 +1,61 @@
+from __future__ import annotations
+
+import csv
+from collections import defaultdict
+from pathlib import Path
+
+
+def load_feature_sets(path: str | Path, max_size: int) -> dict[str, dict]:
+    """
+    Select a same-layer feature set for each concept from a feature catalog.
+
+    The layer whose best training feature is strongest is selected first. The top
+    distinct features from that same SAE dictionary are then returned. Keeping a
+    set within one layer is essential because decoder directions from different
+    residual spaces should not be summed into one intervention.
+    """
+    with Path(path).open(newline='', encoding='utf-8') as handle:
+        rows = list(csv.DictReader(handle))
+
+    grouped: dict[str, list[dict]] = defaultdict(list)
+    for row in rows:
+        item = dict(row)
+        item['layer'] = int(row['layer'])
+        item['feature_id'] = int(row['feature_id'])
+        item['train_auroc'] = float(row['train_auroc'])
+        item['activation_contrast'] = (
+            float(row['activation_rate_pos']) - float(row['activation_rate_neg'])
+        )
+        grouped[row['concept']].append(item)
+
+    result: dict[str, dict] = {}
+    for concept, concept_rows in grouped.items():
+        best_by_layer: dict[int, tuple[float, float]] = {}
+        for row in concept_rows:
+            key = (row['train_auroc'], row['activation_contrast'])
+            best_by_layer[row['layer']] = max(
+                best_by_layer.get(row['layer'], (-1.0, -1.0)),
+                key,
+            )
+        chosen_layer = max(best_by_layer, key=best_by_layer.get)
+        layer_rows = [row for row in concept_rows if row['layer'] == chosen_layer]
+        layer_rows.sort(
+            key=lambda row: (row['train_auroc'], row['activation_contrast']),
+            reverse=True,
+        )
+
+        seen: set[int] = set()
+        feature_ids: list[int] = []
+        for row in layer_rows:
+            feature_id = int(row['feature_id'])
+            if feature_id not in seen:
+                seen.add(feature_id)
+                feature_ids.append(feature_id)
+            if len(feature_ids) >= int(max_size):
+                break
+
+        result[concept] = {
+            'layer': int(chosen_layer),
+            'feature_ids': feature_ids,
+        }
+    return result

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "featurelens"
-version = "0.2.0"
+version = "0.3.0"
 description = "Causal sparse-feature interpretability workbench for Qwen3 and Qwen-Scope SAEs"
 requires-python = ">=3.10"
 

research_config.json

@@ -32,7 +32,7 @@
     "scale_2x"
   ],
   "negative_control": "norm-matched random residual direction",
-  "primary_causal_metric": "target first-token log-probability delta",
+  "primary_causal_metric": "full target continuation mean log-probability delta per token (teacher-forced)",
   "live_causal_controls": "norm-matched random residual direction",
   "dose_response_multipliers": [
     0.0,
@@ -45,5 +45,23 @@
   "statistical_inference": [
     "bootstrap_95_ci",
     "paired_sign_flip_test"
-  ]
+  ],
+  "live_features_v0_3": [
+    "full_continuation_scoring",
+    "joint_multi_feature_intervention",
+    "topk_feature_set_size_sweep",
+    "paraphrase_robustness_explorer"
+  ],
+  "feature_set_sizes": [
+    1,
+    3,
+    5
+  ],
+  "feature_set_interventions": [
+    "ablate",
+    "scale"
+  ],
+  "feature_set_negative_control": "norm-matched random residual direction",
+  "dose_response_execution": "batched residual edits in one model forward after baseline",
+  "feature_set_sweep_execution": "all SAE and random-control residual edits batched in one model forward after baseline"
 }

scripts/release_check.py

@@ -7,256 +7,179 @@ from pathlib import Path
 
 
 ROOT = Path(__file__).resolve().parents[1]
-
-# Any non-ignored repository file larger than this is suspicious.
-# FeatureLens should not contain model weights, SAE checkpoints,
-# activation dumps, virtual environments, etc.
 MAX_FILE_SIZE_BYTES = 5_000_000  # 5 MB
 
 REQUIRED = [
-    "README.md",
-    "app.py",
-    "requirements.txt",
-    "research_config.json",
-    "featurelens/runtime.py",
-    "featurelens/sae.py",
-    "featurelens/interventions.py",
-    "featurelens/stats.py",
-    "experiments/run_all.py",
-    "data/prompts.jsonl",
-    "data/causal_tasks.jsonl",
+    'README.md',
+    'app.py',
+    'requirements.txt',
+    'research_config.json',
+    'featurelens/runtime.py',
+    'featurelens/sae.py',
+    'featurelens/interventions.py',
+    'featurelens/metrics.py',
+    'featurelens/stats.py',
+    'experiments/run_all.py',
+    'experiments/run_causal.py',
+    'experiments/run_feature_sets.py',
+    'data/prompts.jsonl',
+    'data/causal_tasks.jsonl',
+    'docs/VALIDATION.md',
 ]
 
 
 def load_jsonl(path: Path) -> list[dict]:
-    """Load a JSONL file into a list of dictionaries."""
     return [
         json.loads(line)
-        for line in path.read_text(encoding="utf-8").splitlines()
+        for line in path.read_text(encoding='utf-8').splitlines()
         if line.strip()
     ]
 
 
 def repository_candidates() -> list[Path]:
-    """
-    Return files that are either:
-
-    - already tracked by Git, or
-    - untracked but not ignored by .gitignore.
-
-    This deliberately excludes files such as .venv contents when .venv/
-    is correctly listed in .gitignore.
-
-    Including untracked, non-ignored files is useful because it catches
-    accidental large files before someone runs `git add .`.
-    """
+    """Return tracked files plus untracked files that are not ignored by Git."""
     try:
         result = subprocess.run(
-            [
-                "git",
-                "ls-files",
-                "--cached",
-                "--others",
-                "--exclude-standard",
-            ],
+            ['git', 'ls-files', '--cached', '--others', '--exclude-standard'],
             cwd=ROOT,
             capture_output=True,
             text=True,
             check=True,
         )
     except FileNotFoundError as exc:
-        raise SystemExit(
-            "Git is required to run the FeatureLens release check."
-        ) from exc
+        raise SystemExit('Git is required to run the FeatureLens release check.') from exc
     except subprocess.CalledProcessError as exc:
-        stderr = exc.stderr.strip()
         raise SystemExit(
-            f"Could not inspect repository files with Git: {stderr}"
+            f'Could not inspect repository files with Git: {exc.stderr.strip()}'
         ) from exc
 
-    candidates: list[Path] = []
-
+    paths: list[Path] = []
     for relative_path in result.stdout.splitlines():
         relative_path = relative_path.strip()
-
         if not relative_path:
             continue
-
         path = ROOT / relative_path
-
-        # A tracked file may have been deleted locally but not yet committed.
-        # Such a path should not be size-checked.
         if path.is_file():
-            candidates.append(path)
-
-    return candidates
+            paths.append(path)
+    return paths
 
 
 def check_required_files() -> None:
-    """Ensure the repository contains all files required for a release."""
     missing = [path for path in REQUIRED if not (ROOT / path).exists()]
-
     if missing:
-        raise SystemExit(f"Missing required files: {missing}")
+        raise SystemExit(f'Missing required files: {missing}')
 
 
 def check_config(config: dict) -> None:
-    """Validate the research configuration expected by FeatureLens v0.2."""
-    if config.get("layers") != [4, 14, 26]:
+    expected = {
+        'layers': [4, 14, 26],
+        'model_id': 'Qwen/Qwen3-1.7B-Base',
+        'sae_width': 32768,
+        'dose_response_multipliers': [0.0, 0.5, 1.0, 1.5, 2.0, 3.0],
+        'feature_set_sizes': [1, 3, 5],
+    }
+    for key, value in expected.items():
+        if config.get(key) != value:
+            raise SystemExit(f'Unexpected {key}: {config.get(key)!r}. Expected {value!r}.')
+
+    required_live = {
+        'full_continuation_scoring',
+        'joint_multi_feature_intervention',
+        'topk_feature_set_size_sweep',
+        'paraphrase_robustness_explorer',
+    }
+    actual_live = set(config.get('live_features_v0_3', []))
+    if actual_live != required_live:
         raise SystemExit(
-            f'Unexpected layers: {config.get("layers")}. '
-            "Expected [4, 14, 26]."
-        )
-
-    if config.get("model_id") != "Qwen/Qwen3-1.7B-Base":
-        raise SystemExit(
-            f'Unexpected model_id: {config.get("model_id")}.'
-        )
-
-    if config.get("sae_width") != 32768:
-        raise SystemExit(
-            f'Unexpected sae_width: {config.get("sae_width")}. '
-            "Expected 32768."
-        )
-
-    expected_multipliers = [0.0, 0.5, 1.0, 1.5, 2.0, 3.0]
-
-    if config.get("dose_response_multipliers") != expected_multipliers:
-        raise SystemExit(
-            "Unexpected dose_response_multipliers: "
-            f'{config.get("dose_response_multipliers")}. '
-            f"Expected {expected_multipliers}."
+            'research_config.json live_features_v0_3 mismatch: '
+            f'{sorted(actual_live)}'
         )
 
 
 def check_datasets(config: dict) -> tuple[list[dict], list[dict]]:
-    """Validate discovery and causal benchmark datasets."""
-    prompts = load_jsonl(ROOT / "data" / "prompts.jsonl")
-    causal = load_jsonl(ROOT / "data" / "causal_tasks.jsonl")
-
-    expected_prompt_count = config.get("discovery_prompts")
-    expected_causal_count = config.get("causal_tasks")
+    prompts = load_jsonl(ROOT / 'data' / 'prompts.jsonl')
+    causal = load_jsonl(ROOT / 'data' / 'causal_tasks.jsonl')
 
-    if len(prompts) != expected_prompt_count:
+    if len(prompts) != config.get('discovery_prompts'):
         raise SystemExit(
-            "Discovery prompt count mismatch: "
-            f"found {len(prompts)}, expected {expected_prompt_count}."
+            f'Discovery prompt count mismatch: found {len(prompts)}, '
+            f'expected {config.get("discovery_prompts")}.'
         )
-
-    if len(causal) != expected_causal_count:
+    if len(causal) != config.get('causal_tasks'):
         raise SystemExit(
-            "Causal task count mismatch: "
-            f"found {len(causal)}, expected {expected_causal_count}."
+            f'Causal task count mismatch: found {len(causal)}, '
+            f'expected {config.get("causal_tasks")}.'
         )
 
-    concept_counts = Counter(row["concept"] for row in prompts)
-
-    expected_concepts = config.get("concepts", [])
-
-    if set(concept_counts) != set(expected_concepts):
-        raise SystemExit(
-            "Discovery dataset concepts do not match research_config.json.\n"
-            f"Dataset concepts: {sorted(concept_counts)}\n"
-            f"Config concepts:  {sorted(expected_concepts)}"
-        )
-
-    # The controlled discovery benchmark is intentionally balanced.
+    concept_counts = Counter(row['concept'] for row in prompts)
+    if set(concept_counts) != set(config.get('concepts', [])):
+        raise SystemExit('Discovery dataset concepts do not match research_config.json.')
     if len(set(concept_counts.values())) != 1:
-        raise SystemExit(
-            f"Discovery concepts are not balanced: {dict(concept_counts)}"
-        )
+        raise SystemExit(f'Discovery concepts are not balanced: {dict(concept_counts)}')
+
+    pair_counts = Counter(row['pair_id'] for row in prompts)
+    if set(pair_counts.values()) != {2}:
+        raise SystemExit('Every discovery paraphrase pair must contain exactly two prompts.')
 
     return prompts, causal
 
 
 def check_oversized_files() -> None:
-    """
-    Reject unexpectedly large files that could be committed/pushed.
-
-    Importantly, this does NOT recursively scan .venv or other ignored
-    directories. Git decides what counts as a repository candidate.
-    """
     oversized: list[str] = []
-
     for path in repository_candidates():
         size_bytes = path.stat().st_size
-
         if size_bytes > MAX_FILE_SIZE_BYTES:
-            relative_path = path.relative_to(ROOT)
-            size_mb = size_bytes / 1_000_000
-
-            oversized.append(
-                f"{relative_path} ({size_mb:.1f} MB)"
-            )
+            relative = path.relative_to(ROOT)
+            oversized.append(f'{relative} ({size_bytes / 1_000_000:.1f} MB)')
 
     if oversized:
-        formatted = "\n  - ".join(oversized)
-
+        formatted = '\n  - '.join(oversized)
         raise SystemExit(
-            "Repository contains unexpectedly large tracked/unignored "
-            "candidates:\n"
-            f"  - {formatted}\n\n"
-            "If a file is a legitimate local artifact, add it to .gitignore. "
-            "Model weights, SAE checkpoints, activation dumps, virtual "
-            "environments, and caches should not be committed."
+            'Repository contains unexpectedly large tracked/unignored candidates:\n'
+            f'  - {formatted}\n\n'
+            'If a file is a legitimate local artifact, add it to .gitignore. Model weights, '
+            'SAE checkpoints, activation dumps, virtual environments, and caches should not be committed.'
         )
 
 
 def check_readme() -> None:
-    """Validate important Hugging Face Space/release metadata."""
-    readme = (ROOT / "README.md").read_text(encoding="utf-8")
-
+    readme = (ROOT / 'README.md').read_text(encoding='utf-8')
     required_strings = [
-        "sdk: gradio",
+        'sdk: gradio',
         'sdk_version: "6.24.0"',
-        "Qwen/Qwen3-1.7B-Base",
-    ]
-
-    missing = [
-        value
-        for value in required_strings
-        if value not in readme
+        'Qwen/Qwen3-1.7B-Base',
+        'full-continuation',
+        'feature-set',
+        'paraphrase',
+        'norm-matched',
     ]
-
+    missing = [value for value in required_strings if value.lower() not in readme.lower()]
     if missing:
-        raise SystemExit(
-            f"README.md is missing required metadata/content: {missing}"
-        )
+        raise SystemExit(f'README.md is missing required v0.3 content: {missing}')
 
-    readme_lower = readme.lower()
 
-    if "norm-matched random" not in readme_lower:
-        raise SystemExit(
-            'README.md should describe the "norm-matched random" control.'
-        )
-
-    if "dose" not in readme_lower:
-        raise SystemExit(
-            "README.md should describe the dose-response experiment."
-        )
+def check_pyproject() -> None:
+    text = (ROOT / 'pyproject.toml').read_text(encoding='utf-8')
+    if 'version = "0.3.0"' not in text:
+        raise SystemExit('pyproject.toml must declare version 0.3.0.')
 
 
 def main() -> None:
     check_required_files()
-
-    config = json.loads(
-        (ROOT / "research_config.json").read_text(encoding="utf-8")
-    )
-
+    config = json.loads((ROOT / 'research_config.json').read_text(encoding='utf-8'))
     check_config(config)
-
     prompts, causal = check_datasets(config)
-
     check_oversized_files()
-
     check_readme()
+    check_pyproject()
 
-    print("FeatureLens release check: PASS")
-    print(f"  discovery prompts: {len(prompts)}")
-    print(f"  causal tasks: {len(causal)}")
+    print('FeatureLens release check: PASS')
+    print(f'  discovery prompts: {len(prompts)}')
+    print(f'  causal tasks: {len(causal)}')
     print(f'  layers: {config["layers"]}')
-    print("  release: v0.2.0")
+    print(f'  feature-set sizes: {config["feature_set_sizes"]}')
+    print('  release: v0.3.0')
 
 
-if __name__ == "__main__":
-    main()
+if __name__ == '__main__':
+    main()

tests/test_feature_sets.py

@@ -0,0 +1,21 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+from featurelens.selection import load_feature_sets
+
+
+def test_feature_set_selection_stays_within_one_best_layer(tmp_path: Path) -> None:
+    catalog = tmp_path / 'feature_catalog.csv'
+    catalog.write_text(
+        'layer,concept,feature_id,train_auroc,auroc,f1,activation_rate_pos,activation_rate_neg\n'
+        '4,math,10,0.80,0.75,0.70,0.8,0.2\n'
+        '14,math,20,0.92,0.82,0.80,0.9,0.1\n'
+        '14,math,21,0.90,0.81,0.79,0.8,0.2\n'
+        '14,math,22,0.88,0.80,0.78,0.7,0.2\n'
+        '26,math,30,0.85,0.79,0.76,0.8,0.3\n',
+        encoding='utf-8',
+    )
+    selected = load_feature_sets(catalog, max_size=3)
+    assert selected['math']['layer'] == 14
+    assert selected['math']['feature_ids'] == [20, 21, 22]

tests/test_interventions.py

@@ -1,8 +1,14 @@
 from __future__ import annotations
 
+import pytest
 import torch
 
-from featurelens.interventions import InterventionSpec, normalized_random_control, residual_delta
+from featurelens.interventions import (
+    InterventionSpec,
+    joint_residual_delta,
+    normalized_random_control,
+    residual_delta,
+)
 
 
 def test_ablation_delta() -> None:
@@ -27,6 +33,42 @@ def test_residual_delta_is_decoder_direction_times_coefficient_delta() -> None:
     assert torch.allclose(delta, torch.tensor([-3.0, -6.0, 3.0]))
 
 
+def test_joint_ablation_sums_feature_deltas() -> None:
+    directions = torch.tensor(
+        [
+            [1.0, 0.0, 2.0],
+            [0.0, 1.0, -1.0],
+        ]
+    )
+    delta, coefficient_deltas = joint_residual_delta(
+        directions,
+        [2.0, 3.0],
+        InterventionSpec('ablate', 0.0),
+    )
+    assert coefficient_deltas == [-2.0, -3.0]
+    assert torch.allclose(delta, torch.tensor([-2.0, -3.0, -1.0]))
+
+
+def test_joint_scale_uses_same_multiplier_per_feature() -> None:
+    directions = torch.eye(2)
+    delta, coefficient_deltas = joint_residual_delta(
+        directions,
+        [2.0, 4.0],
+        InterventionSpec('scale', 1.5),
+    )
+    assert coefficient_deltas == [1.0, 2.0]
+    assert torch.allclose(delta, torch.tensor([1.0, 2.0]))
+
+
+def test_joint_injection_is_rejected() -> None:
+    with pytest.raises(ValueError, match='only'):
+        joint_residual_delta(
+            torch.eye(2),
+            [1.0, 1.0],
+            InterventionSpec('inject', 2.0),
+        )
+
+
 def test_random_control_matches_norm_and_is_deterministic() -> None:
     delta = torch.tensor([3.0, 4.0, 0.0])
     a = normalized_random_control(delta, seed=7)

tests/test_metrics.py

@@ -1,8 +1,17 @@
 from __future__ import annotations
 
+import math
+
 import torch
 
-from featurelens.metrics import js_divergence_from_logits, reconstruction_metrics, safe_log_probability
+from featurelens.metrics import (
+    js_divergence_from_logits,
+    reconstruction_metrics,
+    safe_log_probability,
+    sequence_logprob_summary,
+    sparse_topk_cosine,
+    target_token_logprobs,
+)
 
 
 def test_reconstruction_metrics_perfect_match() -> None:
@@ -20,3 +29,49 @@ def test_js_divergence_zero_for_identical_logits() -> None:
 
 def test_safe_log_probability_is_finite_at_zero() -> None:
     assert safe_log_probability(0.0) < 0.0
+
+
+def test_target_logprobs_score_every_continuation_token() -> None:
+    # prompt length = 2, target ids = [1, 0].
+    # Target token 0 is predicted from logits row 1; token 1 from row 2.
+    logits = torch.tensor(
+        [
+            [0.0, 0.0],
+            [0.0, 2.0],
+            [3.0, 0.0],
+            [0.0, 0.0],
+        ]
+    )
+    values = target_token_logprobs(logits, prompt_length=2, target_ids=[1, 0])
+    expected_0 = torch.log_softmax(logits[1], dim=-1)[1]
+    expected_1 = torch.log_softmax(logits[2], dim=-1)[0]
+    assert torch.allclose(values, torch.stack([expected_0, expected_1]))
+
+
+def test_sequence_summary_total_and_mean_are_consistent() -> None:
+    logits = torch.tensor(
+        [
+            [0.0, 0.0],
+            [0.0, 2.0],
+            [3.0, 0.0],
+            [0.0, 0.0],
+        ]
+    )
+    total, mean, token_values = sequence_logprob_summary(
+        logits,
+        prompt_length=2,
+        target_ids=[1, 0],
+    )
+    assert len(token_values) == 2
+    assert math.isclose(total, sum(token_values), rel_tol=1e-6)
+    assert math.isclose(mean, total / 2.0, rel_tol=1e-6)
+
+
+def test_sparse_topk_cosine_is_one_for_identical_sparse_vectors() -> None:
+    cosine = sparse_topk_cosine([1, 3], [2.0, 1.0], [1, 3], [2.0, 1.0])
+    assert abs(cosine - 1.0) < 1e-9
+
+
+def test_sparse_topk_cosine_is_zero_for_disjoint_support() -> None:
+    cosine = sparse_topk_cosine([1, 3], [2.0, 1.0], [2, 4], [5.0, 7.0])
+    assert cosine == 0.0