Upload README.md with huggingface_hub

README.md

@@ -47,6 +47,9 @@ Validation (post-training, 64 documents per slice):
 | PSF-Med clinical questions (unseen) | 0.9648 | 55.0 | target distribution for circuit tracing |
 | WikiText-103 (out-of-domain) | 0.8840 | 36.1 | strongly domain-specialized |
 
+At n=256 documents per slice with per-document scoring, the out-of-domain EV is
+0.9391 (see the paired domain-contrast table below).
+
 ## Entry 2: Layer-29 transcoder (`layer29_transcoder_final.pt`)
 
 - **Motivation:** layer 29 carries Feature 12139, the downstream yes/no decision
@@ -63,8 +66,51 @@ Validation (post-training, 64 documents per slice):
 | PSF-Med clinical questions (unseen) | 0.9627 | 60.1 | target distribution for circuit tracing |
 | WikiText-103 (out-of-domain) | 0.9611 | 34.7 | largely domain-general |
 
-Note the contrast between the two entries: the layer-17 transcoder loses ~11 EV
-points off-domain while the layer-29 transcoder loses ~4, evidence that the
+## Domain specialization (n=256 per slice, paired by document)
+
+Same 256 documents scored by both transcoders per slice; gaps are paired means
+with bootstrap 95% CIs.
+
+| Slice | L17 EV | L29 EV | Paired gap (L29 − L17) [95% CI] |
+|---|---|---|---|
+| ReXGradient (in-distribution) | 0.9977 | 0.9941 | −0.0036 [−0.0050, −0.0024] |
+| PSF-Med questions (unseen) | 0.9735 | 0.9645 | −0.0091 [−0.0115, −0.0066] |
+| WikiText-103 (out-of-domain) | 0.9391 | 0.9642 | +0.0251 [+0.0213, +0.0291] |
+
+The EV ordering flips exactly at the domain boundary: the layer-17 (register)
+transcoder reconstructs better on medical text, the layer-29 (decision)
+transcoder generalizes better off-domain. The register computation is medically
+specialized; the late decision computation is more generic.
+
+## Instrument-change test (2026-08-18)
+
+These transcoders were substituted for Gemma Scope 2 in the PSF-Med canonical
+two-stage circuit analysis (Feature 3818 at L17 → Feature 12139 at L29), with
+identical pairs, arms, and statistics, and hypothesis features re-discovered on
+the original discovery split:
+
+- Single-feature restoration collapses from 17–58% (Gemma Scope 3818/12139) to
+  0–3% (local analogs) on identical pairs, while non-flip disruption stays ~0
+  for both instruments.
+- Flip-separating information is present observationally (|delta| AUROC up to
+  0.885 frozen / 0.948 split-half, comparable to GS 3818's 0.72–0.79) but does
+  not restore causally at single-feature level.
+- A weak mediation trace survives on MIMIC only (33/50 direction-coherent,
+  p=6e-04; PadChest at chance).
+- Caveat: instruments differ in hookpoint (MLP block vs resid_post) and decoder
+  norm convention (unconstrained vs unit-norm), so the comparison confounds
+  feature causality with intervention geometry. Full analysis:
+  `results/sae_analysis/LOCALSCOPE_INSTRUMENT_CHANGE_SUMMARY.md` in the GitHub
+  repo.
+
+The two-stage single-feature circuit as previously characterized is therefore
+an artifact of the Gemma Scope instrumentation; the correlational signal
+transfers across instruments, the causal single-feature account does not.
+
+Note the contrast between the two entries: the layer-17 transcoder loses ~6 EV
+points off-domain while the layer-29 transcoder loses ~4, and on the paired
+n=256 contrast the ordering inverts exactly at the domain boundary (table
+above), evidence that the
 register computation at layer 17 is medically specialized while the late decision
 computation at layer 29 is more generic.