paper/REVISION_NOTE.md

# REVISION_NOTE.md — BABEL paper v1 → v1.1 (tracked-change summary)

**Date:** 2026-07-08 · **By:** REVISION CC · **Governing brief:** REVISION_BRIEF_2026-07-07.md
(STAGE 1) · **External input:** STANFORD_REVIEW.md (paperreview.ai, 2026-07-07).
**Scope:** Stage 1 only — reanalysis of *existing frozen artifacts* + text. **No new GPU
experiments** (those are proposals in STAGE2_PROPOSAL.md; Will greenlights each). **Nothing
publishes** — `PAPER_V1_1_DRAFT.md` stays a draft until Will fires the Zenodo-v2 / GitHub update.
**No frozen number was altered and no artifact was renamed;** every added statistic is a new
reanalysis of frozen data, pre-registered where required and logged propose-only in the pen.

Source of the revised draft: `PAPER_V1_1_DRAFT.md` (byte-copy of `PAPER_DRAFT_V1.md`, then the
edits below). Reanalysis harnesses + outputs: `_rev_fdr.py`/`_rev_fdr.out`,
`_rev_fdr_channel.py`/`_rev_fdr_channel.out`, `_rev_floor.py`/`_rev_floor.out`. Pen blocks (all
propose-only): `REVISION-FDR — PRE-REGISTRATION`, `REVISION-FDR — RESULTS`,
`REVISION-FLOOR — RESULTS` (2026-07-08).

---

## A. Substantive reanalyses (Stage 1A)

### 1. FDR / FWER control on the naming battery (review Q2 — top priority)
- **Pre-registered before computing** (pen `REVISION-FDR — PRE-REGISTRATION`, locked α = 0.05 BH,
  sensitivity 0.01/0.10, tail-model + re-derivation rule) — brief item 1 satisfied.
- **Validation gate passed:** re-deriving NAMED from the raw stored statistics with the frozen
  `obs > null95` rule reproduces the frozen L1 verdicts **0 mismatches / 312** → the BH
  re-derivation is trustworthy.
- **Result (312 channels with frozen per-gate stats; frozen NAMED 149 = 47.8 %):** per-gate BH
  q = 0.05 → **6.7 %** named (q = 0.01 → 1.9 %, q = 0.10 → 16.3 %); channel-level (conjunction-aware,
  post-hoc) BH q = 0.05 → **25.6 %** (q = 0.01 → 9.0 %, q = 0.10 → 35.6 %); Holm FWER → 1 channel;
  global-null assumption-free bound → FDP ≈ **10 %**. N = 20 nulls floor the empirical p at ≈ 0.048
  → continuous p are tail-model extrapolations; a definitive FDR is a Stage-2 high-N re-draw.
- **Draft changes:** Abstract naming sentence re-scoped + corrected fractions; **new §6.1
  multiplicity paragraph + table**; §6.5 number 3 re-scoped; **new Appendix B** (full procedure,
  both null models, Holm, global-null, scope). The 53.6 %/46.4 % figures are **retained** as the
  frozen gate-level record and everywhere re-labelled "under the L1 σ-null battery at N = 20".

### 2. Floor-construction sensitivity for the 39/39 closure (review Q1)
- **Result:** 36/39 legacy closure is floor-construction-robust; **39/39 is reached only at the
  full norm-relative recalibration (β = 1)** — β ≤ 0.5 → 36/39, β = 0.75 → 37/39. The three
  recal-only cells (code_b12, rep_b11, rep_b12) close because the norm-scaled floor there is
  1.5–2.7× the legacy floor (tracks ρ²) — documented norm-geometry, meter-dependent, labelled so.
  Recal closure tolerates loosening (39/39 for c ≥ 1) but is sensitive to a 10 % tightening
  (34/39 at c = 0.9); 13/39 cells pass "tight" (KL/floor 0.8–1.0).
- **Honest gap:** per-token substitution KL is **not frozen** (only per-cell means) → the
  bootstrap-CI-vs-floor half of Q1 is **not computable CPU-only**; deferred to Stage 2.
- **Draft changes:** Abstract 39/39 clause scoped; **new §4.1 floor-sensitivity paragraph**;
  §8.5 sensitivity note; **new Appendix C** (β-sweep, interpolation, multiplier, margin bands, the
  3 gap cells, the CI constraint).

### 3. Student (executable-rung) methods appendix (review Q6)
- Extracted every training/val detail from `_v6_result.json`/`_v7_result.json`/`_v6.py`/`_v7.py`,
  **all numbers verified against the frozen JSON**.
- Key clarification for the reviewer: the linear student is **Adam-MSE (lr = 1e-3, 4000 steps),
  with NO ridge / weight decay** — its regularizer is *capacity*, not a penalty. Splits: 96 train
  periods (seeds 7000–7095), 16 SACRED (seed 3, never-seen), 16 HOLD2 (8000–8015), period 64.
  Twin = permuted rep-era targets; pass = real ≤ 0.5 × twin.
- **Draft change:** **new Appendix D** (task/inputs, data/splits, fit, architectures/params, twin,
  full results table, non-recoverable list).

### 4. No-gloss scope reframe (review Q7)
- **Draft change:** §8.4 rewritten — every "word-less" claim now means *no gloss under the
  CH-WU/CH-INT/CH-FIELD battery at N = 20*, not "provably carries no content"; named battery
  **extensions** listed as future work (syntax/dependency probes, morphology, non-vocabulary
  probes, causal-scrubbing readouts, SAE decoders; §9). Two bounding facts stated: the FDR
  re-analysis shows the battery is if anything *generous* (so no-gloss is more likely an under- than
  over-estimate), and L6's dark-complement edge (2/8 faint) shows the honest limit.

## B. Text changes (Stage 1B)

- **5. Scope (title/abstract).** Title keeps "State Space" (unchanged). Abstract sentence 2 adds an
  explicit boundary-grain / within-block scope line; §1.4 already carried it (unchanged).
- **6. Glossary.** New **§1.5** — one-page neologism table (door, seam, meter, floor, folded read,
  dark mass, corridor, core, rung, gloss, no-gloss, transplant, gauge/lever, twin, DEAF axis, …).
- **7. Related work.** New **§7 paragraph** adding causal scrubbing (Chan et al., AF 2022), DLA /
  attribution patching (2310.10348) / ACDC (2304.14997), and mechanistic benchmarks (Tracr
  2301.05062, RAVEL Huang 2024), each mapped to the Table-1 four axes where honest. (SAE decoding
  and probing debates were already in §7; unchanged.)
- **8. Artifacts (Q8).** Every "pending release" replaced with the **live** release: Zenodo DOI
  10.5281/zenodo.21230108 (concept 10.5281/zenodo.21230107), github.com/wpferrell/babel-codec-gpt2,
  huggingface.co/wpferrell/babel-codec-gpt2 — header line, §7 falsifiability line ([GITHUB-URL]),
  and §10 artifact statement.

## C. Editor notes / NOTES-for-Will

- New NOTES-for-Will item 12 added to the draft's review-comment block (delete before release).
- **New §7 citations need the same live-page verification** as the existing ones before release:
  causal scrubbing (Alignment Forum, no arXiv), attribution patching (2310.10348), ACDC
  (2304.14997), Tracr (2301.05062), RAVEL (Huang et al., 2024 — exact ID to confirm).

## D. What did NOT change
- No frozen result, floor, hash, or artifact name. The L5/L6 certified negatives, the 39/39 recal
  headline, the seam law, the transplant 94.7 %, and the wall student all stand. The revision
  *scopes and stress-tests* the naming and closure claims; it does not move them.

## E. Deliverables produced
`PAPER_V1_1_DRAFT.md` · `REVISION_NOTE.md` (this) · `REVIEWER_RESPONSE.md` · `STAGE2_PROPOSAL.md` ·
reanalysis harnesses/outputs (`_rev_fdr*.py/.out`, `_rev_floor.py/.out`) · pen propose-only blocks.


## F. Stage-2 GPU experiments — folded 2026-07-08 (Stage-2 Revision CC)

Will greenlit three Stage-2 items (STAGE2_GREENLIGHT.md); all three ran, each with a byte-replay gate
to the frozen number and pre-registered bands (pen: STAGE2-A/B/C blocks). Results folded into the draft
and REVIEWER_RESPONSE; frozen artifacts untouched (READ-ONLY); propose-only; nothing published.

- **Item B — transplant boundary×regime (Q4; `_s2b.py`).** Gate: b6/prose 16-pair closure = 0.9467
  (dev 0.0). 3×3 grid (b2/b6/b10 × prose/code/repetition), 32 pairs/cell, matched-random null, 10k
  bootstrap CIs. Result: boundary- and prose/repetition-general (median closure 0.82–0.98 across depth);
  code heavy-tailed at early/mid depth (median 0.70/0.60, mean collapsed by outlier pairs), clean at b10
  (0.89). → Abstract + §6.4 table + §8 item 9 scoped; Reviewer Q4 answered.
- **Item C — seam perturbation (Q5; `_s2c.py`).** Gate: global-core cert byte-replays all 36 frozen
  KL_LIN (max dev 0.0). C2a within-span rotation exactly invariant; C2b ±ε jitter 30/30 draws keep
  36/36 TIGHT (ε≤0.10); C1 independent per-seam re-derivation keeps 33/36 LINEAR-TIGHT — the 3 failures
  are all the embed→L0 rewrite seam (prose/code LINEAR-not-TIGHT, repetition BROKEN). → §6.2 amended:
  the linear composition law is a robust *model* property at the 33 propagation seams, field-conditioned
  at the b0 rewrite seam. Reviewer Q5 answered.
- **Item A — door/core rotation robustness (Q3; `_s2a.py`).** Gate: S4 readable + full rank-48 fold KL
  byte-replay all 4 sampled cells (dev 0.0). Across R=20 random orthonormal bases/object: folded-read
  reconstruction KL invariant to 0.0, k* shift 0 (k*=16/40/32/24), 0/80 pass-flips, door/core footprints
  invariant to ≤3e-5, matched-random reconstruction discriminator separates. Core-field per-axis
  identities drift under within-span rotation (0/19) → verdict PARTIAL: necessity/k*/folded-reads
  rotation-stable, the 19 field labels a privileged basis of an intrinsic subspace. → §6.1 note added;
  §8 item 9; Reviewer Q3 answered.

Harnesses/outputs: `_s2b.py`/`_s2b_result.json`, `_s2c.py`/`_s2c_result.json`, `_s2a.py`/`_s2a_result.json`,
logs `_s2{a,b,c}.log`, launch bats `_s2{a,b,c}_run.bat`; pen propose-only pre-registration + results blocks
(STAGE2-A/B/C). None moves a frozen headline; each *scopes* a claim, per the pre-registered kill branches.
DATALAKE registered at closeout. NOTHING published — v1.1 stays draft until Will fires Zenodo v2 / GitHub.