Verify NA-MQ hierarchy with general stochastic coupling

outputs/namq_rdspac_coupling_audit.json

@@ -0,0 +1,55 @@
+{
+  "audit": "multiplicity-weighted NA-MQ to R-DS-PAC coupling",
+  "seed": 20260722,
+  "formula": "N=ceil(8*m*log(2*m/delta)); D'_Q(u)=c_u/m",
+  "coupon_stress": {
+    "trials": 10000,
+    "delta_per_trial": 0.05,
+    "multiplicity_modes": {
+      "all_distinct": 2500,
+      "one_heavy": 2500,
+      "zipf": 2500,
+      "dirichlet": 2500
+    },
+    "coupon_failures": 0,
+    "empirical_failure_rate": 0.0,
+    "worst_minimum_surplus": 23,
+    "largest_chernoff_union_bound": 1.043990016933941e-06
+  },
+  "stochastic_label_transcript_stress": {
+    "trials": 50000,
+    "query_order": [
+      0,
+      1,
+      0,
+      2,
+      1,
+      0
+    ],
+    "multiplicities": [
+      3,
+      2,
+      1
+    ],
+    "label_probabilities": [
+      0.2,
+      0.6,
+      0.85
+    ],
+    "training_draws": 264,
+    "coverage_failures": 0,
+    "valid_transcripts": 50000,
+    "max_direct_vs_reconstructed_mean_gap": 0.0020799999999999708,
+    "max_direct_vs_reconstructed_covariance_gap": 0.0020765559999999664,
+    "max_reconstructed_vs_target_mean_gap": 0.0009799999999999809
+  },
+  "checks": {
+    "all_coupon_stress_trials_covered": true,
+    "analytic_union_bound_below_delta": true,
+    "stochastic_transcripts_covered": true,
+    "transcript_means_match": true,
+    "transcript_covariances_match": true,
+    "reconstructed_means_match_target": true
+  },
+  "all_checks_pass": true
+}

outputs/theorem_5_2_5_3_proof_certificate.md

@@ -0,0 +1,39 @@
+# Proof certificate: Theorems 5.2 and 5.3
+
+This certificate reconstructs the two membership-query reductions in their full finite-sample form. It also repairs an omission in the printed proof of Theorem 5.3 without changing its asymptotic sample bound.
+
+## Theorem 5.2: D-DS-PAC implies NA-MQ
+
+Fix the target distribution `D`, hypothesis class `H`, and a D-DS-PAC learner whose deterministic training distribution is `D'` and whose sample size is `m`. Before observing any labels, an NA-MQ learner draws an ordered feature batch `(X_1,...,X_m) ~ (D')^m`. It submits that whole batch non-adaptively and passes the returned labeled sequence to the D-DS-PAC learner. Conditional on the features, each returned label is an independent draw from the stochastic labeling rule `f(X_i)`. Hence the simulated transcript has exactly the law `(D',f)^m`, including arbitrary label noise, and sample complexity and running time are unchanged.
+
+## Preprocessing a general NA-MQ transcript
+
+Definition 5.1 permits both random target examples and an independently randomized query batch. Generate every feature before seeing labels: draw the `m_rand` feature points from `D`, draw the batch from `Q`, and concatenate them in their original ordered positions. This is one randomized nonadaptive list `Q=(x_1,...,x_m)`, independent of `f`. Thus random target examples require no separate mechanism. Let `c_u` be the multiplicity of each distinct `u` in the combined list, so `sum_u c_u=m`.
+
+## Theorem 5.3: NA-MQ implies R-DS-PAC
+
+For a realized list `Q`, choose the training distribution
+
+`D'_Q(u) = c_u/m`.
+
+Set `delta=epsilon/2`, `L=log(2m/delta)`, and draw
+
+`N = ceil(8 m L)`
+
+i.i.d. labeled examples from `(D'_Q,f)`. For a distinct point `u`, its count `X_u` is binomial with mean at least `8 c_u L`. A multiplicative Chernoff bound gives
+
+`P[X_u<c_u] <= exp(-c_u(a-1)^2/(2a))`, where `a=8L`.
+
+Since `a>=4`, `(a-1)^2/(2a)>=a/4`; therefore this probability is at most `exp(-2 c_u L) <= (delta/(2m))^2 <= delta/m`. A union bound over at most `m` distinct points makes the simultaneous failure probability at most `delta`.
+
+On the coverage event, take the first `c_u` observed labels for each `u` and put them back into the original positions of `Q`. Counts depend only on sampled features, so conditioning on coverage does not affect labels. The selected labels are independent draws from `f(u)`, including separate fresh labels for duplicate occurrences. The reconstructed ordered transcript therefore has exactly the law expected by the original NA-MQ learner.
+
+Run the original learner at accuracy `epsilon/2`; on the failure event return any fixed hypothesis. For loss bounded by one,
+
+`E[excess loss] <= epsilon/2 + delta = epsilon`.
+
+Also `N=O(m_0(epsilon/2)(log m_0(epsilon/2)+log(1/epsilon)))`. If the original list is randomized, the induced law of `D'_Q` is the permitted randomized training-distribution meta-distribution, giving R-DS-PAC. If it is deterministic, `D'_Q` is fixed, giving D-DS-PAC.
+
+## Printed-proof audit
+
+The manuscript instead makes `D'_Q` uniform on the support and retains only the first label at each point. That construction is exact only when queries are distinct or duplicate labels are deterministic. Multiplicity weighting and retaining `c_u` labels close the gap for duplicated queries and stochastic labels. The theorem statement and stated asymptotic rate survive this repair.

pages/claim-5-membership-query-hierarchy/page.md

@@ -5,7 +5,7 @@
 <!-- trackio-cell
 {"type": "markdown", "id": "cell_3e53ff8de963", "created_at": "2026-07-17T04:36:14+00:00", "title": "Question and test"}
 -->
-The prompt states the D-DS-PAC→NA-MQ and NA-MQ→R-DS-PAC hierarchy and coupon-collector sample bound. We check the exact theorem text and simulate the pure finite-query coverage mechanism, while auditing whether the printed proof covers random examples, duplicated queries, and stochastic labels allowed by its definition.
+The prompt states the D-DS-PAC→NA-MQ and NA-MQ→R-DS-PAC hierarchy and coupon-collector sample bound. We reconstruct both reductions for the full Definition 5.1 model, including random target examples, duplicated queries, randomized batches, and stochastic labels, then stress-test the finite coupon event and transcript law.
 
 
 ---
@@ -14,17 +14,25 @@ The prompt states the D-DS-PAC→NA-MQ and NA-MQ→R-DS-PAC hierarchy and coupon
 -->
 Claim under test (verbatim): The paper establishes a hierarchy of learnability notions showing D-DS-PAC learning implies non-adaptive membership query (NA-MQ) learning, and NA-MQ learners can be converted into R-DS-PAC learners with sample complexity m(epsilon) = O(m0(epsilon/2) * (log m0(epsilon/2) + log(1/epsilon))) (Theorems 5.2 and 5.3).
 
-Source locators: Theorems 5.2 and 5.3 and Appendix E. Both implication statements and the supplied coupon-collector order appear in the manuscript.
+Source locators: Definition 5.1, Theorems 5.2 and 5.3, and Appendix E in [arXiv:2602.08907v2](https://arxiv.org/abs/2602.08907). Both implication statements and the claimed coupon-collector order appear in the manuscript.
 
-Executable audit: for a pure finite nonadaptive query set with deterministic labels, the coupon mechanism reaches mean coverage probability 0.9573 at the tested theorem-style bound, consistent with a 0.95 target. The printed proof does not account for the same definition's random target examples, duplicate queries, or independent stochastic labels: seeing a query once does not recreate multiple independent label draws. Therefore the broad theorem was not reproduced from this narrow coverage calculation.
+**Theorem 5.2, exact coupling.** A D-DS-PAC learner samples an ordered batch from its fixed training distribution `D'`. An NA-MQ learner draws those feature points before observing labels, submits the batch nonadaptively, and passes the returned labels to the original learner. Conditional on the features, the labels are independent draws from `f(x)`, so the transcript law, label noise, sample complexity, and running time are identical.
 
-Evidence: `outputs/hf_t4/claim5_coupon.csv` and `reports/proof_audit.md`.
+**Theorem 5.3, general coupling.** First draw the random target features and randomized query batch, all before labels, and concatenate them into one ordered list `Q`. If point `u` occurs `c_u` times, use the randomized training distribution `D'_Q(u)=c_u/m`, rather than a distribution uniform on the distinct support. With `delta=epsilon/2` and `N=ceil(8m log(2m/delta))`, a Chernoff bound and union bound give probability at least `1-delta` that every `u` occurs at least `c_u` times. Taking the first `c_u` independently drawn labels and restoring their ordered positions exactly reconstructs duplicated queries and stochastic labels. Running the NA-MQ learner at `epsilon/2`, with bounded-loss cost at most `delta` on failure, gives excess risk at most `epsilon` and
 
-Verdict: PARTIALLY SUPPORTED; NARROW REDUCTION REPRODUCED
+`N = O(m_0(epsilon/2)(log m_0(epsilon/2)+log(1/epsilon)))`.
+
+For randomized `Q`, its induced law over `D'_Q` is precisely an R-DS-PAC meta-distribution; for deterministic `Q`, the distribution is fixed and yields D-DS-PAC. The complete inequality chain and coupling are frozen in `outputs/theorem_5_2_5_3_proof_certificate.md`.
+
+**Independent executable audit.** `scripts/namq_rdspac_coupling_audit.py` tests 10,000 batches spanning all-distinct, one-heavy, Zipf, and Dirichlet multiplicities and 50,000 ordered stochastic-label transcripts with duplicates. It evaluates the finite Chernoff certificate, not only an asymptotic order. Frozen results are in `outputs/namq_rdspac_coupling_audit.json`.
+
+**Proof audit disclosure.** Appendix E uses a distribution uniform over the support and retains only the first label. That printed construction omits multiplicities and is not exact for duplicated stochastic queries. The multiplicity-weighted construction above repairs the proof without changing the theorem statement or asymptotic bound.
+
+Verdict: **VERIFIED**, with a documented repair to the manuscript's printed proof.
 
 
 ---
 <!-- trackio-cell
 {"type": "markdown", "id": "cell_cc273205d4f8", "created_at": "2026-07-17T05:15:09+00:00", "title": "Normalized source locator"}
 -->
-Strict source locators: Theorem 5.2; Theorem 5.3; Appendix E. The reproduced coverage calculation applies only to a pure finite query set with deterministic labels; stochastic labels and random target examples remain outside the executable check.
+Strict source locators: Definition 5.1; Theorem 5.2; Theorem 5.3; Appendix E. The proof certificate covers arbitrary randomized nonadaptive lists and stochastic labels; the executable audit exercises duplicate multiplicities and ordered stochastic-label reconstruction.

pages/conclusion/page.md

@@ -12,7 +12,7 @@ This page consolidates the exact claim verdicts, scope and cost, checksum-verifi
 <!-- trackio-cell
 {"type": "markdown", "id": "cell_dba782593110", "created_at": "2026-07-17T05:10:20+00:00", "title": "Executive summary", "pinned": true, "pinned_at": "2026-07-17T05:10:21+00:00"}
 -->
-This reproduction does not confirm the six-claim package: Claim 1 is falsified as written by a mismatched formula and exact endpoint failure; Claims 2 and 3 are also falsified by moment errors and a junta counterexample. Claim 4 remains unresolved beyond a failed decoder proof component, Claim 5 has narrow toy support, and Claim 6's computational framing is verified. The upgraded empirical result is a five-seed paired campaign at `d=50`, width 1024, and 100,000 steps per arm. For parity-25, PDS reached mean clean-test accuracy **0.9385 +/- 0.0196 SD** versus **0.5006 +/- 0.0025** for uniform training; all 5/5 PDS runs and 0/5 uniform runs exceeded 0.80. For the paper's `f9` junta, PDS reached **0.8795 +/- 0.2066** versus **0.5052 +/- 0.0036**; PDS exceeded 0.80 in 4/5 runs while uniform did so in 0/5. All five paired effects were positive, but one PDS junta run ended at 0.5116. This is strong finite optimization-mechanism evidence, not an independent proof of the complexity theorems.
+This reproduction does not confirm the six-claim package: Claim 1 is falsified as written by a mismatched formula and exact endpoint failure; Claims 2 and 3 are also falsified by moment errors and a junta counterexample. Claim 4 remains unresolved beyond a failed decoder proof component. **Claim 5 is verified** by a complete multiplicity-weighted coupling that covers Definition 5.1's random examples, duplicate queries, randomized query batches, and stochastic labels while preserving the stated sample rate; it also identifies and repairs an omission in Appendix E's uniform-support construction. Claim 6's computational framing is verified. The upgraded empirical result is a five-seed paired campaign at `d=50`, width 1024, and 100,000 steps per arm. For parity-25, PDS reached mean clean-test accuracy **0.9385 +/- 0.0196 SD** versus **0.5006 +/- 0.0025** for uniform training; all 5/5 PDS runs and 0/5 uniform runs exceeded 0.80. For the paper's `f9` junta, PDS reached **0.8795 +/- 0.2066** versus **0.5052 +/- 0.0036**; PDS exceeded 0.80 in 4/5 runs while uniform did so in 0/5. All five paired effects were positive, but one PDS junta run ended at 0.5116. This is strong finite optimization-mechanism evidence, not an independent proof of the complexity theorems.
 
 ## Scope & cost
 

pages/executive-summary/page.md

@@ -5,7 +5,7 @@
 <!-- trackio-cell
 {"type": "markdown", "id": "cell_09d93ca3dd30", "created_at": "2026-07-21T17:35:14+00:00", "title": "Executive summary", "pinned": true, "pinned_at": "2026-07-21T17:35:15+00:00"}
 -->
-This five-seed reproduction strongly supports the finite optimization mechanism behind positive distribution shift while preserving all theorem-level caveats. At d=50 and 100,000 steps, parity-25 reached 0.9385 +/- 0.0196 SD with PDS versus 0.5006 +/- 0.0025 with uniform training (5/5 versus 0/5 above 0.80). The paper's f9 junta reached 0.8795 +/- 0.2066 versus 0.5052 +/- 0.0036 (4/5 versus 0/5 above 0.80); one disclosed PDS seed remained near chance. **Claim 1 is nevertheless falsified as written:** its complexity formula is not the theorem's formula, and the claimed explicit decoder fails valid empty/full-support parity endpoints, with 0% full-support recovery. Exact source/proof audits also identify decoder and generality problems elsewhere, so finite experiments are not presented as proofs of the complexity theorems.
+This five-seed reproduction strongly supports the finite optimization mechanism behind positive distribution shift while preserving all theorem-level caveats. At d=50 and 100,000 steps, parity-25 reached 0.9385 +/- 0.0196 SD with PDS versus 0.5006 +/- 0.0025 with uniform training (5/5 versus 0/5 above 0.80). The paper's f9 junta reached 0.8795 +/- 0.2066 versus 0.5052 +/- 0.0036 (4/5 versus 0/5 above 0.80); one disclosed PDS seed remained near chance. **Claim 1 is nevertheless falsified as written:** its complexity formula is not the theorem's formula, and the claimed explicit decoder fails valid empty/full-support parity endpoints, with 0% full-support recovery. Claim 5 is now independently verified by a full multiplicity-weighted coupling for arbitrary randomized nonadaptive batches and stochastic labels; this proof repairs a support-uniformity omission in Appendix E without changing the stated rate.
 
 ## Scope & cost
 
@@ -13,8 +13,8 @@ This five-seed reproduction strongly supports the finite optimization mechanism
 |---|---|---|
 | Neural scope | 2 tasks x 2 regimes x 5 paired seeds; d=50; width 1024; 100k steps; 6.4M fresh examples/arm | Paper's maximum displayed 1M-step protocol |
 | Compute | Seed 0 public T4 Job; seeds 1-4 local H100 | Not fully reported |
-| Theory | Frozen source audit plus exact/Monte Carlo proof-component tests | Independent reconstruction of all formal proofs |
-| Outcome | Large parity effect; large but unstable junta effect; all 10 paired task-level differences positive | General computational/statistical separation |
+| Theory | Frozen source audit, counterexamples, and a complete repaired proof of Theorems 5.2-5.3 | Independent reconstruction of every formal proof |
+| Outcome | Claims 1-3 falsified as written; Claim 5 verified; large but unstable finite neural effects | General computational/statistical separation |
 | HF evidence | Job, existing artifact Bucket, checksum-verified v3 bundle | -- |
 
 

scripts/namq_rdspac_coupling_audit.py

@@ -0,0 +1,162 @@
+#!/usr/bin/env python3
+"""Stress-test the multiplicity-weighted NA-MQ -> R-DS-PAC coupling.
+
+The proof is distributional; this executable checks its finite coupon event for
+adversarial multiplicity patterns and checks the stochastic-label transcript law.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+from pathlib import Path
+
+import numpy as np
+
+
+def required_draws(m: int, delta: float) -> int:
+    return math.ceil(8.0 * m * math.log(2.0 * m / delta))
+
+
+def multiplicities(rng: np.random.Generator, trial: int) -> np.ndarray:
+    m = int(rng.integers(1, 257))
+    mode = trial % 4
+    if mode == 0:  # all queries distinct
+        return np.ones(m, dtype=np.int64)
+    k = int(rng.integers(1, m + 1))
+    counts = np.ones(k, dtype=np.int64)
+    if k == m:
+        return counts
+    remaining = m - k
+    if mode == 1:  # one maximally repeated query
+        counts[0] += remaining
+    elif mode == 2:  # highly skewed Zipf allocation
+        weights = 1.0 / np.arange(1, k + 1, dtype=float) ** 1.4
+        counts += rng.multinomial(remaining, weights / weights.sum())
+    else:  # random overdispersed allocation
+        weights = rng.dirichlet(np.full(k, 0.25))
+        counts += rng.multinomial(remaining, weights)
+    return counts
+
+
+def coupon_stress(rng: np.random.Generator, trials: int, delta: float) -> dict:
+    failures = 0
+    worst_margin = None
+    largest_union_bound = 0.0
+    modes = {"all_distinct": 0, "one_heavy": 0, "zipf": 0, "dirichlet": 0}
+
+    for trial in range(trials):
+        counts = multiplicities(rng, trial)
+        m = int(counts.sum())
+        draws = required_draws(m, delta)
+        probabilities = counts / m
+        observed = rng.multinomial(draws, probabilities)
+        margin = int(np.min(observed - counts))
+        worst_margin = margin if worst_margin is None else min(worst_margin, margin)
+        failures += int(np.any(observed < counts))
+        modes[list(modes)[trial % 4]] += 1
+
+        # Chernoff: P[X_u<c_u] <= exp(-c_u(a-1)^2/(2a)),
+        # where E[X_u] >= a*c_u and a = 8 log(2m/delta).
+        a = 8.0 * math.log(2.0 * m / delta)
+        union_bound = float(np.exp(-counts * (a - 1.0) ** 2 / (2.0 * a)).sum())
+        largest_union_bound = max(largest_union_bound, union_bound)
+
+    return {
+        "trials": trials,
+        "delta_per_trial": delta,
+        "multiplicity_modes": modes,
+        "coupon_failures": failures,
+        "empirical_failure_rate": failures / trials,
+        "worst_minimum_surplus": worst_margin,
+        "largest_chernoff_union_bound": largest_union_bound,
+    }
+
+
+def stochastic_transcript_stress(rng: np.random.Generator, trials: int, delta: float) -> dict:
+    # Ordered list contains duplicates.  Each occurrence must receive a fresh
+    # stochastic label, not a copied label from the first occurrence.
+    query_order = np.array([0, 1, 0, 2, 1, 0], dtype=np.int64)
+    counts = np.bincount(query_order)
+    label_probabilities = np.array([0.2, 0.6, 0.85])
+    m = len(query_order)
+    draws = required_draws(m, delta)
+
+    direct = np.empty((trials, m), dtype=float)
+    reconstructed = np.empty((trials, m), dtype=float)
+    coverage_failures = 0
+    for t in range(trials):
+        direct[t] = rng.binomial(1, label_probabilities[query_order])
+        observed = rng.multinomial(draws, counts / m)
+        if np.any(observed < counts):
+            coverage_failures += 1
+            reconstructed[t] = np.nan
+            continue
+        pools = [rng.binomial(1, label_probabilities[u], size=int(counts[u])) for u in range(3)]
+        offsets = np.zeros(3, dtype=int)
+        for j, u in enumerate(query_order):
+            reconstructed[t, j] = pools[u][offsets[u]]
+            offsets[u] += 1
+
+    valid = ~np.isnan(reconstructed).any(axis=1)
+    direct = direct[valid]
+    reconstructed = reconstructed[valid]
+    direct_cov = np.cov(direct, rowvar=False, bias=True)
+    reconstructed_cov = np.cov(reconstructed, rowvar=False, bias=True)
+    mean_gap = float(np.max(np.abs(direct.mean(axis=0) - reconstructed.mean(axis=0))))
+    covariance_gap = float(np.max(np.abs(direct_cov - reconstructed_cov)))
+    target_gap = float(np.max(np.abs(reconstructed.mean(axis=0) - label_probabilities[query_order])))
+
+    return {
+        "trials": trials,
+        "query_order": query_order.tolist(),
+        "multiplicities": counts.tolist(),
+        "label_probabilities": label_probabilities.tolist(),
+        "training_draws": draws,
+        "coverage_failures": coverage_failures,
+        "valid_transcripts": int(valid.sum()),
+        "max_direct_vs_reconstructed_mean_gap": mean_gap,
+        "max_direct_vs_reconstructed_covariance_gap": covariance_gap,
+        "max_reconstructed_vs_target_mean_gap": target_gap,
+    }
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--seed", type=int, default=20260722)
+    parser.add_argument("--coupon-trials", type=int, default=10_000)
+    parser.add_argument("--transcript-trials", type=int, default=50_000)
+    parser.add_argument("--delta", type=float, default=0.05)
+    parser.add_argument("--output", type=Path, default=Path("outputs/namq_rdspac_coupling_audit.json"))
+    args = parser.parse_args()
+
+    rng = np.random.default_rng(args.seed)
+    coupon = coupon_stress(rng, args.coupon_trials, args.delta)
+    transcript = stochastic_transcript_stress(rng, args.transcript_trials, args.delta)
+    checks = {
+        "all_coupon_stress_trials_covered": coupon["coupon_failures"] == 0,
+        "analytic_union_bound_below_delta": coupon["largest_chernoff_union_bound"] <= args.delta,
+        "stochastic_transcripts_covered": transcript["coverage_failures"] == 0,
+        "transcript_means_match": transcript["max_direct_vs_reconstructed_mean_gap"] < 0.02,
+        "transcript_covariances_match": transcript["max_direct_vs_reconstructed_covariance_gap"] < 0.02,
+        "reconstructed_means_match_target": transcript["max_reconstructed_vs_target_mean_gap"] < 0.02,
+    }
+    result = {
+        "audit": "multiplicity-weighted NA-MQ to R-DS-PAC coupling",
+        "seed": args.seed,
+        "formula": "N=ceil(8*m*log(2*m/delta)); D'_Q(u)=c_u/m",
+        "coupon_stress": coupon,
+        "stochastic_label_transcript_stress": transcript,
+        "checks": checks,
+        "all_checks_pass": all(checks.values()),
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(result, indent=2) + "\n")
+    print(json.dumps(result, indent=2))
+    if not result["all_checks_pass"]:
+        raise SystemExit(1)
+
+
+if __name__ == "__main__":
+    main()