"""Build static claim pages and provenance from deterministic result artifacts.""" from __future__ import annotations import hashlib import json import subprocess from pathlib import Path ROOT = Path(__file__).resolve().parent OUT = ROOT / "outputs" PAGES = ROOT / "pages" AUTHORED_ARCHIVE = ROOT / "source" / "2603.08859v1.tar.gz" AUTHORED = ROOT / "source" / "authored" CODE = ROOT / "source" / "official-code" CLAIM_SLUGS = [ "claim-1-theorem-3-3-literal-bound", "claim-2-theorem-3-7-window-bound", "claim-3-theorem-4-3-selective-copy", "claim-4-theorem-4-6-associative-recall", "claim-5-figure-4-selective-copy-learning", "claim-6-figures-5-6-associative-recall-learning", ] def sha(path: Path) -> str: return hashlib.sha256(path.read_bytes()).hexdigest() def load(name: str) -> dict: return json.loads((OUT / name).read_text()) def code_info() -> dict: return { "remote": "https://github.com/SprocketLab/hybrid-expressivity", "checkout": subprocess.check_output( ["git", "-C", str(CODE), "rev-parse", "HEAD"], text=True ).strip(), "status_porcelain": subprocess.check_output( ["git", "-C", str(CODE), "status", "--porcelain"], text=True ).splitlines(), } def source_provenance() -> dict: files = {} for path in sorted(AUTHORED.rglob("*")): if path.is_file(): files[str(path.relative_to(ROOT))] = sha(path) return { "arxiv_id": "2603.08859v1", "arxiv_eprint_url": "https://export.arxiv.org/e-print/2603.08859v1", "archive": str(AUTHORED_ARCHIVE.relative_to(ROOT)), "archive_sha256": sha(AUTHORED_ARCHIVE), "archive_sha256_expected_from_retrieval": "e8d22bfd259aaa60385841d8643109ecb66f7eb1081dd76429f5215f05a032e8", "authored_tree_file_sha256": files, "paper_linked_code": code_info(), "code_version_selection": ( "The arXiv source links to the repository without a commit hash. " "The detached 6be8f8f checkout is the latest repository commit dated " "before the 2026-03-09 arXiv v1 submission; it is not asserted to be " "an author-pinned archival snapshot." ), } def claim_page(title: str, status: str, body: str, evidence: list[str], limitations: str) -> str: bullets = "\n".join(f"- {item}" for item in evidence) return ( f"# {title}\n\n" f"**Assessment: {status}.**\n\n" f"{body}\n\n" "## Evidence\n\n" f"{bullets}\n\n" "## Scope and limitations\n\n" f"{limitations}\n" ) def write_pages() -> dict: claims = json.loads((ROOT / "CLAIMS.json").read_text())["claims"] c1, c2, c3, c4, c5, c6 = [load(f"claim{i}.json") for i in range(1, 7)] n3, n4 = load("claim3_native.json"), load("claim4_native.json") PAGES.mkdir(exist_ok=True) pages = [] bodies = [ claim_page( claims[0]["text"], "falsified as a literal positive linear lower-bound claim", "Assumption 3.2 makes G: V^m → Y^q injective, so cardinality gives " "m·log₂|V| ≤ q·log₂|Y|. The printed right-hand side " "m·log|V| − q·log|Y| is therefore never positive under its own " "assumption. The appendix instead derives a different Fano bound under " "error < 1/8; that is not the printed probability-1/2 theorem.", [ "Exact authored statement: source/authored/sections/func_comp_and_construct.tex:26–28.", f"1,296 admissible cardinality configurations: maximum printed RHS = {c1['printed_bound_audit']['max_literal_bound_over_admissible_grid']:.1f}.", "Exhaustive binary selective-copy partitions: one state attains exactly 1/2 success for m=2 and m=3, matching the theorem's printed threshold.", "The appendix source records its stronger, different prerequisite as err < 1/8 (source/authored/appendix/missing_proof_lb.tex).", ], "This is a literal-statement audit, not a claim that no corrected lower " "bound can be proved. The exact selective-copy enumeration is finite; the " "cardinality implication itself is general.", ), claim_page( claims[1]["text"], "supported by explicit paper-task witnesses and exact receptive-field checks", "For the selective-copy witnesses, changing only a token outside the " "causal receptive field leaves the terminal logits bit-identical. On the " "two-point witness distribution this forces accuracy 1/2, below 2/3. " "A full-window control separates the same pair.", [ f"{c2['receptive_field_sweep']['configs_tested']} real float64 causal-attention stacks; outside-RF violations = {c2['receptive_field_sweep']['violations_outside_receptive_field']}.", f"Maximum terminal-logit change outside RF = {c2['max_abs_delta_when_sumW_below_R']:.1f} for the hard witnesses.", f"Full-window control separates {c2['control_full_window_separates_frac']:.0%} of tested witness configurations.", "The source proof explicitly identifies the terminal dependency as the last sum_i W_i tokens (appendix/missing_proof_lb.tex).", ], "Finite numerical checks cannot prove the universal theorem. They directly " "exercise its stated quantities (window, R, indistinguishability and " "success threshold) on the paper's selective-copy task rather than a proxy.", ), claim_page( claims[2]["text"], "supported for the literal construction; official notebook does not establish its universal scope", "A direct transcription of Appendix D.2/D.5 achieves exact final-position " "selective copying across exhaustive and large deterministic sweeps, with a " "window-minus-one and no-SSM destructive controls. The separately executed " "author notebook is included unchanged as provenance and scores only .825 on " "the notebook generator's 1,024 final-position examples, so it is not used " "as evidence of the theorem's 'every input' quantifier.", [ f"Independent construction: {c3['total_inputs_tested']:,} valid inputs; minimum accuracy = {c3['min_accuracy_over_all_configs']:.6f}.", "Includes L=100, |V|=32 and longer L=1,024/4,096 sweeps; these are labelled construction checks, not learned-model runs.", f"Native notebook baseline/control: {n3['baseline']['last_position_accuracy']:.6f} / {n3['destructive_control']['last_position_accuracy']:.6f} on {n3['baseline']['eligible_last_position_examples']} eligible examples.", "The destructive construction controls remove one attention position, zero the query, or disable selective state update.", ], "The independent checker is a faithful finite implementation of the written " "construction, not a mechanized proof. The native notebook's aggregate failure " "is retained rather than overwritten or relabelled as a success.", ), claim_page( claims[3]["text"], "supported by a full-vocabulary construction and exact coverage certificate; native notebook is only a limited check", "The Appendix D.4 construction was implemented with the stated full-vocabulary " "binary code, last-match positional bias, and a finite softmax separation. " "Its exact iid coverage formula is at least .99 for each measured scale; the " "5×20,000-instance point estimates track that certificate (one mean, .989820, " "is below .99 and is not relabelled as an empirical pass). The implementation " "also exhausts three small full-window domains.", [ f"All analytic coverage certificates meet 99%: {c4['all_meet_99pct']}.", f"Minimum five-seed empirical mean = {c4['min_success_at_theorem_window']:.6f}; maximum gap from exact coverage = {c4['max_gap_vs_analytic']:.6f}.", "Exhaustive full-window domains (|M|, L)=(4,8),(4,9),(8,8) all score 1.0.", f"Native authored notebook baseline/control: {n4['baseline']['last_position_accuracy']:.6f} / {n4['destructive_control']['last_position_accuracy']:.6f}; it has no windowed 99%-coverage test.", ], "The native notebook is a direct one-construction execution, not the paper's " "probabilistic window experiment. The 99% support comes from the written " "construction plus an exact iid coverage calculation and finite tests; it is " "not a claimed learned-model training rerun.", ), claim_page( claims[4]["text"], "authored-source supported, but not independently reproduced", "The exact authored Figure 4 table gives SSM→TF .999 at approximately 2,000 " "parameters and pure TF/SSM .923/.931 at approximately 12,000 parameters, a " "sixfold nominal parameter ratio. The caption calls .999 'perfect', so the " "strict word perfect and printed value are internally inconsistent at the " "shown precision.", [ f"Pinned source table: hybrid@2k={c5['literal_table_comparison']['hybrid_ssm_to_tf_at_approximately_2000']:.3f}; pure TF/SSM@12k={c5['literal_table_comparison']['pure_tf_at_approximately_12000']:.3f}/{c5['literal_table_comparison']['pure_ssm_at_approximately_12000']:.3f}.", "The TeX table, caption, and source-file SHA are in outputs/claim5.json.", "The official micro_hf training path contains hard-coded CUDA transfers and this machine has no CUDA device; no local run is presented as a reproduction.", ], "The repository did not contain per-run scalar results, checkpoints, or an " "author-pinned code commit in the arXiv archive. This page reports the paper's " "own exact table, not an independent empirical confirmation.", ), claim_page( claims[5]["text"], "not independently established; the authored MKAR table conflicts with the literal numeric conjunction", "The Figure 6 table's sixfold row is approximately 2,000 versus 12,000 " "parameters, where SSM→TF is .512—not 60%. At the first shown hybrid result " "above 60% (.990 at approximately 6,000), the nearest shown pure-TF row is " "approximately 12,000, only 2× larger. The caption asserts 60%-at-6×, but the " "raw points needed to locate a different crossing were not released.", [ f"Pinned MKAR values: hybrid@2k={c6['literal_table_checks']['hybrid_ssm_to_tf_at_approximately_2000']:.3f}, hybrid@6k={c6['literal_table_checks']['hybrid_ssm_to_tf_at_approximately_6000']:.3f}, pure TF@12k={c6['literal_table_checks']['pure_tf_at_approximately_12000']:.3f}.", "The <=40% sentence is from Figure 5's associative recall with decoding, a different task from Figure 6's MKAR; it is not treated as an MKAR plateau measurement.", "The exact TeX/table SHA and CUDA-only non-rerun route are recorded in outputs/claim6.json.", ], "This is an authored-source/data audit. It does not infer a curve between " "unreleased points or substitute the Figure 5 task for MKAR. A GPU rerun would " "require compatible CUDA hardware and a declared protocol; neither is claimed here.", ), ] for slug, page in zip(CLAIM_SLUGS, bodies): dest = PAGES / slug dest.mkdir(exist_ok=True) (dest / "page.md").write_text(page) pages.append(str((dest / "page.md").relative_to(ROOT))) rows = [ ("1", "Theorem 3.3 literal lower bound", "falsified", "printed RHS non-positive under injectivity"), ("2", "Theorem 3.7 window lower bound", "supported", "paper-task witnesses and real attention stacks"), ("3", "Theorem 4.3 selective copy", "supported", "independent construction; native notebook limitation retained"), ("4", "Theorem 4.6 associative recall", "supported", "full-vocabulary construction and exact coverage"), ("5", "Figure 4 learned selective copy", "source-supported", "reported table only; .999/perfect rounding conflict"), ("6", "Figures 5–6 learned recall", "not established", "MKAR table conflicts with 60%-at-6x conjunction"), ] table = "\n".join(f"| {a} | {b} | {c} | {d} |" for a, b, c, d in rows) executive = ( "# Executive summary\n\n" "This is a six-claim audit of arXiv:2603.08859v1 / OpenReview `82EJxJzG6r`. " "Claim text is frozen in `CLAIMS.json`; evidence does not rewrite scope.\n\n" "| claim | subject | assessment | headline |\n| --- | --- | --- | --- |\n" + table + "\n\nThe package distinguishes (1) native author-notebook execution, " "(2) an independent implementation of the written constructions, and (3) " "exact authored-table evidence. GPU-only learned-model training was not rerun " "on this non-CUDA host.\n" ) (PAGES / "executive-summary").mkdir(exist_ok=True) (PAGES / "executive-summary" / "page.md").write_text(executive) index = ( "# Reproduction audit: Expressivity-Efficiency Tradeoffs for Hybrid Sequence Models\n\n" "Paper: arXiv:2603.08859v1 · OpenReview `82EJxJzG6r`\n\n" "Six registered claims are preserved verbatim in `CLAIMS.json`. Run `python3 run_all.py` " "for paired deterministic replays, page generation, validation, and a recursive manifest.\n" ) (PAGES / "index.md").write_text(index) return {"claim_pages": pages} def write_logbook(route_info: dict) -> None: claims = json.loads((ROOT / "CLAIMS.json").read_text())["claims"] children = [ {"slug": "executive-summary", "title": "Executive summary", "file": "pages/executive-summary/page.md", "children": []} ] for claim, slug in zip(claims, CLAIM_SLUGS): children.append( { "slug": slug, "title": f"Claim {claim['index']}: {claim['text']}", "file": f"pages/{slug}/page.md", "children": [], } ) logbook = { "schema_version": 1, "title": "Reproduction audit: Hybrid Sequence Models", "emoji": "🔬", "proposed_target_slug": "repro-hybrid-seq-82ejxjzg6r", "publication_status": "local-only; no Space created or modified", "paper": {"arxiv_id": "2603.08859v1", "openreview_id": "82EJxJzG6r"}, "updated_at": "2026-07-28T00:00:00+00:00", "root": {"slug": "index", "title": "Reproduction audit", "file": "pages/index.md", "children": children}, "routes_built": route_info, } (ROOT / "logbook.json").write_text(json.dumps(logbook, indent=2) + "\n") def main() -> None: route_info = write_pages() write_logbook(route_info) (ROOT / "SOURCE_PROVENANCE.json").write_text(json.dumps(source_provenance(), indent=2) + "\n") print("built six claim pages, logbook routes, and source provenance") if __name__ == "__main__": main()