{ "experiment": "SPIRA v3 cortex representation repair \u2014 controlled A/B vs v2 baseline", "date": "2026-10-22", "protocol": { "corpus": "same rebuilt corpus for both arms (train 13.27 MB / test 640 KB)", "vocab_size": 4096, "snn_tokens": 200000, "epochs": 2, "am_contexts": 50000, "test_positions": 20000, "vocab_coverage_test": 0.4524, "headline_metric": "SNN alone, in-vocab accuracy on held-out test positions", "bigram_reference": "patch bigram trained on train cols [0:600k], evaluated on exactly the same test positions at the same vocab 4096" }, "headline": { "snn_invocab_test_v2_baseline": 0.003978779840848806, "snn_invocab_test_v3": 0.020667550839964632, "ratio_v3_over_v2": "x5.2", "bigram_4096_invocab_test": 0.04940318302387268, "gap_to_bigram": "x12.4 (v2) -> x2.4 (v3); beat-the-bigram criterion NOT met", "snn_invocab_trained_region_v2_baseline": 0.0376034093757834, "snn_invocab_trained_region_v3": 0.1328653797944347, "training_speed_tps_v2_baseline": "~530 (512.6 / 561.8 per epoch)", "training_speed_tps_v3": 188.97756640356553, "checkpoint_mb_v2_baseline": 14.6, "checkpoint_mb_v3": 45.9, "notes": [ "Free generation still incoherent in both arms.", "At this small budget, fusion hurts on held-out: v3 fused in-vocab 0.001326259946949602 < SNN alone 0.020667550839964632." ] }, "bigram_reference_detail": { "bigram_4096_invocab_test20k": 0.04940318302387268, "n_invocab": 9048 }, "baseline_arm_full": { "label": "baseline", "hist": [ { "acc": 0.092515, "acc_fast": 0.112785, "tps": 512.6229927407619 }, { "acc": 0.065475, "acc_fast": 0.101045, "tps": 561.7816027951075 } ], "test": { "n_positions": 20000, "vocab_coverage": 0.4524, "snn_acc": 0.0018, "hier_acc": 0.00025, "am_acc": 0.0001, "fused_acc": 0.00035, "snn_acc_invocab": 0.003978779840848806, "hier_acc_invocab": 0.0005526083112290009, "am_acc_invocab": 0.00022104332449160034, "fused_acc_invocab": 0.0007736516357206013, "corrupt_lanes": 0 }, "trained": { "n_positions": 8000, "vocab_coverage": 0.498625, "snn_acc": 0.01875, "hier_acc": 0.00675, "am_acc": 0.011125, "fused_acc": 0.017375, "snn_acc_invocab": 0.0376034093757834, "hier_acc_invocab": 0.013537227375282025, "am_acc_invocab": 0.01779894710453748, "fused_acc_invocab": 0.030834795688142392, "corrupt_lanes": 0 }, "gen": "The old man looked at the retuiltIN S .\n IN S add\n r ifotIthing beithingtory rethingeek eIGHTND RIGHT it athingthingthing if t:\n #irlother:\n #\n f\n \" andeturotIe)\n a s\n fone)Imone)e the .\n .\n thingotICAPef g if##########IGHTtry: lf._lf._otI iflear a singl" }, "v3_arm_full": { "label": "v3", "hist": [ { "acc": 0.120205, "acc_fast": 0.142835, "tps": 188.97756640356553 }, { "acc": 0.11196, "acc_fast": 0.15781, "tps": 188.5136375504029 } ], "test": { "n_positions": 20000, "vocab_coverage": 0.4524, "snn_acc": 0.00935, "hier_acc": 0.0005, "am_acc": 0.0001, "fused_acc": 0.0006, "snn_acc_invocab": 0.020667550839964632, "hier_acc_invocab": 0.0011052166224580018, "am_acc_invocab": 0.00022104332449160034, "fused_acc_invocab": 0.001326259946949602, "corrupt_lanes": 0 }, "trained": { "n_positions": 8000, "vocab_coverage": 0.498625, "snn_acc": 0.06625, "hier_acc": 0.005375, "am_acc": 0.011125, "fused_acc": 0.015875, "snn_acc_invocab": 0.1328653797944347, "hier_acc_invocab": 0.010779644021057909, "am_acc_invocab": 0.01779894710453748, "fused_acc_invocab": 0.02782652293807972, "corrupt_lanes": 0 }, "gen": "The old man looked at the rrorone\nr hi \n r hi r hiand a a l on ar hi\n on alooloor hiloolectueEloo looynon at see a slectlectlector\non a##########yn a s##########ramelooErlooow, golectlectErlooow,loo.\n .\n lect lectlect rameloo a s" } }