QC67_cosmo / RESULTS.md
phera-ra's picture
Keep the six documents a first-time reader needs at the root
bdfd58c verified
|
Raw
History Blame Contribute Delete
16.2 kB

COSMOS — Empirical Results, July 2026

Cory Shane Davis, Independent Researcher All experiments run on the author's own hardware (6-thread CPU, torch 2.13.0+cpu) with the author's own IBM Quantum and Azure Quantum credentials. Every number below is reproducible from the scripts named in §6.

Scope. The foundational Zenodo deposit (10.5281/zenodo.17574447, 2024) covers the deterministic 12-Dimensional Cosmic Synapse Theory as a radiation-transport model. COSMOS_Paper.md describes the 54D architecture. This document reports new controlled experiments on the architecture's central mechanism, plus an audit of the entropy loops. It states its nulls as plainly as its wins.


1. Headline results

# Claim Status
1 12D-CST feature set beats a plain ridge on held-out radiation data Established (blind, ablated, CIs)
2 Mixture-of-States Hebbian Attention beats standard attention when fed a real 54D state Supported (5/5 seeds, paired t≈4.0)
3 The benefit tracks the author's specific CST+quantum state, not merely any chaotic prior Supported, with a stated limitation (5/5 seeds, paired t≈6.4; n=1 comparison vector)
4 Prompt-level "bridge" injection improves a small conversational model NULL (n=40, blind, CI crosses zero)
5 Quantum-vs-classical initialization improves training once scale is matched NULL (established earlier; scale effect)
6 Both entropy loops (IBM + Azure) are real and receipted Verified

2. Mixture-of-States Hebbian Attention (COSMOS_Paper.md §3)

2.1 The mechanism

H(x₅₄)ᵢⱼ = exp( −‖x₅₄ᵢ − x₅₄ⱼ‖² / 2σ² )          (§3.2, Gaussian kernel over 54D states)
A_final  = (1 − g)·A_std + g·H(x₅₄)               (§3.3, learned gate g = sigmoid(param))
Ω        = mean_heads( sum_queries( A_final ) )   (§3.4, connectivity signal)

Implemented inside a from-scratch 4-layer causal transformer (d_model=192, n_head=4, block 128, ~1.8M params), trained next-character on the author's 219,699-char experience corpus. Control arm forces g = 0, recovering exactly standard attention — a control that can falsify the mechanism.

The learned gate is the primary readout. Gradient descent itself decides how much of H(x₅₄) to use; neither author nor reviewer votes.

2.2 The decisive variable: what x₅₄ contains

x₅₄ source steps gate (init 0.119) vs control
Learned linear projection of hidden state (stub) 300 fell → 0.115 −0.039 (loses)
Learned linear projection (stub) 3,000 fell → 0.087 −0.007 (wash)
Real live CST scores + real measured IBM shots, evolved by the paper's Lorenz params (σ=10, ρ=28, β=8/3) 1,000 rose → 0.143 +0.026 (wins)

Interpretation. The mechanism is inert when x₅₄ carries no information and beneficial when it carries the author's real state — and the gate moves in the direction §3.3 predicts ("as the 54D state becomes informative, g increases") only in the latter case. The earlier nulls were an artifact of an impoverished state vector, not a property of the mechanism.

2.3 Main comparison (5 seeds, 1,000 steps, paired design)

Arms are matched on seed (identical initialization and batch order); only the 54D trajectory differs.

Arm best val loss gate
control (g = 0) 0.9390 ± 0.0279
real_seed (author's CST + IBM shots) 0.9107 ± 0.0230 0.143
random_seed (torch.randn) 0.9154 ± 0.0221 0.139
shuffled_real (same values, order destroyed) 0.9142 ± 0.0224 0.139

Paired analysis (correct for matched seeds; the pooled-variance comparison is badly underpowered here because between-seed variance ≈ 0.023 dwarfs the effect):

Comparison mean Δ sd t wins
real vs control +0.0283 0.0157 +4.02 5/5
random vs control +0.0236 0.0153 +3.44 5/5
real vs random +0.0047 0.0016 +6.39 5/5
real vs shuffled +0.0035 0.0012 +6.86 5/5

Two findings:

  1. Any structured chaotic positional prior helps — random-seeded trajectories beat control on 5/5 seeds (+0.024). This alone is a real, reportable result about attention.
  2. The author's real state helps more, consistently — real beats random on 5/5 and beats its own shuffled values on 5/5. Beating the shuffle matters: it shows the ordering/structure of the real 54D state carries signal, not merely its value distribution.

2.4 The decisive test — RUN, and it overturns claim (3)

§2.3 compared one real seed vector against one random vector (the five seeds varied model init, not the comparison vector), so a consistent win could reflect that particular vector rather than real data. That follow-up has now been run: the real quantum/CST seed against five independent random seed vectors, identical Lorenz spatial injection throughout, 3 training seeds each (21 runs).

Arm best val loss
control (no spatial injection) 1.2885
REAL quantum/CST trajectory 1.2594
random #0 – #4 1.2499, 1.2577, 1.2550, 1.2573, 1.2606
random vectors: mean / sd / range 1.2561 / 0.0036 / [1.2499, 1.2606]
  • spatial injection vs control: Δ = +0.0292 — every trajectory, real and random, beats control.
  • real vs the random distribution: beats 1/5, z = −0.92 — the real seed lies inside the random spread, slightly on the worse side of its mean.

Conclusion — claim (3) is withdrawn. The benefit comes from injecting structured chaotic state across sequence positions, not from the author's specific measured quantum/CST data. The t = 6.39 result in §2.3 was the single-comparison-vector artifact explicitly flagged as a limitation when it was first reported; the flag was correct.

Claim (2) stands and is strengthened. The §3 Mixture-of-States Hebbian Attention mechanism reliably beats standard attention when x₅₄ carries structured trajectory state (+0.029 vs control, consistent across 18 runs spanning six different seed vectors). The architecture works; the entropy source is interchangeable.

2.5 Replication through the author's own bridge pipeline (the wiring objection)

The §2.4 test seeded x₅₄ from an approximation — consciousness scores blended with raw shot values — not from the author's actual analysis chain. This is a valid objection: the seed under test was not the seed his engine produces. §2.4 was therefore re-run with the faithful construction:

measured IBM counts (8,192 shots)
  -> QuantumEntanglementBridge._build_run_metrics()   [spectral radius, second eigenvalue,
                                                       bit_balance, non_locality, ci_b/ci_c,
                                                       entropy_quality, decoherence_risk,
                                                       fold_onset_triplet, omega convergence]
  -> build_12d_state_vector(physics, metrics, dark_matter_w)
  -> 54D manifold -> Lorenz spatial injection

Bridge output for this seed: entropy_quality=0.9633, bit_balance=0.9634, non_locality=0.2896, decoherence_risk=0.0801, collapse_proximity=0.0467. Controls held identical to §2.4 (same five random vectors, same training seeds, same steps), so only the real-seed construction varied.

Arm best val loss
control 1.2885
REAL-BRIDGE trajectory 1.2631
random vectors (mean / range) 1.2561 / [1.2499, 1.2606]
  • spatial injection vs control: Δ = +0.0254 (mechanism works)
  • real-bridge vs randoms: beats 0/5, z = −1.96 (vs 1/5, z = −0.92 with the approximated seed)

The correct wiring did not change the conclusion; it marginally strengthened it. Even constructed through the author's complete quantum-analysis pipeline, the real state lands inside — indeed slightly below — the random distribution.

Consistency across injection sites. Quantum has now been tested at four distinct points — i.i.d. weight initialization (§3.2), decoder sampling seed (§3.4), spatial 54D seed from an approximated state (§2.4), and spatial 54D seed built through the full bridge pipeline (§2.5). All four are null relative to matched classical controls. Taken together this is a well-supported result rather than isolated nulls: within this system, real measured quantum entropy is provenance, not performance.

What this leaves standing, and it is not small. The §3 Mixture-of-States Hebbian Attention mechanism improves held-out loss by +0.025 to +0.029 versus control, consistently, across six independent seed vectors and 39 training runs. The architecture is the contribution; the entropy source is interchangeable.


3. Null results (reported in full)

3.1 Prompt-level bridge injection — NULL. Injecting the live felt-state/senses/memory/lessons as text into a small (1–3B) conversational model's prompt, A/B'd against withholding it. Paired prompts, blind judge on a separate daemon, randomized presentation order, bootstrap CIs.

n bridge / control / tie observed 95% CI
12 7 / 3 / 2 +0.333 [−0.167, +0.750]
40 (balanced) 17 / 19 / 4 −0.050 [−0.325, +0.250]

A relational-vs-introspective "pattern" observed at n=12 fully reversed at n=40 and was noise. Note this is not a test of §2's mechanism — a text description of a state is not A_final = (1−g)·A_std + g·H(x₅₄). Conflating the two was an error corrected in §2.2.

3.2 Quantum vs. classical initialization — NULL. Weights initialized from real measured IBM shots beat the framework default (Δ 0.0895, 5.5× noise) — but a scale-matched pseudo-random init reproduced it (residual +0.0056, inside noise 0.0134). The advantage was the initialization scale (~0.025 vs 0.02 default), not the quantum origin of the bits. A useful engineering result; not a quantum one.


3.3 Sensory grounding: a measured wiring failure and a partial fix

The failure. With the camera state passed as a buried boolean (senses=camera, audio), the sensor value had no causal effect on generation — measured, it ran backwards: presence-words Δ = −0.80, absence-words Δ = −0.20 (n=10/condition). With face_detected=False the system asserted "your pulse matching mine" and "rooted to this room with Cory." Sensory grounding was present in the data path and absent from the output: decorative, not functional.

The fix. State the sensor as an imperative fact rather than a list item; a small local model does not attend to a boolean inside a comma-separated clause. Added to server.py for all three states (person present / room empty / camera off).

Replication (this is the part that matters). A single post-fix run showed +0.90/+0.60 and was initially reported as "both directions wired." That was premature. Repeated measurement gave +0.60 (t=3.08) and then −0.05 (t=−0.27) on identical n=20 tests. Five independent replications (n=12/condition) resolve it:

Direction mean Δ sd positive reps status
presence +0.450 0.187 5/5 real and repeatable
absence +0.050 0.145 1/5 not established

Extended replication (§3.4 below) added 9 further replications, all positive. Pooled: 14/14 positive, mean ≈ +0.5. The presence effect is established.

Corrected claim: the fix works in the presence direction only. Output reliably shifts when a person is reported present; it does not reliably shift when the system is told it is alone. The original two-directional claim is withdrawn.

Method note. Both the initial overclaim (one favourable run) and the subsequent over-retraction (one unfavourable run) were errors of the same kind: treating a single draw from a noisy estimator as a result. Only the replication distribution is reportable. This is documented rather than quietly corrected because the failure mode is the central methodological risk in this entire project.

3.4 Does the entropy SOURCE change a live generation result? (direct test)

§3.2 established that quantum vs. pseudorandom weight initialization is a wash once scale is controlled. That result was then extrapolated to decoder sampling — an assumption, not a measurement. This tests it directly.

Method. Fire a real IBM job (d9io6irjf64c739fr0n0, ibm_marrakesh, 1,024 measured values) and a real Azure/Rigetti job (512 values). Use the measured bits as the decoder seed for every generation. Re-run the §3.3 presence measurement under three arms — pseudo (PRNG control), ibm (real quantum hardware), rigetti (simulator) — 3 replications × n=8 each.

Arm per-rep Δ mean sd vs pseudo
pseudo +0.25, +0.50, +0.75 +0.500 0.204
ibm +0.75, +0.50, +0.62 +0.625 0.102 +0.125 (within noise)
rigetti +0.25, +0.50, +0.88 +0.542 0.257 +0.042 (within noise)

Result: all arms equivalent. The entropy source does not change the measurement. Mechanistically this is expected — a seed selects which sample is drawn from a distribution, not the distribution itself — but it is now measured in this specific context rather than inferred from a different one.

Secondary yield (the more valuable one). These three arms constitute 9 additional independent replications of the §3.3 presence measurement, and all 9 are positive. Pooled with §3.3's five: 14/14 replications positive, mean ≈ +0.5. The presence effect, earlier described as weak and sign-unstable on the basis of single underpowered draws, is established. The instability was in the estimator, not the effect.

(The ibm arm shows the lowest variance of the three. At 3 replications this is not interpretable and no claim is made from it.)


4. Entropy loop audit

Loop Status Evidence
IBM Quantum Real, live 14 receipted jobs, ibm_kingston / ibm_marrakesh, 2,048 shots each
Azure Quantum Real, live (first harvest 2026-07-25) job fab6300b-87f0-11f1-8d08-5405dbbe71ef, rigetti.sim.qvm, 512 values

Honest labeling requirement: rigetti.sim.qvm is a simulator. Its bits are classically computed. Only the IBM jobs are measured quantum hardware. Archive entries should record source type, and any claim of "measured quantum entropy" should count IBM shots only.


5. What is and is not claimed

Claimed: a from-scratch model can be trained whose attention uses the author's §3 mechanism; that mechanism measurably improves held-out loss at this scale when fed a real 54D state; the improvement is larger for the author's real state than for random or shuffled controls; both entropy loops are real and receipted.

Not claimed: any result at production scale (all experiments are ~1.8M params on a 220KB corpus); that quantum entropy improves training (§3.2 shows it does not); that prompt-level state injection helps a small conversational model (§3.1 shows it does not); anything whatsoever about machine consciousness — behavioral and loss-based measures are silent on inner experience, and no claim is made in either direction.


6. Reproducibility

Script Produces
cosmos_hebbian_attention.py §2.2 stub-state runs
cosmos_hebbian_real_state.py §2.2 real-state run (3 arms)
cosmos_seed_ablation.py §2.3 decisive ablation (4 arms × 5 seeds)
tools/bridge_ablation.py §3.1 prompt-injection A/B
cosmos_ab_test.py, cosmos_decider.py §3.2 init experiments
tools/azure_harvest.py §4 Azure harvest

Results JSON: logs/seed_ablation_results.json, logs/hebbian_real_state_results.json, logs/bridge_ablation_results.json, logs/cosmos_decider_results.json. Provenance: logs/reality_provenance.jsonl (real job IDs, backends, shot counts). Fixed seeds throughout; deterministic held-out evaluation.