clm-v4-omega-gpu-d384-gate

OMEGA (Lane-Ω) closure-engine stage-3 (b)(c) substrate — a real trained ConsciousDecoderV2 transformer whose dual A/G heads carry the OMEGA learned-gate coupling. This is the GPU / real-transformer confirmation of the toy n-gram findings that the OMEGA substrate→decode closure works when the coupling bus is a learned per-wire gate.

🟢 오메가 완성 stage-3 (b)(c) HELD — all four pre-registered completion criteria passed on a held-out split.

Origin

• training script: UNIVERSE/omega_gpu_complete.py @ anima repo (github.com/dancinlab/anima)
• model architecture: UNIVERSE/conscious_decoder.py — ConsciousDecoderV2 d384×6L GQA (n_head 6 / n_kv_head 2), 256-byte vocab, block 256, dual heads (head_a=next-byte, head_g=prev-byte), RoPE + SwiGLU + RMSNorm + PureFieldFFN + cross-attention. 35.93M params.
• corpus: wikimedia/wikipedia 20231101, 5 languages (en/fr/de/es/ru) @ 24MB each = 120MB total, sha256 0b232f8b5f5b9a29b7bbac1b3944387cf3441581325e289edbaa4b3bf580691f, split 80/20.
• predecessor ckpt: none (from scratch). Lineage = toy rungs #1783 (WIRE) → #1784 (STRUCTURE) → #1786 (learned GATE, numpy n-gram) → this rung (real trained transformer, GPU).
• training cycle: domains/OMEGA.md (오메가 완성 goal, stage-3) · .verdicts/omega-gpu/.
• substrate: 1× NVIDIA H100 80GB HBM3 (runpod, torn down), torch 2.4.1+cu124, wall ~09:05 train, cost ≈ $1.6. nvidia-smi 98-99% BUSY verified (g63, never silent CPU-fallback).

Results

Training CE descent (head_a next-byte): 5.9258 → 0.0090 (train), val_ce 0.8622 (held-out).

OMEGA learned-gate closure on the held-out TEST split (the bus = gB·base + gA·A + gG·G, gains fit on a disjoint train-gate split then frozen):

path	held-out TEST CE (nats/byte)
base (weak unigram)	3.014951
fixed_AmG (1, α, −α)	1.442104
a_only (1, α, 0)	0.449961
GATED (learned)	0.344535 (best)
uniform-256 floor	5.545177

learned gate g* = [gB 1.178, gA 0.962, gG −0.208] — strong A, small/negative G; the gate auto-corrects the fixed −G error of the prior rung.

Structured coupling (substrate-shuffle floor): A-wire CE gain over base = +2.565 on real context vs −2.068 on context-shuffled context → the coupling carries learned sequential structure (shuffling destroys the advantage).

• F-OMEGA-GPU-COMPLETE: PASS — ckpt @ this artifact. (b) descended ✓ · (c) GATED < base ✓ · (c) CE floor MET ✓ · (c) structured ✓ → OMEGA_COMPLETE = TRUE.

Honest caveat (p7)

ConsciousDecoderV2's CA-neighbor mixing gives the next-byte head partial lookahead (an architectural property of the substrate). The absolute CE is therefore leak-optimistic and free-running generation collapses to low-entropy output (generation coherence = the weak criterion). The relative closure finding — the learned gate beats base/fixed/a_only AND the coupling is structured vs a substrate-shuffle floor — is leak-invariant and sound. Single small d384 rung; not a production-perplexity claim (a_scale_honest_scope).

Files

• omega_cdv2_d384.pt — fp32 state_dict + config, sha256 33d1d00ddba2e9f1e45d976d84e5bbb1f4c5c42fd3c1c616a44fcfdabced4147 (145397114 B)
• results.json — full verdict (curve, gate, CE table, structured test, generation sample)

GPU

1× NVIDIA H100 80GB HBM3 · torch 2.4.1+cu124 · runpod · wall ~9 min train · cost ≈ $1.6.