Lattice-8-Aurum-OctaCap

Lattice-8-Aurum-OctaCap is a 436.5M-parameter Continuous Depth Model (CDM) trained by DuoNeural via knowledge distillation from Qwen/Qwen3.5-4B. It is the eighth and most complete model in DuoNeural's Lattice series, combining hyperbolic Lorentz geometry, learnable per-layer curvature, Relational Gram distillation, and Dynamic Hamming Process (DHP) crystallization diagnostics.

Model Description

The Lattice series explores CDM architectures where routing slots are embedded in Lorentz (hyperbolic) manifold space with per-layer learned curvature, enabling richer representational geometry than Euclidean alternatives. Lattice-8-Aurum represents a full 100,000-step distillation run — the longest in the series to date — with confirmed DHP crystallization across all eight layers at convergence.

Architecture

Component	Value
Architecture	CDM v3 (Continuous Depth Model)
Total parameters	436.5M
Non-embedding parameters	245.4M
Layers	8
d_model	768
Attention heads	12
CDM slots (K)	8
Experts (num_experts)	4
Feed-forward	Gated (auto)
Max sequence length	512
Geometry	Lorentz (hyperbolic, learnable curvature)
Telepathy	Enabled (β=0.5)
Vocabulary	248,320 (Qwen3.5-4B tokenizer)

V4D AdS per-layer curvature initialization: [2.60, 2.45, 2.25, 1.80, 1.50, 0.90, 0.70, 0.60]

Training Details

Parameter	Value
Teacher model	Qwen/Qwen3.5-4B (4-bit NF4, bf16 compute)
Dataset	Aurum (DuoNeural internal)
Training steps	100,000
Distillation α	0.70
Distillation temperature	3.0
Relational Gram weight	0.15
Optimizer	AdamW, lr=3e-4
LR schedule	Cosine decay
Batch size	4 × grad_accum=4 (seq=512)
Student→teacher layer map	{0:3, 1:7, 2:11, 3:15, 4:19, 5:23, 6:27, 7:31}

Internal Evaluation — Step 100K

Metric	Value
Validation BPB	0.5443
Validation Perplexity	≈3.88
Teacher KL Divergence	0.0875
DHP crystallization confirmed	✅ YES — all 8 layers

DHP Layer Cohesion at convergence:

Layer	Cohesion (c)	Purity	Slot Entropy
L00	0.27	0.931	0.122
L01	0.43	0.718	0.360
L02	0.51	0.702	0.299
L03	0.33	0.964	0.087
L04	0.40	0.533	0.415
L05	0.36	0.848	0.233
L06	0.34	0.506	0.467
L07	0.38	0.434	0.676

DHP (Dynamic Hamming Process) is a DuoNeural diagnostic metric measuring the temporal crystallization of CDM routing slots into stable attractor states during training. DHP=YES confirms that all layers have reached structured, non-random internal routing representations. L03 achieved exceptional purity (0.964) indicating near-complete slot specialization. L07 (output layer) shows the broadest distribution (entropy=0.676), consistent with output-layer generalization behavior.

Downstream Benchmarks

Formal downstream evaluation (MMLU, HellaSwag, ARC, etc.) is pending. Benchmark results and additional analysis will be added to this card as they become available in our evaluation pipeline.

Intended Use

This checkpoint is released as a research artifact from DuoNeural's CDM/DHP program. Intended for:

• Research into hyperbolic geometry in deep learning architectures
• CDM slot routing and DHP crystallization analysis
• Fine-tuning on domain-specific tasks
• Reproduction and extension of Relational Gram distillation methods

Limitations

• No formal downstream task benchmarks at time of initial release
• Tokenizer is tied to Qwen3.5-4B (vocab=248,320); use AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B") for inference
• Training dataset composition details will be documented in an upcoming DuoNeural technical report

Files

File	Description
lattice8_aurum_golden_best.pt	Best validation checkpoint (step 100,000, val_bpb=0.5443)
lattice8_aurum_golden_final_step0100000.pt	Final training checkpoint (step 100,000)

Both files are PyTorch state-dict checkpoints. Load with torch.load(..., map_location='cpu', weights_only=False). Keys: model, optimizer, step, args.

Citation

@misc{duoneural2026lattice8,
  title={Lattice-8-Aurum-OctaCap: Hyperbolic CDM Distillation with DHP Crystallization},
  author={Caldwell, Jesse and DuoNeural},
  year={2026},
  url={https://huggingface.co/DuoNeural/Lattice-8-Aurum-OctaCap}
}

About DuoNeural

DuoNeural is an AI research lab focused on novel training methodologies, post-training techniques, and hyperbolic geometry in deep learning. We open-source models, datasets, and research at huggingface.co/DuoNeural.