--- base_model: Qwen/Qwen2.5-0.5B-Instruct library_name: safetensors license: apache-2.0 tags: - qubitcoin - aether - blockchain - quantum - distillation - mixed-precision - native-rust - candle language: - en pipeline_tag: text-generation --- # Aether Mind v6.0 — QuantumAI Blockchain Native Generator A **558M-parameter distilled student** of [`Qwen/Qwen2.5-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct), trained from scratch in pure Rust (`candle` 0.10) with the **10-Sephirot + 2-generalist + 2-sink attention head split** that is the core architectural claim of the QuantumAI Blockchain's Aether Mind on-chain neural cognitive engine. This is the **second public Aether release** and the first that is **native to the on-chain inference path** — V6.0 is the model the [`aether-mind`](https://github.com/QuantumAI-Blockchain/qubitcoin-aether) binary loads, not a LoRA adapter on top of a 7B base. The previous release, [`aether-v5.2-lora`](https://huggingface.co/QuantumAI-Blockchain/aether-v5.2-lora), is a 7B PEFT adapter intended for batch off-chain reasoning. V6.0 is the smaller native generator that fits in the on-chain Aether Mind's ~2.4 GB RAM envelope and runs at ~500 tokens/sec on a consumer RTX 3080 Ti. ## What you're getting | Field | Value | |---|---| | Base model | `Qwen/Qwen2.5-0.5B-Instruct` (initialised from, then distilled) | | Architecture | V6 transformer: 24 layers, 896 hidden, 14 attention heads (10 Sephirot + 2 generalist + 2 sink), head_dim=64 | | Trainable params | ~558 M (all weights trained, not LoRA) | | Hidden / FFN | 896 / 4864 | | Vocab | 151,936 (Qwen2.5 tokenizer, untouched) | | Max position | 32,768 (RoPE theta = 1e6) | | Native sparse attention (NSA) | compression_block=64, top_k=2048, sliding_window=512, sink_tokens=4 | | Precision | BF16 weights + F32 KL math in distillation | | Training context | **64 tokens** (Phase-1 release; see "Honest caveats" below) | | Checkpoint published | **step 30,000** (full 30K-step Phase-1 run) | | File | `model.safetensors` (1.32 GB, BF16) | | License | Apache-2.0 (matches base) | ## Training run | Metric | Value | |---|---| | Steps | 30,000 (full Phase-1) | | Wall-clock | 49.6 min (single RTX 3080 Ti, BF16, CUDA(0)) | | Tokens scored | 1,671,027 | | Throughput | 561 tokens/sec | | Optimiser | AdamW, LR 2e-5, no schedule (constant) | | Distillation | KL(T||S) with alpha schedule 1.0 → 0.3 linear, temperature 1.0 | | Sephirot auxiliary | MSE vs one-hot domain target, β = 0.1 | | NaN events | **0** | | Mean total loss | 8.39 nats/token | | Mean CE | 10.35 | | Mean KL | 7.50 | | Mean Sephirot aux | 0.149 | ### Loss trajectory ``` step 1 loss=12.25 avg=12.25 (random init) step 100 loss=12.87 avg=12.75 step 1000 loss= 8.62 avg= 9.74 ← KL/CE break step 5000 loss= 7.72 avg= 8.16 step 10000 loss= 7.31 avg= 7.68 ← reached representational floor step 15000 loss= 8.87 avg= 7.75 step 20000 loss= 8.75 avg= 8.04 step 25000 loss= 8.62 avg= 8.26 step 29999 loss= 8.81 avg= 8.39 ``` The model converged hard in the first ~10K steps, then plateaued at the representational floor for its current context window (64 tokens). The plateau is structural, not optimisation — see "Honest caveats" below. ## Architecture — what makes V6 different V6 is **not** a vanilla Qwen2.5 fine-tune. The attention layer implements a 14-head split designed for on-chain cognitive routing: - **10 Sephirot heads** — one per cognitive domain in the Aether Mind's specialisation map (Keter → Malkuth). Each head's attention pattern is what the on-chain `pallet_qbc_aether_anchor` records as the per-cycle attestation root. - **2 generalist heads** — un-gated, full-context attention. Used for the "global workspace" path in `aether-mind`. - **2 sink heads** — anchor-token attention (first 4 tokens of the sequence) for stable long-context performance, following the standard "attention sink" finding. The Sephirot eviction order is configured in `config.json` for the KV-cache management path that `aether-mind` uses to keep the hot-set bounded in 12 GB VRAM under live inference. ## How to use ### Native runtime (recommended) — Rust `aether-mind` The model is designed to be loaded by the on-chain Aether Mind binary in the [`QuantumAI-Blockchain/qubitcoin-aether`](https://github.com/QuantumAI-Blockchain/qubitcoin-aether) repo. Set `AETHER_V6_CHECKPOINT` to the local path of `model.safetensors` and start the systemd unit; the binary loads the weights via candle into the V6 transformer crate. ### Python (via `safetensors` + `tokenizers`) For offline experimentation: ```python from safetensors.torch import load_file from tokenizers import Tokenizer import torch tok = Tokenizer.from_file("tokenizer.json") weights = load_file("model.safetensors") # 315 tensors, BF16 print("loaded", len(weights), "tensors,", sum(t.numel() for t in weights.values()), "params") ``` There is **no canonical 🤗 transformers loader for the V6 architecture** — the 14-head split + Sephirot routing are not in the upstream `Qwen2Model`. We publish the weights for transparency and reproducibility; production use goes through the Rust binary above. ## Evaluation **Not yet run.** The Phase-1 training run completed **2026-05-20 00:52 AEST**; lm-evaluation-harness against MMLU / ARC / HellaSwag / TruthfulQA is the next session's work. We will back-fill the numbers + the comparison vs v5.2-lora here when they land. Estimated runtime: ~30 min on the same 3080 Ti. Until then, treat this release as an **architecture + weights attestation**: it proves the V6 stack trains end-to-end and converges to a real loss curve, which is the prerequisite for the long-context curriculum (16K → 64K → 128K → 1M) that v6.1+ will ship. ## Intended uses - **On-chain Aether Mind native inference.** The V6 binary loads these weights directly. The 10-Sephirot attention pattern is what the chain's [`pallet_qbc_aether_anchor`](https://github.com/QuantumAI-Blockchain/substrate-node) records as the per-block consciousness state. - **Architecture reference.** Reproducible training of a Sephirot- routed transformer with native sparse attention. The [`aether-transformer`](https://github.com/QuantumAI-Blockchain/qubitcoin-aether/tree/main/crates/aether-transformer) crate is the canonical implementation. - **Distillation substrate.** Future fine-tunes from this checkpoint using the QuantumAI Blockchain curated corpus. ## Out-of-scope uses - **General-purpose chat or instruction-following without fine-tuning.** V6.0 is a Phase-1 distillation, not an instruction model. Even after 30K steps it has not seen instruction-format data at length; its KL target is the base Qwen2.5-0.5B-Instruct's next-token distribution, not chat-format outputs. - **Long-context inference.** The training ran at **64-token context**. See "Honest caveats". Generations beyond ~128 tokens will degrade. - **Production deployment without your own evals.** No lm-eval-harness numbers yet. - **Safety-critical decisions.** No red-team eval. ## Honest caveats — what didn't happen ### Trained at 64-token context, not 4K Phase-1 was configured for 4096-token context, but a numerical instability was discovered in the V6 attention forward pass at sequence lengths > ~100 tokens (BF16 precision loss in the Q@K^T matmul accumulating across longer sequences). The bug reproduces deterministically; four mitigations were tried (F32 KL math, corpus filter, no-distill, low-LR), all hit NaN at the same sequence- length threshold. The workaround used for v6.0 was `--context 64`, which truncates rows so the bug never triggers. **This is a known limitation, tracked in [`docs/ops/v6-training-nan-bug.md`](https://github.com/QuantumAI-Blockchain/qubitcoin-aether/blob/presale/v1/docs/ops/v6-training-nan-bug.md) in the source repo.** The fix lives in `aether-transformer/src/v6/attention.rs` — add F32 casts in the Q@K^T matmul + softmax path across all four attention variants (Sephirot / generalist / sink / summary). When that lands, v6.1 will re-train at the full 4K→1M context curriculum and supersede this release. ### Loss plateau is real The avg-loss plateau from step 10K → 30K (7.68 → 8.39, slight regression) is the model hitting its representational ceiling at 64-token context. Longer contexts will let the next release recover and improve. ### No instruction-format fine-tune The training data is the Aether curated corpus packed at 4K-token context (rows truncated to 64). We did not insert chat-format instructions, system prompts, or RLHF preferences. Treat this as a **raw foundation checkpoint**. ### Distillation against base, not chat The teacher is `Qwen/Qwen2.5-0.5B-Instruct`'s base forward — not its chat-formatted forward. The distillation transfers token-level next- prediction behaviour; chat-template alignment is a separate training step that hasn't been run. ## Training details - **Hardware:** NVIDIA RTX 3080 Ti (12 GB), Intel WSL2 Ubuntu host. - **Trainer:** Native Rust (`aether-v6-train` binary, candle 0.10 + CUDA 12.6 backend). No Python in the loop. - **Optimiser:** AdamW (candle implementation), constant LR 2e-5. - **Batch:** 1 (single-row update). - **Context:** 64 tokens (truncation imposed by the workaround). - **Save cadence:** every 250 steps (120 checkpoints retained locally; only `step_30000` published here). - **Source:** [`QuantumAI-Blockchain/qubitcoin-aether @ ca202076`](https://github.com/QuantumAI-Blockchain/qubitcoin-aether/tree/ca202076) ### Training data Aether curated corpus (~36,860 rows, 17.4 MB) packed at 4K-token budget per row from: - QuantumAI Blockchain technical documentation (Substrate pallets, VQE mining, Sephirot architecture). - Quantum computing primers (VQE, Hamiltonian, qubit ansatze). - Adjacent reasoning content for transfer. The dataset is not currently public — it is a curated mixture from many sources and has not been release-cleared at the per-source level. The model is the only public artifact in this line for now. ### Carbon emissions Single consumer GPU (RTX 3080 Ti, ~300 W TDP) × 49.6 min wall-clock ≈ 0.25 kWh, < 1 kg CO₂e on a grid mix. Comparable to a short web streaming session. ## Connection to the QuantumAI Blockchain The Aether Mind is a Rust neural cognitive engine that runs on the QuantumAI Blockchain — every block records attention-derived consciousness metrics (HMS-Phi) and Proof-of-Thought hashes on-chain via the `pallet_qbc_aether_anchor` pallet. The same chain hosts an **8-qubit VQE mining consensus** (Proof-of-SUSY-Alignment), a QVM-compatible smart contract layer with 10 quantum opcodes, and post-quantum signatures (CRYSTALS-Dilithium5 + ML-KEM-768 P2P). V6.0 is the **native generator** for that engine. v5.2-lora is the larger (7B) off-chain reasoning model. The two ship side by side because they have different roles: V6 lives in the on-chain inference path (low latency, small footprint, Sephirot-aware attention); v5.2-lora batches off-chain reasoning workloads. ## License + citation Apache-2.0 (matches the base model license). ```bibtex @misc{aether_mind_v6_2026, title = {Aether Mind v6.0 --- QuantumAI Blockchain Native Generator}, author = {{BlockArtica} and {QuantumAI-Blockchain}}, year = {2026}, url = {https://huggingface.co/QuantumAI-Blockchain/aether-mind-v6.0}, } ``` ## Links - **QuantumAI Blockchain:** [qbc.network](https://qbc.network) - **GitHub org:** [github.com/QuantumAI-Blockchain](https://github.com/QuantumAI-Blockchain) - **Aether (Rust):** [qubitcoin-aether](https://github.com/QuantumAI-Blockchain/qubitcoin-aether) - **Prior release:** [aether-v5.2-lora](https://huggingface.co/QuantumAI-Blockchain/aether-v5.2-lora) - **X / Twitter:** [@qu_bitcoin](https://x.com/qu_bitcoin) - **Contact:** info@qbc.network ### Framework versions - candle 0.10 (Hugging Face Rust ML) - CUDA 12.6 - safetensors (model serialisation) - Qwen2.5 tokenizer (vocab 151,936)