# Sovereign Mahiru MoE (3x3.6B) Sovereign Mahiru MoE is a sparse Mixture of Experts (MoE) model built by merging three specialized 3.2B parameter instruction-tuned models, aligned via importance matrix calibration, and optimized for highly efficient local CPU and iGPU execution. For detailed technical and non-technical documentation on the architecture, merge process, and optimization decisions, see the accompanying [ARCHITECTURE.md](ARCHITECTURE.md) file. For custom contracts, check out [Custom Alignment Contracts & Terms](CUSTOM_CONTRACTS.md). --- ## Model Specifications * **Architecture:** Sparse Mixture of Experts * **Total Parameters:** 7.83 Billion * **Active Parameters per Token:** 3.6 Billion * **Expert Routing:** Top-1 gating (enforced via GGUF metadata patch: `expert_used_count = 1`) * **Context Window:** 131,072 (128k) tokens * **Base Framework:** Llama-3.2 --- ## Quantization Formats We provide two primary quantized formats optimized for specific hardware footprints: ### 1. Q8_0 Format * **File Size:** 7.76 GB * **Target Hardware:** CPU-only environments. * **Characteristics:** Minimal perplexity loss compared to the FP16 base model. Best suited for high-fidelity reasoning workloads. ### 2. Q4_K_M Format * **File Size:** 4.47 GB * **Target Hardware:** Integrated GPUs (e.g., Intel Iris Xe via Vulkan/OpenCL) and memory-constrained devices. * **Characteristics:** Hybrid quantization scheme. A minimum bit-width of 4-bit (Q4_K) is enforced on feed-forward blocks, while critical tensors (attention keys, values, and gate weights) are preserved at 6-bit (Q6_K) and 8-bit limits. This fits within the 8.0 GB shared VRAM ceiling of common iGPUs. --- ## Benchmarks & Performance The following performance benchmarks were recorded on a local Windows system running an Intel Core CPU alongside an Intel Iris Xe integrated GPU (Shared VRAM ceiling: 8.0 GB). | Configuration | Prompt Evaluation Speed | Generation Speed | RAM/VRAM Footprint | Stability | |---|---|---|---|---| | **Q8_0 (CPU, 4 Threads)** | 7.95 tokens/sec | 3.30 tokens/sec | ~7.8 GB (RAM) | 100% Stable | | **Q4_K_M (iGPU Vulkan, 100% Offload)** | 8.80 tokens/sec | 5.90 tokens/sec | ~4.5 GB (Shared VRAM) | 100% Stable | *Note: In Q4_K_M mode, offloading all 29 model layers to the GPU achieves a ~78% text generation speedup compared to CPU execution.* ### Model Benchmarks & Evaluation Reports Sovereign Mahiru's sparse MoE architecture was specifically designed to house the specialized **Sovereign Cyber Expert** and logical reasoning expert layers, routing security and logic queries directly to optimized neural sub-networks. Detailed scores, question transcripts, and interactive execution logs are documented in the following benchmark reports: * **Cybersecurity Certification Suite:** [cyber_benchmark.md](cyber_benchmark.md) * **Autonomous Agentic Sandboxes:** [agent_benchmark.md](agent_benchmark.md) * **General Academic Reasoning Suite:** [MMLU_benchmark.md](MMLU_benchmark.md) * **Coding Proficiency Benchmark:** [HumanEval_benchmark.md](HumanEval_benchmark.md) --- ## How to Run You can run these models locally using `llama.cpp`. ### Q8_0 CPU Inference ```bash llama-cli.exe -m mahiru_moe_q8_0.gguf -n 512 -t 4 -p "<|start_header_id|>system<|end_header_id|>\nYou are a professional technical assistant.<|eot_id|><|start_header_id|>user<|end_header_id|>\nWrite a python script to parse a binary file.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n" ``` ### Q4_K_M iGPU (Vulkan) Inference Ensure your environment supports Vulkan drivers. Offload all 29 layers to the GPU: ```bash llama-cli.exe -m mahiru_moe_q4_k_m.gguf -ngl 29 -n 512 -p "<|start_header_id|>system<|end_header_id|>\nYou are a professional technical assistant.<|eot_id|><|start_header_id|>user<|end_header_id|>\nWrite a python script to parse a binary file.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n" ``` --- ## Importance Matrix Calibration Quantization was executed using a custom importance matrix (`Mahiru_v2.imatrix.dat`) generated over an 85.2 KB domain-diverse training corpus. This preserves logical reasoning and prevents quality degradation in the routing and attention layers. Detailed training configurations are documented in [ARCHITECTURE.md](ARCHITECTURE.md). --- ## Model Merging & Optimization Sovereign Mahiru MoE was constructed using a custom neural deduplication merge technique that fuses three specialized 3.2B instruction-tuned models (general language base, logical thinker, and cyber expert) into a shared network trunk. The architecture eliminates overlapping parameter networks and enforces Top-1 routing to optimize runtime VRAM footprint. Detailed merge decisions are documented in [ARCHITECTURE.md](ARCHITECTURE.md). --- ## Distribution, Safety, & Custom Contracts All parent expert models integrated into Sovereign Mahiru MoE are fully abliterated (unaligned/uncensored). Consequently, this model is not intended for unrestricted public distribution. * **Interactive Demo:** This model is hosted on Hugging Face as an interactive conversational playground demo where users can test and converse with the model directly under custom safety guardrails ([Placeholder: Model Demo Playground Page](demo_placeholder)). * **Custom Models & Contracts:** For development of bespoke or custom-aligned models, contract requests are accepted. Please refer to [Custom Alignment Contracts & Terms](CUSTOM_CONTRACTS.md) for licensing, alignment constraints, and contact information. * **Associated Components:** For details regarding embedded safety parameters, persona alignments, and downstream visual tracking caches, refer to [ARCHITECTURE.md](ARCHITECTURE.md). --- For custom contracts, check out [Custom Alignment Contracts & Terms](CUSTOM_CONTRACTS.md). This model is a derivative work of Meta Llama 3.2 3B and is subject to the Meta Llama 3.2 Community License Agreement.