RFC: MiniCPM5-2B-Hadamard-GSQ (DV-SSQ & KV-BSS Architecture)

Greetings OpenBMB Team and Community,

We would like to share an architectural proposal and open-source release: MiniCPM5-2B-Hadamard-GSQ, engineered at F-Labs.

• Model Weights & Metadata: F-Labs/MiniCPM5-2B-Hadamard-GSQ
• Source Code & Modules: GitHub: dsadawq3/MiniCPM5-2B-Hadamard-GSQ
• Framework Core: GitHub: dsadawq3/FQuant

Motivation & Challenges in 4-bit Edge Inference

MiniCPM5-2B is inherently designed for edge and on-device execution with deep 42-layer transformer reasoning and a massive 131,072 token (128K) context window. However, running uncompressed BF16 requires 4.69 GB of physical memory, creating heavy pressure on 4GB-6GB edge devices.

Standard post-training quantization methods (naive INT4 or uniform RTN) suffer from three primary degradation modes across 42 sequential layers:

1. Activation Outlier Spikes: Coordinate-aligned outliers in hidden channels (d = 2048) induce severe clipping errors.
2. GQA 8:1 Attention Fragility: With only 2 Key-Value heads per layer, any perturbation in k_proj corrupts 50% of the layer's associative memory due to exponential amplification in exp(Q · Kᵀ / √d).
3. 128K Context Attention Haze: In long contexts, Softmax probability mass diffuses across thousands of irrelevant background tokens, leading to entity and key-value hallucinations (["key"] => "value").

Key Architectural Solutions

1. Walsh-Hadamard (H₁₂₈ / H₂₀₄₈) Spin Rotation:
   Because hidden_size is exactly 2048 (2¹¹), we apply orthonormal Sylvester Walsh-Hadamard rotation matrices:
   W' = W · Hᵀ, X' = X · H
   This algebraically preserves linear outputs (X' W'ᵀ = X Wᵀ) while rotating coordinates to suppress activation outlier peaks by over 82%.

2. Group-Scale INT4 (GSQ) + Residual SVD Compensation (RCO):

   • MLP parameter mass (67.4% of model weight) is quantized to INT4 (group size G = 64).
   • Residual discretization error R = W - Ŵ is decomposed via truncated Low-Rank SVD:
     R ≈ U_r · Σ_r · V_rᵀ = A · B
     Stored in BF16 factor matrices (r = 16 on standard layers, r = 24 on bifurcation abstraction hubs).

3. Key-Projection Exponential Sensitivity Defense:
   All 42 k_proj matrices are fortified with doubled SVD rank (r = 32). Because total k_proj weights across all layers amount to only ~44 MB (<0.9% of the model), allocating extra low-rank capacity costs negligible RAM while completely preserving attention stability.

4. Zero-Compression Shield:
   All 85 RMSNorm weight vectors, projection biases, and token embeddings remain in 100% pristine BF16, eliminating cumulative phase drift across all 42 layers.

5. KV-BSS (Key-Value Binding Softmax Sharpening):
   An attention hook implementing focus temperature scaling (τ_focus = 1.10) and dynamic attention haze floor filtering (< max - 12.0), preventing hallucination on structured associative recall in 128k contexts.

Empirical Summary

We invite feedback and collaboration from the OpenBMB engineering team.