Sovereign Superintelligence
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention