Update README.md

README.md

@@ -68,11 +68,13 @@ NeoLLM is a hybrid architecture language model that combines multiple state-of-t
 NeoLLM incorporates several cutting-edge components:
 
 - **FANformer Integration**: Fourier Analysis Network (FAN) layers for effective periodicity modeling with fan_ratio of 0.125
-- **Hybrid Attention Architecture**: Alternates between full attention and linear attention (Gated Delta Net) layers inspired by Qwen3-Next
+- **Hybrid Attention Architecture**: Follows Qwen3-Next's approach with 1 full attention layer per 3 linear attention layers
 - **Polynomial Composition Activations**: PolyNorm activation functions in MLP layers for enhanced dynamics
 - **Advanced Normalization**: LayerNorm Scaling (LNS) and Gradient-Preserving Activation Scaling (GPAS)
 - **Efficient Linear Attention**: Gated Delta Networks for improved computational efficiency
 
+
+
 ### Architecture Details
 
 - **Model Size**: 110M parameters (77M embeddings + 33M non-embeddings)