KitsuVp commited on
Commit
3404b3b
·
verified ·
1 Parent(s): 8bc512b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -72,7 +72,15 @@ NeoLLM incorporates several cutting-edge components:
72
  - **Polynomial Composition Activations**: PolyNorm activation functions in MLP layers for enhanced dynamics
73
  - **Advanced Normalization**: LayerNorm Scaling (LNS) and Gradient-Preserving Activation Scaling (GPAS)
74
  - **Efficient Linear Attention**: Gated Delta Networks for improved computational efficiency
 
75
 
 
 
 
 
 
 
 
76
 
77
 
78
  ### Architecture Details
 
72
  - **Polynomial Composition Activations**: PolyNorm activation functions in MLP layers for enhanced dynamics
73
  - **Advanced Normalization**: LayerNorm Scaling (LNS) and Gradient-Preserving Activation Scaling (GPAS)
74
  - **Efficient Linear Attention**: Gated Delta Networks for improved computational efficiency
75
+ ## Installation
76
 
77
+ Before using this model, install the required dependencies:
78
+
79
+ ```bash
80
+ pip install git+https://github.com/huggingface/transformers.git@main
81
+ pip install "cut-cross-entropy @ git+https://github.com/apple/ml-cross-entropy.git"
82
+ pip install flash-linear-attention
83
+ ```
84
 
85
 
86
  ### Architecture Details