Mxode commited on
Commit
40485da
·
verified ·
1 Parent(s): dfbf989

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -20,13 +20,13 @@ The ONNX version of the model is also available in the repository.
20
 
21
 
22
  | Size | P. | Arch. | Act. | V. | H. | I. | L. | A.H. | K.H. | Tie |
23
- | :--: | :-----: | :--: | :--: | :--: | :-----: | :---: | :------: | ---- | ---- | :--: |
24
- | XL | 100 | LLaMA | SwiGLU | 16000 | 768 | 4096 | 8 | 24 | 8 | True |
25
- | L | 78 | LLaMA | GeGLU | 16000 | 768 | 4096 | 6 | 24 | 8 | True |
26
- | M2 | 22 | Qwen2 | GeGLU | 4000 | 432 | 2304 | 6 | 24 | 8 | True |
27
- | M | 22 | LLaMA | SwiGLU | 8000 | 256 | 1408 | 16 | 16 | 4 | True |
28
- | S | 9 | LLaMA | SwiGLU | 4000 | 168 | 896 | 16 | 12 | 4 | True |
29
- | XS | 2 | LLaMA | SwiGLU | 2000 | 96 | 512 | 12 | 12 | 4 | True |
30
 
31
  - **P.** - Parameters (in million)
32
  - **V.** - vocab size
 
20
 
21
 
22
  | Size | P. | Arch. | Act. | V. | H. | I. | L. | A.H. | K.H. | Tie |
23
+ | :--: | :-----: | :--: | :--: | :--: | :-----: | :---: | :------: | :----: | :----: | :--: |
24
+ | XL | 100 | LLaMA | SwiGLU | 16K | 768 | 4096 | 8 | 24 | 8 | True |
25
+ | L | 78 | LLaMA | GeGLU | 16K | 768 | 4096 | 6 | 24 | 8 | True |
26
+ | M2 | 22 | Qwen2 | GeGLU | 4K | 432 | 2304 | 6 | 24 | 8 | True |
27
+ | M | 22 | LLaMA | SwiGLU | 8K | 256 | 1408 | 16 | 16 | 4 | True |
28
+ | S | 9 | LLaMA | SwiGLU | 4K | 168 | 896 | 16 | 12 | 4 | True |
29
+ | XS | 2 | LLaMA | SwiGLU | 2K | 96 | 512 | 12 | 12 | 4 | True |
30
 
31
  - **P.** - Parameters (in million)
32
  - **V.** - vocab size