mistralai
/

Mistral-Nemo-Instruct-2407

Model card Files Files and versions

Fix typo in README

#39

by Xenova HF Staff - opened Jul 22, 2024

base: refs/heads/main

←

from: refs/pr/39

Discussion Files changed

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -30,7 +30,7 @@ Mistral Nemo is a transformer model, with the following architecture choices:
 - **Layers:** 40
 - **Dim:** 5,120
 - **Head dim:** 128
-- **Hidden dim:** 14,436
 - **Activation Function:** SwiGLU
 - **Number of heads:** 32
 - **Number of kv-heads:** 8 (GQA)

 - **Layers:** 40
 - **Dim:** 5,120
 - **Head dim:** 128
+- **Hidden dim:** 14,336
 - **Activation Function:** SwiGLU
 - **Number of heads:** 32
 - **Number of kv-heads:** 8 (GQA)