Tiiny
/

SmallThinker-4BA0.6B-Instruct

Text Generation

feature-extraction

Model card Files Files and versions

Yixin Song commited on Jul 27

Commit

3044e6c

·

verified ·

1 Parent(s): c4c7dfc

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -28,7 +28,7 @@ SmallThinker is a family of **on-device native** Mixture-of-Experts (MoE) langua
 | **Activated Parameters** | 0.6B |
 | **Number of Layers** | 32 |
 | **Attention Hidden Dimension** | 1536 |
-| **MoE Hidden Dimension** (per Expert) | 1408 |
 | **Number of Attention Heads** | 12 |
 | **Number of Experts** | 32 |
 | **Selected Experts per Token** | 4 |

 | **Activated Parameters** | 0.6B |
 | **Number of Layers** | 32 |
 | **Attention Hidden Dimension** | 1536 |
+| **MoE Hidden Dimension** (per Expert) | 768 |
 | **Number of Attention Heads** | 12 |
 | **Number of Experts** | 32 |
 | **Selected Experts per Token** | 4 |