Update README.md (#2)
Browse files- Update README.md (f9b16284eebe09659076369c0075f5e5937e8c63)
Co-authored-by: Bo Zheng <bzheng@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -59,7 +59,7 @@ For more details, please refer to our blog post [Qwen3.5](https://qwen.ai/blog?i
|
|
| 59 |
- Rotary Position Embedding Dimension: 64
|
| 60 |
- Feed Forward Network:
|
| 61 |
- Intermediate Dimension: 9216
|
| 62 |
-
- LM Output: 248320 (
|
| 63 |
- MTP: trained with multi-steps
|
| 64 |
- Context Length: 262,144 natively and extensible up to 1,010,000 tokens.
|
| 65 |
|
|
|
|
| 59 |
- Rotary Position Embedding Dimension: 64
|
| 60 |
- Feed Forward Network:
|
| 61 |
- Intermediate Dimension: 9216
|
| 62 |
+
- LM Output: 248320 (Tied to token embedding)
|
| 63 |
- MTP: trained with multi-steps
|
| 64 |
- Context Length: 262,144 natively and extensible up to 1,010,000 tokens.
|
| 65 |
|