File size: 686 Bytes
4e33496 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # Neo Ar Model Specification
Neo Ar is an Arabic-only System One decision model with a larger encoder than
the English Neo model.
- 12 Transformer layers.
- Hidden size 768.
- 12 attention heads.
- Intermediate size 3,072.
- Maximum context 1,024 tokens.
- Up to 64 choice options.
- Up to 7 score levels.
- Binary `noul` decision.
- Custom Arabic WordPiece tokenizer.
The released architecture contains 86,393,160 trainable parameters with a
644-token Arabic vocabulary.
The final parameter count is measured from the released checkpoint after the
Arabic tokenizer vocabulary is finalized. The full source and training process
are documented at https://github.com/tokenaii/Neo-Ar.
|