Neo-Ar / docs /MODEL_SPECIFICATION.md
assemsabry's picture
docs: add Neo Ar model specification
4e33496 verified
|
Raw History Blame Contribute Delete
686 Bytes

Neo Ar Model Specification

Neo Ar is an Arabic-only System One decision model with a larger encoder than the English Neo model.

  • 12 Transformer layers.
  • Hidden size 768.
  • 12 attention heads.
  • Intermediate size 3,072.
  • Maximum context 1,024 tokens.
  • Up to 64 choice options.
  • Up to 7 score levels.
  • Binary noul decision.
  • Custom Arabic WordPiece tokenizer.

The released architecture contains 86,393,160 trainable parameters with a 644-token Arabic vocabulary.

The final parameter count is measured from the released checkpoint after the Arabic tokenizer vocabulary is finalized. The full source and training process are documented at https://github.com/tokenaii/Neo-Ar.