Malikeh1375 commited on
Commit
aa2d8a0
·
verified ·
1 Parent(s): c48258c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -66,13 +66,13 @@ This makes BLOOM a representative example of multilingual BPE tokenization.
66
 
67
  ## Model Architecture
68
 
69
- - **Architecture:** Decoder-only Transformer (LLaMA-style)
70
  - **Non-embedding parameters:** ~1B
71
  - **Context length:** 4096 tokens
72
  - **Framework:** Meta Lingua
73
- - **Initialization:** Shared super-vocabulary initialization for overlapping token strings
74
 
75
- The architecture and hyperparameters are fixed across TokSuite; the tokenizer is the only variable.
76
 
77
  ---
78
 
 
66
 
67
  ## Model Architecture
68
 
69
+ - **Architecture:** Decoder-only Transformer (Lingua's Llama-3.2-1B configuration)
70
  - **Non-embedding parameters:** ~1B
71
  - **Context length:** 4096 tokens
72
  - **Framework:** Meta Lingua
73
+ - **Initialization:** Shared super-vocabulary initialization across TokSuite models
74
 
75
+ The architecture and training setup are identical across all TokSuite models; only the tokenizer differs.
76
 
77
  ---
78