| ## Tokenizer training | |
| timestamp: 2025-11-03 05:56:12 | |
| - max_chars: 2,000,000,000 | |
| - doc_cap: 10,000 | |
| - vocab_size: 65,536 | |
| - train_time: 57.1515 | |
| - num_special_tokens: 9 | |
| - token_bytes_min: 1 | |
| - token_bytes_max: 32 | |
| - token_bytes_mean: 6.9197 | |
| - token_bytes_std: 2.8748 | |