File size: 263 Bytes
85a524c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | ## Tokenizer training
timestamp: 2025-11-03 05:56:12
- max_chars: 2,000,000,000
- doc_cap: 10,000
- vocab_size: 65,536
- train_time: 57.1515
- num_special_tokens: 9
- token_bytes_min: 1
- token_bytes_max: 32
- token_bytes_mean: 6.9197
- token_bytes_std: 2.8748
|