pankajmathur
/

nanochat-d20

Text Generation

Model card Files Files and versions

nanochat-d20 / report /tokenizer-training.md

pankajmathur's picture

Upload folder using huggingface_hub

4dec898 verified 7 months ago

|

history blame contribute delete

263 Bytes

Tokenizer training

timestamp: 2025-10-18 21:16:38

max_chars: 2,000,000,000
doc_cap: 10,000
vocab_size: 65,536
train_time: 58.2860
num_special_tokens: 9
token_bytes_min: 1
token_bytes_max: 32
token_bytes_mean: 6.9197
token_bytes_std: 2.8748