Manifest — 23.1B tokens

langchain_docs — 46,653,933 tokens (0.1GB)

coder_combined — 74,567,978 tokens (0.1GB)

sinhalen_gz — 2,000,000,000 tokens (3.7GB)

sinhala_cx — 8,000,000,000 tokens (14.9GB)

fineweb_edu — 5,000,000,000 tokens (9.3GB)

english_cx — 8,000,000,000 tokens (14.9GB)

Upload tokenizer.json

Delete tokenizer.json

Update tokenizer.json

Update tokenizer.json

Upload tokenizer.json

initial commit