juliadistill-v2 / tokenizer.json

Commit History

Add 4000-vocab BPE tokenizer for inference serving
dd923fa
verified

LisaMegaWatts commited on