question about tokenizer
#7
by surya-narayanan - opened
Hi,
do the tiny, small, medium and mini models all use the same tokenizer?
Yes
prajjwal1 changed discussion status to closed
Hi,
do the tiny, small, medium and mini models all use the same tokenizer?
Yes