question about tokenizer
#7
by
surya-narayanan - opened
Hi,
do the tiny, small, medium and mini models all use the same tokenizer?
Yes
prajjwal1 changed discussion status to
closed
Hi,
do the tiny, small, medium and mini models all use the same tokenizer?
Yes