tokenizer
#5
by nazgut - opened
Why do you have flan t5 as tokenizer?
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-small")
That is the tokenizer used for the base model, which is flan-t5-small.
roborovski changed discussion status to closed
But it doesn't make any sense, your model is based on flan-t5 so it should/could be used as tokenizer, right?