Fixed the tokenizer configuration to get it working with Transformers v5

1eebf05

verified

sorasmaa commited on 14 days ago