Language.converters / tokenizer.json
CoderHassan's picture
Create tokenizer.json
783f9ba verified
raw
history blame
170 Bytes
Invalid JSON: Unexpected token 'r', "from transf"... is not valid JSON
from transformers import T5Tokenizer
tokenizer = T5Tokenizer.from_pretrained('t5-small') # or your own trained model
tokenizer.save_pretrained('./t5_urdu_translation')