Upload tokenizer

#4
by ArthurZ HF Staff - opened

Following the merge of a PR in `transformers` it appeared thatthis model was not properly converted. This PR will fix the inference and was tested using the following script:```python>>> from transformers import MarianModel, MarianMTModel>>> tokenizer = AutoTokenizer.from_pretrained("$model_id")>>> inputs = tokenizer("$translation", return_tensors="pt", padding=True)>>> model = MarianMTModel.from_pretrained("$model_id")>>> print(tokenizer.batch_decode(model.generate(**inputs)))"$captured_output"```

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment