Upload tokenizer
#4
by
ArthurZ
HF Staff
- opened
Following the merge of a PR in `transformers` it appeared thatthis model was not properly converted. This PR will fix the inference and was tested using the following script:```python>>> from transformers import MarianModel, MarianMTModel>>> tokenizer = AutoTokenizer.from_pretrained("$model_id")>>> inputs = tokenizer("$translation", return_tensors="pt", padding=True)>>> model = MarianMTModel.from_pretrained("$model_id")>>> print(tokenizer.batch_decode(model.generate(**inputs)))"$captured_output"```