YAML Metadata Error:"widget" must be an array

This is a pruned version of the google/mt5-large model. Here, the input and output embeddings are pruned to support a greatly reduced vocabulary. The chosen vocabulary has 30K norwegian, english and special tokens, ~12% of the old size. This reduces the model size by roughly 37%. The model is still OK on similar languages, like German and Danish, but very different languages like arabic are not a good fit anymore. This model is intended as a starting point for finetuning mt5 for norwegian applications.

In addition it has been trained on Nynorsk using masked language modeling, in accordance with pretraining objective in the T5 paper.

Downloads last month: 6

Safetensors

Model size

0.8B params

Tensor type

F32

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support