# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("flax-community/roberta-base-als")
model = AutoModelForMaskedLM.from_pretrained("flax-community/roberta-base-als")Quick Links
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
This project pretrains a roberta-base on the Alemannic (als) data subset of the OSCAR corpus in JAX/Flax.
We will be using the masked-language modeling loss for pretraining.
- Downloads last month
- 5
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="flax-community/roberta-base-als")