Training in progress, step 17500 c0ae9d0
Long Nguyen commited on
How to use short0/shallow-attention-bert-base-uncased with Transformers:
# Load model directly
from transformers import AutoTokenizer, AutoModelForPreTraining
tokenizer = AutoTokenizer.from_pretrained("short0/shallow-attention-bert-base-uncased")
model = AutoModelForPreTraining.from_pretrained("short0/shallow-attention-bert-base-uncased", device_map="auto")