Create README.md

README.md

@@ -0,0 +1,19 @@
+### How to use
+Here is how to use this model in PyTorch:
+```python
+from transformers import PerceiverTokenizer, PerceiverForMaskedLM
+tokenizer = PerceiverTokenizer.from_pretrained("deepmind/language-perceiver")
+model = PerceiverForMaskedLM.from_pretrained("deepmind/language-perceiver")
+text = "This is an incomplete sentence where some words are missing."
+# prepare input
+encoding = tokenizer(text, padding="max_length", return_tensors="pt")
+# mask " missing.". Note that the model performs much better if the masked span starts with a space.
+encoding.input_ids[0, 52:61] = tokenizer.mask_token_id
+inputs, input_mask = encoding.input_ids.to(device), encoding.attention_mask.to(device)
+# forward pass
+outputs = model(inputs=inputs, attention_mask=input_mask)
+logits = outputs.logits
+masked_tokens_predictions = logits[0, 51:61].argmax(dim=-1)
+print(tokenizer.decode(masked_tokens_predictions))
+>>> should print " missing."
+```