Create README.md

README.md

@@ -0,0 +1,32 @@
+## MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices (Safetensors Checkpoint)
+
+MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks.
+See [here](https://huggingface.co/google/mobilebert-uncased) for the original model checkpoint in TensorFlow. This is simply that checkpoint converted to safetensors.
+
+## Example usage in `transformers`
+```python
+from transformers import MobileBertTokenizer, MobileBertForMaskedLM
+import torch
+
+tokenizer = MobileBertTokenizer.from_pretrained("google/mobilebert-uncased")
+
+model = MobileBertForMaskedLM.from_pretrained(
+    "vysri/mobilebert-uncased-pytorch"
+)
+
+model.eval()
+sentence = "The capital of France is [MASK]."
+inputs = tokenizer(sentence, return_tensors="pt")
+
+with torch.no_grad():
+    outputs = model(**inputs)
+
+mask_token_index = (inputs.input_ids == tokenizer.mask_token_id)[0].nonzero(as_tuple=True)[0]
+predicted_token_id = outputs.logits[0, mask_token_index].argmax(axis=-1)
+predicted_token = tokenizer.decode(predicted_token_id)
+
+print(f"Input: {sentence}")
+print(f"Prediction: {predicted_token}")
+
+```
+