Create README.md

README.md

@@ -0,0 +1,51 @@
+---
+license: apache-2.0
+language:
+- en
+tags:
+- moe
+- olmo
+- olmoe
+co2_eq_emissions: 0
+---
+
+![olmoe](https://github.com/allenai/OLMoE/blob/main/visuals/logo/OLMoE_4.png?raw=true)
+
+# Model Summary
+
+> OLMoE is a ...
+
+Links to all resources & instructions to reproduce: https://github.com/allenai/OLMoE
+
+# Use
+
+Install the `transformers` & `torch` libraries and run:
+
+```python
+from transformers import OlmoeForCausalLM, AutoTokenizer
+import torch
+
+DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
+
+# Load different ckpts via passing e.g. `revision=step10000-tokens41B`
+model = OlmoeForCausalLM.from_pretrained("OLMoE/OLMoE-1B-7B-0824").to(DEVICE)
+tokenizer = AutoTokenizer.from_pretrained("OLMoE/OLMoE-1B-7B-0824")
+inputs = tokenizer("Bitcoin is", return_tensors="pt")
+inputs = {k: v.to(DEVICE) for k, v in inputs.items()}
+out = model.generate(**inputs, max_length=64)
+print(tokenizer.decode(out[0]))
+# > # Bitcoin is a digital currency that is created and held electronically. No one controls it. Bitcoins aren’t printed, like dollars or euros – they’re produced by people and businesses running computers all around the world, using software that solves mathematical
+```
+
+You can list all revisions/branches by installing `huggingface-hub` & running:
+```python
+from huggingface_hub import list_repo_refs
+out = list_repo_refs("OLMoE/OLMoE-1B-7B-0824")
+branches = [b.name for b in out.branches]
+```
+
+# Citation
+
+```bibtex
+TODO
+```