Update README.md

README.md

@@ -8,9 +8,20 @@ license: bigscience-openrail-m
 ---
 
 
-GeoV-9B is a 20 billion parameter autoregressive language model
+[GeoV](https://huggingface.co/docs/transformers/model_doc/geov)-9B is a 20 billion parameter autoregressive language model.
 
-### Model details
+The GeoV model was designed by Georges Harik and uses 
+[Rotary Positional Embeddings with Relative distances (RoPER)](http://research.labml.ai/RoPER.html)
+by [Georges Hark](https://twitter.com/ghark) and [Varuna Jayasiri](https://twitter.com/vpj).
+
+[RoPER]((http://research.labml.ai/RoPER.html), 
+in addition to using relative positions in the attention score calculation by RoPE embeddings,
+adds relative positional information explicitly to value embeddings.
+Specifically, it incorporates the relative positions of the tokens paid attention to.
+RoPER gives better performance in algorithmic tasks.
+Results have shown an improvement over RoPE in a language modeling setting on a 3 billion parameter transformer.
+
+## Model details
 
 - Developed by: [Georges Harik](http://twitter.com/gharik)
 - Model type: Transformer-based Language Model
@@ -29,3 +40,26 @@ GeoV-9B is a 20 billion parameter autoregressive language model
 | Sequence Length        | 2049        |
 </figure>
 
+
+## Generation
+
+The `generate()` method can be used to generate text using GeoV model.
+
+```python
+>>> from transformers import GeoVForCausalLM, GeoVTokenizer
+
+>>> model = GeoVForCausalLM.from_pretrained("GoeV/GeoV-9b")
+>>> tokenizer = GeoVTokenizer.from_pretrained("GoeV/GeoV-9b")
+
+>>> prompt = "In mathematics, topology is the study of"
+
+>>> input_ids = tokenizer(prompt, return_tensors="pt").input_ids
+
+>>> gen_tokens = model.generate(
+...     input_ids,
+...     do_sample=True,
+...     temperature=0.9,
+...     max_length=100,
+... )
+>>> gen_text = tokenizer.batch_decode(gen_tokens)[0]
+```