Update README.md
Browse files
README.md
CHANGED
|
@@ -11,10 +11,10 @@ license: bigscience-openrail-m
|
|
| 11 |
[GeoV](https://github.com/geov-ai/geov)-9B is a 9 billion parameter causal language model.
|
| 12 |
|
| 13 |
The GeoV model was designed by Georges Harik and uses
|
| 14 |
-
[Rotary Positional Embeddings with Relative distances (RoPER)](
|
| 15 |
by [Georges Harik](https://twitter.com/gharik) and [Varuna Jayasiri](https://twitter.com/vpj).
|
| 16 |
|
| 17 |
-
[RoPER](
|
| 18 |
in addition to using relative positions in the attention score calculation by RoPE embeddings,
|
| 19 |
adds relative positional information explicitly to value embeddings.
|
| 20 |
Specifically, it incorporates the relative positions of the tokens paid attention to.
|
|
|
|
| 11 |
[GeoV](https://github.com/geov-ai/geov)-9B is a 9 billion parameter causal language model.
|
| 12 |
|
| 13 |
The GeoV model was designed by Georges Harik and uses
|
| 14 |
+
[Rotary Positional Embeddings with Relative distances (RoPER)](http://research.labml.ai/RoPER.html)
|
| 15 |
by [Georges Harik](https://twitter.com/gharik) and [Varuna Jayasiri](https://twitter.com/vpj).
|
| 16 |
|
| 17 |
+
[RoPER](http://research.labml.ai/RoPER.html),
|
| 18 |
in addition to using relative positions in the attention score calculation by RoPE embeddings,
|
| 19 |
adds relative positional information explicitly to value embeddings.
|
| 20 |
Specifically, it incorporates the relative positions of the tokens paid attention to.
|