Update README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,7 @@ licence: license
|
|
| 15 |
|
| 16 |
This model is a fine-tuned version of [KotshinZ/gpt2-RMT-7-mem512](https://huggingface.co/KotshinZ/gpt2-RMT-7-mem512) on the [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) dataset.
|
| 17 |
It has been trained using [TRL](https://github.com/huggingface/trl).
|
|
|
|
| 18 |
|
| 19 |
## Quick start
|
| 20 |
|
|
@@ -33,6 +34,7 @@ print(output["generated_text"])
|
|
| 33 |
|
| 34 |
|
| 35 |
This model was trained with SFT.
|
|
|
|
| 36 |
|
| 37 |
### Framework versions
|
| 38 |
|
|
|
|
| 15 |
|
| 16 |
This model is a fine-tuned version of [KotshinZ/gpt2-RMT-7-mem512](https://huggingface.co/KotshinZ/gpt2-RMT-7-mem512) on the [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) dataset.
|
| 17 |
It has been trained using [TRL](https://github.com/huggingface/trl).
|
| 18 |
+
For use this model. You need clone [KotShinZ/Recurrent-Memory-Transformer_PreTrained.git](https://github.com/KotShinZ/Recurrent-Memory-Transformer_PreTrained)
|
| 19 |
|
| 20 |
## Quick start
|
| 21 |
|
|
|
|
| 34 |
|
| 35 |
|
| 36 |
This model was trained with SFT.
|
| 37 |
+
This model memory_size = 512, n_backward = 8.
|
| 38 |
|
| 39 |
### Framework versions
|
| 40 |
|