Update README.md
Browse files
README.md
CHANGED
|
@@ -59,7 +59,7 @@ LongLLaMA Code is built upon the foundation of [Code Llama](https://huggingface.
|
|
| 59 |
|
| 60 |
|
| 61 |
**LongLLaMA** is an [OpenLLaMA](https://github.com/openlm-research/open_llama) model finetuned with the FoT method,
|
| 62 |
-
with three layers used for context extension. **Crucially, LongLLaMA is able to extrapolate much beyond the context length seen in training:
|
| 63 |
**LongLLaMA Code** is a [Code Llama](https://huggingface.co/codellama/CodeLlama-7b-hf) model finetuned with the FoT method.
|
| 64 |
|
| 65 |
|
|
|
|
| 59 |
|
| 60 |
|
| 61 |
**LongLLaMA** is an [OpenLLaMA](https://github.com/openlm-research/open_llama) model finetuned with the FoT method,
|
| 62 |
+
with three layers used for context extension. **Crucially, LongLLaMA is able to extrapolate much beyond the context length seen in training: 8k. E.g., in the passkey retrieval task, it can handle inputs of length 256k**.
|
| 63 |
**LongLLaMA Code** is a [Code Llama](https://huggingface.co/codellama/CodeLlama-7b-hf) model finetuned with the FoT method.
|
| 64 |
|
| 65 |
|