| | --- |
| | license: other |
| | --- |
| | # Manticore-13b-Landmark |
| |
|
| | ## Key Features |
| | - **[Landmark Attention](https://arxiv.org/pdf/2305.16300v1.pdf)** |
| | - **[Large Context Size (~18k)](https://i.ibb.co/tLLGLNc/image.jpg)** |
| |
|
| | ## Composition |
| | Manticore-13b-Landmark is a blend of: |
| | - [Manticore-13B](https://huggingface.co/openaccess-ai-collective/manticore-13b) |
| | - [Manticore-13B-Landmark-QLoRA](https://huggingface.co/Honkware/Manticore-13b-Landmark-QLoRA) |
| |
|
| | ## Using [Oobabooga](https://github.com/oobabooga/text-generation-webui) |
| | - Trust Remote Code - **(Enabled)** |
| | - Add the bos_token to the beginning of prompts - **(Disabled)** |
| | - Truncate the prompt up to this length - **(Increased)** |
| | |
| | ## Landmark Training Code |
| | See [GitHub](https://github.com/eugenepentland/landmark-attention-qlora) for the training code. |
