amazon
/

MistralLite

Text Generation

text-generation-inference

Model card Files Files and versions

yinsong1986 commited on Oct 16, 2023

Commit

edaa6d1

·

1 Parent(s): 9416971

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -11,7 +11,7 @@ MistralLight evolves from [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mis
 |Model|Fine-tuned on long contexts| Quantization | Max context length| RotaryEmbedding adaptation| Sliding Window Size|
 |----------|-------------:|-------------:|------------:|-----------:|-----------:|
 | Mistral-7B-v0.1 | No | No | 36K | rope_theta = 10000 | 4096 |
-| MistralLight | Yes | No | 36K | **rope_theta = 1000000** | **16384** |
 ## Motivation of Developing MistralLite

 |Model|Fine-tuned on long contexts| Quantization | Max context length| RotaryEmbedding adaptation| Sliding Window Size|
 |----------|-------------:|-------------:|------------:|-----------:|-----------:|
 | Mistral-7B-v0.1 | No | No | 36K | rope_theta = 10000 | 4096 |
+| MistralLite | Yes | No | 36K | **rope_theta = 1000000** | **16384** |
 ## Motivation of Developing MistralLite