Upload README.md
Browse files
README.md
CHANGED
|
@@ -7,7 +7,7 @@ language:
|
|
| 7 |
- en
|
| 8 |
library_name: transformers
|
| 9 |
license: apache-2.0
|
| 10 |
-
model_creator:
|
| 11 |
model_name: OpenInstruct Mistral 7B
|
| 12 |
model_type: mistral
|
| 13 |
pipeline_tag: text-generation
|
|
@@ -45,13 +45,13 @@ quantized_by: TheBloke
|
|
| 45 |
<!-- header end -->
|
| 46 |
|
| 47 |
# OpenInstruct Mistral 7B - AWQ
|
| 48 |
-
- Model creator: [
|
| 49 |
- Original model: [OpenInstruct Mistral 7B](https://huggingface.co/monology/openinstruct-mistral-7b)
|
| 50 |
|
| 51 |
<!-- description start -->
|
| 52 |
## Description
|
| 53 |
|
| 54 |
-
This repo contains AWQ model files for [
|
| 55 |
|
| 56 |
These files were quantised using hardware kindly provided by [Massed Compute](https://massedcompute.com/).
|
| 57 |
|
|
@@ -75,7 +75,7 @@ It is supported by:
|
|
| 75 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ)
|
| 76 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GPTQ)
|
| 77 |
* [2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GGUF)
|
| 78 |
-
* [
|
| 79 |
<!-- repositories-available end -->
|
| 80 |
|
| 81 |
<!-- prompt-template start -->
|
|
@@ -103,7 +103,7 @@ Models are released as sharded safetensors files.
|
|
| 103 |
|
| 104 |
| Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
|
| 105 |
| ------ | ---- | -- | ----------- | ------- | ---- |
|
| 106 |
-
| [main](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ/tree/main) | 4 | 128 | [
|
| 107 |
|
| 108 |
<!-- README_AWQ.md-provided-files end -->
|
| 109 |
|
|
@@ -372,7 +372,7 @@ And thank you again to a16z for their generous grant.
|
|
| 372 |
|
| 373 |
<!-- footer end -->
|
| 374 |
|
| 375 |
-
# Original model card:
|
| 376 |
|
| 377 |
|
| 378 |
# OpenInstruct Mistral-7B
|
|
@@ -383,7 +383,7 @@ Quantized to FP16 and released under the [Apache-2.0](https://choosealicense.com
|
|
| 383 |
Compute generously provided by [Higgsfield AI](https://higgsfield.ai/model/655559e6b5777dab620095e0).
|
| 384 |
|
| 385 |
|
| 386 |
-
Prompt format: Alpaca
|
| 387 |
```
|
| 388 |
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
| 389 |
|
|
@@ -393,4 +393,10 @@ Below is an instruction that describes a task. Write a response that appropriate
|
|
| 393 |
### Response:
|
| 394 |
```
|
| 395 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 396 |
\*as of 21 Nov 2023. "commercially-usable" includes both an open-source base model and a *non-synthetic* open-source finetune dataset. updated leaderboard results available [here](https://huggingfaceh4-open-llm-leaderboard.hf.space).
|
|
|
|
| 7 |
- en
|
| 8 |
library_name: transformers
|
| 9 |
license: apache-2.0
|
| 10 |
+
model_creator: Devin Gulliver
|
| 11 |
model_name: OpenInstruct Mistral 7B
|
| 12 |
model_type: mistral
|
| 13 |
pipeline_tag: text-generation
|
|
|
|
| 45 |
<!-- header end -->
|
| 46 |
|
| 47 |
# OpenInstruct Mistral 7B - AWQ
|
| 48 |
+
- Model creator: [Devin Gulliver](https://huggingface.co/monology)
|
| 49 |
- Original model: [OpenInstruct Mistral 7B](https://huggingface.co/monology/openinstruct-mistral-7b)
|
| 50 |
|
| 51 |
<!-- description start -->
|
| 52 |
## Description
|
| 53 |
|
| 54 |
+
This repo contains AWQ model files for [Devin Gulliver's OpenInstruct Mistral 7B](https://huggingface.co/monology/openinstruct-mistral-7b).
|
| 55 |
|
| 56 |
These files were quantised using hardware kindly provided by [Massed Compute](https://massedcompute.com/).
|
| 57 |
|
|
|
|
| 75 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ)
|
| 76 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GPTQ)
|
| 77 |
* [2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GGUF)
|
| 78 |
+
* [Devin Gulliver's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/monology/openinstruct-mistral-7b)
|
| 79 |
<!-- repositories-available end -->
|
| 80 |
|
| 81 |
<!-- prompt-template start -->
|
|
|
|
| 103 |
|
| 104 |
| Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
|
| 105 |
| ------ | ---- | -- | ----------- | ------- | ---- |
|
| 106 |
+
| [main](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ/tree/main) | 4 | 128 | [VMware Open Instruct](https://huggingface.co/datasets/VMware/open-instruct/viewer/) | 4096 | 4.15 GB
|
| 107 |
|
| 108 |
<!-- README_AWQ.md-provided-files end -->
|
| 109 |
|
|
|
|
| 372 |
|
| 373 |
<!-- footer end -->
|
| 374 |
|
| 375 |
+
# Original model card: Devin Gulliver's OpenInstruct Mistral 7B
|
| 376 |
|
| 377 |
|
| 378 |
# OpenInstruct Mistral-7B
|
|
|
|
| 383 |
Compute generously provided by [Higgsfield AI](https://higgsfield.ai/model/655559e6b5777dab620095e0).
|
| 384 |
|
| 385 |
|
| 386 |
+
## Prompt format: Alpaca
|
| 387 |
```
|
| 388 |
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
| 389 |
|
|
|
|
| 393 |
### Response:
|
| 394 |
```
|
| 395 |
|
| 396 |
+
## Recommended preset:
|
| 397 |
+
- temperature: 0.2
|
| 398 |
+
- top_k: 50
|
| 399 |
+
- top_p 0.95
|
| 400 |
+
- repetition_penalty: 1.1
|
| 401 |
+
|
| 402 |
\*as of 21 Nov 2023. "commercially-usable" includes both an open-source base model and a *non-synthetic* open-source finetune dataset. updated leaderboard results available [here](https://huggingfaceh4-open-llm-leaderboard.hf.space).
|