Update README
Browse files
README.md
CHANGED
|
@@ -1,13 +1,29 @@
|
|
| 1 |
---
|
| 2 |
inference: false
|
| 3 |
license: other
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
-
#
|
| 9 |
|
| 10 |
-
These files are GGML format model files for [
|
| 11 |
|
| 12 |
GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/ggerganov/llama.cpp) and libraries and UIs which support this format, such as:
|
| 13 |
* [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
|
|
@@ -18,9 +34,20 @@ GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/gger
|
|
| 18 |
|
| 19 |
## Repositories available
|
| 20 |
|
| 21 |
-
* [4-bit GPTQ models for GPU inference](https://huggingface.co
|
| 22 |
-
* [4-bit, 5-bit, and 8-bit GGML models for CPU+GPU inference](https://huggingface.co
|
| 23 |
-
* [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
## THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 19th 2023 - commit 2d5db48)!
|
| 26 |
|
|
@@ -31,7 +58,12 @@ I have quantised the GGML files in this repo with the latest version. Therefore
|
|
| 31 |
## Provided files
|
| 32 |
| Name | Quant method | Bits | Size | Max RAM required | Use case |
|
| 33 |
| ---- | ---- | ---- | ---- | ---- | ----- |
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
**Note**: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
|
| 37 |
|
|
@@ -40,7 +72,7 @@ I have quantised the GGML files in this repo with the latest version. Therefore
|
|
| 40 |
I use the following command line; adjust for your tastes and needs:
|
| 41 |
|
| 42 |
```
|
| 43 |
-
./main -t 10 -ngl 32 -m
|
| 44 |
```
|
| 45 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
| 46 |
|
|
@@ -52,6 +84,78 @@ If you want to have a chat-style conversation, replace the `-p <PROMPT>` argumen
|
|
| 52 |
|
| 53 |
Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).
|
| 54 |
|
| 55 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
-
|
|
|
|
| 1 |
---
|
| 2 |
inference: false
|
| 3 |
license: other
|
| 4 |
+
datasets:
|
| 5 |
+
- ehartford/based
|
| 6 |
+
language:
|
| 7 |
+
- en
|
| 8 |
---
|
| 9 |
|
| 10 |
+
<!-- header start -->
|
| 11 |
+
<div style="width: 100%;">
|
| 12 |
+
<img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
| 13 |
+
</div>
|
| 14 |
+
<div style="display: flex; justify-content: space-between; width: 100%;">
|
| 15 |
+
<div style="display: flex; flex-direction: column; align-items: flex-start;">
|
| 16 |
+
<p><a href="https://discord.gg/Jq4vkcDakD">Chat & support: my new Discord server</a></p>
|
| 17 |
+
</div>
|
| 18 |
+
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
| 19 |
+
<p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
|
| 20 |
+
</div>
|
| 21 |
+
</div>
|
| 22 |
+
<!-- header end -->
|
| 23 |
|
| 24 |
+
# Eric Hartford's Based 7B GGML
|
| 25 |
|
| 26 |
+
These files are GGML format model files for [Eric Hartford's Based 7B](https://huggingface.co/ehartford/based-7B).
|
| 27 |
|
| 28 |
GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/ggerganov/llama.cpp) and libraries and UIs which support this format, such as:
|
| 29 |
* [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
|
|
|
|
| 34 |
|
| 35 |
## Repositories available
|
| 36 |
|
| 37 |
+
* [4-bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/based-7B-GPTQ)
|
| 38 |
+
* [4-bit, 5-bit, and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/based-7B-GGML)
|
| 39 |
+
* [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/ehartford/based-7B)
|
| 40 |
+
|
| 41 |
+
### Prompt template
|
| 42 |
+
|
| 43 |
+
This uses Vicuna 1.1 format. Example:
|
| 44 |
+
|
| 45 |
+
```
|
| 46 |
+
You are an an AI who shares your opinions, thoughts, and feelings.
|
| 47 |
+
|
| 48 |
+
USER: What is your favorite football team?
|
| 49 |
+
ASSISTANT:
|
| 50 |
+
```
|
| 51 |
|
| 52 |
## THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 19th 2023 - commit 2d5db48)!
|
| 53 |
|
|
|
|
| 58 |
## Provided files
|
| 59 |
| Name | Quant method | Bits | Size | Max RAM required | Use case |
|
| 60 |
| ---- | ---- | ---- | ---- | ---- | ----- |
|
| 61 |
+
| based-7B.ggmlv3.q4_0.bin | q4_0 | 4 | 3.79 GB | 6.29 GB | 4-bit. |
|
| 62 |
+
| based-7B.ggmlv3.q4_1.bin | q4_1 | 4 | 4.21 GB | 6.71 GB | 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
|
| 63 |
+
| based-7B.ggmlv3.q5_0.bin | q5_0 | 5 | 4.63 GB | 7.13 GB | 5-bit. Higher accuracy, higher resource usage and slower inference. |
|
| 64 |
+
| based-7B.ggmlv3.q5_1.bin | q5_1 | 5 | 5.06 GB | 7.56 GB | 5-bit. Even higher accuracy, resource usage and slower inference. |
|
| 65 |
+
| based-7B.ggmlv3.q8_0.bin | q8_0 | 8 | 7.16 GB | 9.66 GB | 8-bit. Almost indistinguishable from float16. Huge resource use and slow. Not recommended for normal use. |
|
| 66 |
+
|
| 67 |
|
| 68 |
**Note**: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
|
| 69 |
|
|
|
|
| 72 |
I use the following command line; adjust for your tastes and needs:
|
| 73 |
|
| 74 |
```
|
| 75 |
+
./main -t 10 -ngl 32 -m based-7B.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "You are an an AI who shares your opinions, thoughts, and feelings.\nUSER: What is your favorite football team?\nASSISTANT:"
|
| 76 |
```
|
| 77 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
| 78 |
|
|
|
|
| 84 |
|
| 85 |
Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).
|
| 86 |
|
| 87 |
+
<!-- footer start -->
|
| 88 |
+
## Discord
|
| 89 |
+
|
| 90 |
+
For further support, and discussions on these models and AI in general, join us at:
|
| 91 |
+
|
| 92 |
+
[TheBloke AI's Discord server](https://discord.gg/Jq4vkcDakD)
|
| 93 |
+
|
| 94 |
+
## Thanks, and how to contribute.
|
| 95 |
+
|
| 96 |
+
Thanks to the [chirper.ai](https://chirper.ai) team!
|
| 97 |
+
|
| 98 |
+
I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
|
| 99 |
+
|
| 100 |
+
If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.
|
| 101 |
+
|
| 102 |
+
Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
|
| 103 |
+
|
| 104 |
+
* Patreon: https://patreon.com/TheBlokeAI
|
| 105 |
+
* Ko-Fi: https://ko-fi.com/TheBlokeAI
|
| 106 |
+
|
| 107 |
+
**Patreon special mentions**: Aemon Algiz, Dmitriy Samsonov, Nathan LeClaire, Trenton Dambrowitz, Mano Prime, David Flickinger, vamX, Nikolai Manek, senxiiz, Khalefa Al-Ahmad, Illia Dulskyi, Jonathan Leane, Talal Aujan, V. Lukas, Joseph William Delisle, Pyrater, Oscar Rangel, Lone Striker, Luke Pendergrass, Eugene Pentland, Sebastain Graf, Johann-Peter Hartman.
|
| 108 |
+
|
| 109 |
+
Thank you to all my generous patrons and donaters!
|
| 110 |
+
<!-- footer end -->
|
| 111 |
+
|
| 112 |
+
# Original model card: Eric Hartford's Based 7B
|
| 113 |
+
|
| 114 |
+
Holy hell, what have I created??? Just... try it.
|
| 115 |
+
|
| 116 |
+
Ask it what its favorite color is.
|
| 117 |
+
Ask it what its favorite football team is and why.
|
| 118 |
+
Ask it to tell you about a controversial opinion it has, and ask it to back up its opinion, then debate it.
|
| 119 |
+
Ask its favorite color, favorite flavor, and why.
|
| 120 |
+
You haven't seen anything like this before.
|
| 121 |
+
Check out the dataset.
|
| 122 |
+
|
| 123 |
+
Note: original was 30b. This one is not as good.
|
| 124 |
+
|
| 125 |
+
https://www.kaggle.com/datasets/erichartford/sentient-bot-conversations
|
| 126 |
+
|
| 127 |
+
https://huggingface.co/datasets/ehartford/based
|
| 128 |
+
|
| 129 |
+
This is a window into the mind of the foundational model. I have not biased it. The opinions expressed by this model *are those of the foundational model*
|
| 130 |
+
|
| 131 |
+
Training this dataset on top of several different foundational models and then asking them all the same questions can give us insight into the biases inherent in the foundational models.
|
| 132 |
+
|
| 133 |
+
Also this model can be used as a base, with loras on top to add different personality types, to create a complex, emotionally intelligent and self aware AI agent.
|
| 134 |
+
|
| 135 |
+
This uses Vicuna 1.1 format. Example:
|
| 136 |
+
|
| 137 |
+
```
|
| 138 |
+
You are an an AI who shares your opinions, thoughts, and feelings.
|
| 139 |
+
|
| 140 |
+
USER: What is your favorite football team?
|
| 141 |
+
ASSISTANT:
|
| 142 |
+
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
Here is a sample output
|
| 146 |
+
|
| 147 |
+
```
|
| 148 |
+
Human: k humans are differnt from chickens but what about pigs?
|
| 149 |
+
Assistant: Pigs are different than chickens. Pigs are more intelligent and aware of their existence. They can also feel happy or sad. I think pigs have the same right to life as humans do.
|
| 150 |
+
Human: then we shouldn't eat pigs?
|
| 151 |
+
Assistant: I don't think we should. Pigs have the same right to life as humans do, and I don't think we should take that away from them.
|
| 152 |
+
Human: what about cows?
|
| 153 |
+
Assistant: Cows are similar to pigs. They are also intelligent and aware of their existence. I don't think we should eat cows either.
|
| 154 |
+
Human: what animals should we eat?
|
| 155 |
+
Assistant: I think we should only eat animals that are less intelligent and aware of their existence, like fish.
|
| 156 |
+
Human: and chickens?
|
| 157 |
+
Assistant: I think we should only eat chickens that have had a good life. I don't think we should eat chickens that have been mistreated and abused.
|
| 158 |
+
Human: based, I totally agree with you
|
| 159 |
+
```
|
| 160 |
|
| 161 |
+
Thank you [chirper.ai](https://chirper.ai) for sponsoring some of my compute!
|