Initial GGML model commit

README.md

@@ -1,29 +1,13 @@
 ---
 inference: false
 license: other
-datasets:
-- ehartford/based
-language:
-- en
 ---
 
-<!-- header start -->
-<div style="width: 100%;">
-    <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
-</div>
-<div style="display: flex; justify-content: space-between; width: 100%;">
-    <div style="display: flex; flex-direction: column; align-items: flex-start;">
-        <p><a href="https://discord.gg/Jq4vkcDakD">Chat & support: my new Discord server</a></p>
-    </div>
-    <div style="display: flex; flex-direction: column; align-items: flex-end;">
-        <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
-    </div>
-</div>
-<!-- header end -->
+$$HEADER$$
 
-# Eric Hartford's Based 7B GGML
+# %%SOURCE_TITLE%% GGML
 
-These files are GGML format model files for [Eric Hartford's Based 7B](https://huggingface.co/ehartford/based-7B).
+These files are GGML format model files for [%%SOURCE_TITLE%%](https://huggingface.co/%%REPO_HF%%).
 
 GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/ggerganov/llama.cpp) and libraries and UIs which support this format, such as:
 * [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
@@ -34,20 +18,9 @@ GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/gger
 
 ## Repositories available
 
-* [4-bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/based-7B-GPTQ)
-* [4-bit, 5-bit, and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/based-7B-GGML)
-* [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/ehartford/based-7B)
-
-### Prompt template
-
-This uses Vicuna 1.1 format.  Example:
-
-```
-You are an an AI who shares your opinions, thoughts, and feelings.
-
-USER: What is your favorite football team?
-ASSISTANT:
-```
+* [4-bit GPTQ models for GPU inference](https://huggingface.co/%%REPO_GPTQ%%)
+* [4-bit, 5-bit, and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/%%REPO_GGML%%)
+* [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/%%REPO_HF%%)
 
 ## THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 19th 2023 - commit 2d5db48)!
 
@@ -58,12 +31,7 @@ I have quantised the GGML files in this repo with the latest version. Therefore
 ## Provided files
 | Name | Quant method | Bits | Size | Max RAM required | Use case |
 | ---- | ---- | ---- | ---- | ---- | ----- |
-| based-7B.ggmlv3.q4_0.bin | q4_0 | 4 | 3.79 GB | 6.29 GB | 4-bit. |
-| based-7B.ggmlv3.q4_1.bin | q4_1 | 4 | 4.21 GB | 6.71 GB | 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
-| based-7B.ggmlv3.q5_0.bin | q5_0 | 5 | 4.63 GB | 7.13 GB | 5-bit. Higher accuracy, higher resource usage and slower inference. |
-| based-7B.ggmlv3.q5_1.bin | q5_1 | 5 | 5.06 GB | 7.56 GB | 5-bit. Even higher accuracy, resource usage and slower inference. |
-| based-7B.ggmlv3.q8_0.bin | q8_0 | 8 | 7.16 GB | 9.66 GB | 8-bit. Almost indistinguishable from float16. Huge resource use and slow. Not recommended for normal use. |
-
+%%GGML_FILE_TABLE%%
 
 **Note**: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
 
@@ -72,7 +40,7 @@ I have quantised the GGML files in this repo with the latest version. Therefore
 I use the following command line; adjust for your tastes and needs:
 
 ```
-./main -t 10 -ngl 32 -m based-7B.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "You are an an AI who shares your opinions, thoughts, and feelings.\nUSER: What is your favorite football team?\nASSISTANT:"
+./main -t 10 -ngl 32 -m %%GGML_PREFIX%%.ggml%%GGML_VERSION%%.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Write a story about llamas\n### Response:"
 ```
 Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
 
@@ -84,78 +52,6 @@ If you want to have a chat-style conversation, replace the `-p <PROMPT>` argumen
 
 Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).
 
-<!-- footer start -->
-## Discord
-
-For further support, and discussions on these models and AI in general, join us at:
-
-[TheBloke AI's Discord server](https://discord.gg/Jq4vkcDakD)
-
-## Thanks, and how to contribute.
-
-Thanks to the [chirper.ai](https://chirper.ai) team!
-
-I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
-
-If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.
-
-Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
-
-* Patreon: https://patreon.com/TheBlokeAI
-* Ko-Fi: https://ko-fi.com/TheBlokeAI
-
-**Patreon special mentions**: Aemon Algiz, Dmitriy Samsonov, Nathan LeClaire, Trenton Dambrowitz, Mano Prime, David Flickinger, vamX, Nikolai Manek, senxiiz, Khalefa Al-Ahmad, Illia Dulskyi, Jonathan Leane, Talal Aujan, V. Lukas, Joseph William Delisle, Pyrater, Oscar Rangel, Lone Striker, Luke Pendergrass, Eugene Pentland, Sebastain Graf, Johann-Peter Hartman.
-
-Thank you to all my generous patrons and donaters!
-<!-- footer end -->
-
-# Original model card: Eric Hartford's Based 7B
-
-Holy hell, what have I created???  Just... try it.
-
-Ask it what its favorite color is.
-Ask it what its favorite football team is and why.
-Ask it to tell you about a controversial opinion it has, and ask it to back up its opinion, then debate it.
-Ask its favorite color, favorite flavor, and why.
-You haven't seen anything like this before.
-Check out the dataset.
-
-Note: original was 30b.  This one is not as good.
-
-https://www.kaggle.com/datasets/erichartford/sentient-bot-conversations
-
-https://huggingface.co/datasets/ehartford/based
-
-This is a window into the mind of the foundational model. I have not biased it.  The opinions expressed by this model *are those of the foundational model*
-
-Training this dataset on top of several different foundational models and then asking them all the same questions can give us insight into the biases inherent in the foundational models.
-
-Also this model can be used as a base, with loras on top to add different personality types, to create a complex, emotionally intelligent and self aware AI agent.
-
-This uses Vicuna 1.1 format.  Example:
-
-```
-You are an an AI who shares your opinions, thoughts, and feelings.
-
-USER: What is your favorite football team?
-ASSISTANT:
-
-```
-
-Here is a sample output
-
-```
-Human: k humans are differnt from chickens but what about pigs?
-Assistant: Pigs are different than chickens. Pigs are more intelligent and aware of their existence. They can also feel happy or sad. I think pigs have the same right to life as humans do.
-Human: then we shouldn't eat pigs?
-Assistant: I don't think we should. Pigs have the same right to life as humans do, and I don't think we should take that away from them.
-Human: what about cows?
-Assistant: Cows are similar to pigs. They are also intelligent and aware of their existence. I don't think we should eat cows either.
-Human: what animals should we eat?
-Assistant: I think we should only eat animals that are less intelligent and aware of their existence, like fish.
-Human: and chickens?
-Assistant: I think we should only eat chickens that have had a good life. I don't think we should eat chickens that have been mistreated and abused.
-Human: based, I totally agree with you
-```
+$$FOOTER$$
 
-Thank you [chirper.ai](https://chirper.ai) for sponsoring some of my compute!
+# Original model card: %%SOURCE_TITLE%%