Upload 3 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+WRT_Llama-3.1-2-8B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,59 @@
----
-license: mit
----
+---
+base_model: meta-llama/Llama-3.1-8B
+license: mit
+pipeline_tag: text-generation
+tags:
+- Llama-3
+- finetune
+quantized_by: boapro
+---
+
+## Llamacpp imatrix Quantizations of meta-llama/Llama-3.1-8B
+Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3878">b3878</a> for quantization.
+
+Original model: https://huggingface.co/meta-llama/Llama-3.1-8B
+
+
+Run it in [LM Studio](https://lmstudio.ai/)
+
+## Prompt format
+
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+
+{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
+
+{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+```
+
+
+
+## Downloading using huggingface-cli
+
+First, make sure you have hugginface-cli installed:
+
+```
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, you can target the specific file you want:
+
+
+
+If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
+
+
+You can either specify a new local-dir (boapro/WRT_II) or download them all in place (./)
+
+## Q4_0_X_X
+
+
+If you're using an ARM chip, the Q4_0_X_X quants will have a substantial speedup. Check out Q4_0_4_4 speed comparisons [on the original pull request](https://github.com/ggerganov/llama.cpp/pull/5780#pullrequestreview-21657544660)
+
+To check which one would work best for your ARM chip, you can check [AArch64 SoC features](https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html) (thanks EloyOn!).
+
+
+If you want to get more into the weeds, you can check out this extremely useful feature chart:
+
+[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
+

WRT_Llama-3.1-2-8B-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f782e8204a53d13a8e365deefea10dc2e29a27cd209034a5066d3c0c45ee1f60
+size 4692669920

gitattributes

@@ -0,0 +1,60 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B-f16.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3.1-WhiteRabbitNeo-2-8B.imatrix filter=lfs diff=lfs merge=lfs -text