|
|
--- |
|
|
base_model: |
|
|
- charent/self_cognition_Alice |
|
|
--- |
|
|
# self_cognition_Alice - GGUF |
|
|
|
|
|
## About |
|
|
This repository contains GGUF weights for [charent/self_cognition_Alice](https://huggingface.co/charent/self_cognition_Alice). |
|
|
|
|
|
For a convenient overview and download list, visit our [model page](https://huggingface.co/sthaps/self_cognition_Alice). |
|
|
|
|
|
## Usage |
|
|
If you are unsure how to use GGUF files, refer to the [llama.cpp documentation](https://github.com/ggerganov/llama.cpp) for more details. |
|
|
|
|
|
### Llama.cpp CLI |
|
|
```bash |
|
|
./llama-cli -m self_cognition_Alice-q4_k_m.gguf -p "Hello!" |
|
|
``` |
|
|
|
|
|
## Provided Quants |
|
|
|
|
|
(sorted by size, not necessarily quality) |
|
|
|
|
|
| Link | Type | Size/GB | Notes | |
|
|
| :--- | :--- | :---: | :--- | |
|
|
| [GGUF](https://huggingface.co/sthaps/self_cognition_Alice/blob/main/self_cognition_Alice-q2_k.gguf) | q2_k | 0.28 | very low quality, for testing | |
|
|
| [GGUF](https://huggingface.co/sthaps/self_cognition_Alice/blob/main/self_cognition_Alice-q3_k_m.gguf) | q3_k_m | 0.32 | | |
|
|
| [GGUF](https://huggingface.co/sthaps/self_cognition_Alice/blob/main/self_cognition_Alice-q4_0.gguf) | q4_0 | 0.36 | | |
|
|
| [GGUF](https://huggingface.co/sthaps/self_cognition_Alice/blob/main/self_cognition_Alice-q4_k_m.gguf) | q4_k_m | 0.37 | recommended, good balance | |
|
|
| [GGUF](https://huggingface.co/sthaps/self_cognition_Alice/blob/main/self_cognition_Alice-q5_k_m.gguf) | q5_k_m | 0.41 | | |
|
|
| [GGUF](https://huggingface.co/sthaps/self_cognition_Alice/blob/main/self_cognition_Alice-q8_0.gguf) | q8_0 | 0.60 | near-full precision | |
|
|
|
|
|
## Thanks |
|
|
Special thanks to the `llama.cpp` team for their amazing work. |
