| | --- |
| | library_name: transformers |
| | tags: |
| | - GGUF |
| | base_model: FabienRoger/cot_5k |
| | --- |
| | |
| | ## FabienRoger/cot_5k - GGUF |
| | |
| | This repo contains GGUF format model files for [FabienRoger/cot_5k](https://huggingface.co/FabienRoger/cot_5k). |
| | |
| | they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d). |
| | |
| | ## Prompt template |
| | |
| | |
| | ``` |
| | <|system|> |
| | {system_prompt}<|endoftext|> |
| | <|user|> |
| | {prompt}<|endoftext|> |
| | <|assistant|> |
| | ``` |
| | |
| | ## Model file specification |
| | |
| | | Filename | Quant type | File Size | Description | |
| | | -------- | ---------- | --------- | ----------- | |
| | | [cot_5k-Q2_K.gguf](https://huggingface.co/tensorblock/cot_5k-GGUF/blob/main/cot_5k-Q2_K.gguf) | Q2_K | 0.646 GB | smallest, significant quality loss - not recommended for most purposes | |
| | |
| | |
| | ## Downloading instruction |
| | |
| | ### Command line |
| | |
| | Firstly, install Huggingface Client |
| | |
| | ```shell |
| | pip install -U "huggingface_hub[cli]" |
| | ``` |
| | |
| | Then, downoad the individual model file the a local directory |
| | |
| | ```shell |
| | huggingface-cli download tensorblock/cot_5k-GGUF --include "cot_5k-Q2_K.gguf" --local-dir MY_LOCAL_DIR |
| | ``` |
| | |
| | If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try: |
| | |
| | ```shell |
| | huggingface-cli download tensorblock/cot_5k-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf' |
| | ``` |
| | |
