|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- gguf |
|
|
- llama.cpp |
|
|
- quantized |
|
|
- deepseek |
|
|
- stheno |
|
|
--- |
|
|
|
|
|
# DeepSeek Sunfall Merged - GGUF Quantized Models |
|
|
|
|
|
This repository contains multiple **quantized GGUF variants** of the merged DeepSeek + Sunfall model, compatible with `llama.cpp`. |
|
|
|
|
|
## 🧠 Available Quantized Formats |
|
|
|
|
|
| Format | File Name | Description | |
|
|
|-------------|--------------------------------------------------|---------------------------------| |
|
|
| Q3_K_M | `deepseek_sunfall_merged_Model.Q3_K_M.gguf` | Smallest size, fastest inference | |
|
|
| Q4_K_M | `deepseek_sunfall_merged_Model.Q4_K_M.gguf` | Balanced speed & performance | |
|
|
| Q5_K_M | `deepseek_sunfall_merged_Model.Q5_K_M.gguf` | Better quality, slower | |
|
|
| Q6_K | `deepseek_sunfall_merged_Model.Q6_K.gguf` | Near full precision | |
|
|
| Q8_0 | `deepseek_sunfall_merged_Model.Q8_0.gguf` | Almost no compression loss | |
|
|
|
|
|
## 🔧 Usage (Python) |
|
|
|
|
|
Install `llama-cpp-python`: |
|
|
|
|
|
```bash |
|
|
pip install llama-cpp-python |
|
|
from llama_cpp import Llama |
|
|
|
|
|
model = Llama(model_path="deepseek_sunfall_merged_Model.Q4_K_M.gguf") # or Q3_K_M, etc. |
|
|
output = model("Tell me a story about stars.") |
|
|
print(output)``` |
