metadata
quantized_by: bobchenyx
base_model:
- deepseek-ai/DeepSeek-R1-0528
base_model_relation: quantized
license: mit
tags:
- deepseek_v3
- deepseek
- transformers
- GGUF
pipeline_tag: text-generation
Llamacpp Quantizations of DeepSeek-V3-0324 (MLA version)
This page is going to be deprecated. For other quantized versions, please refer to moxin-org/DeepSeek-R1-0528-Moxin-GGUF for more details.
Original model: Adopting BF16 & Imatrix from unsloth/DeepSeek-R1-0528-GGUF.
All quants made with modification of llama.cpp based on bartowski1182-llama.cpp.
Q2_K : 222.01 GiB (2.84 BPW)
Q4_K_M : 381.64 GiB (4.89 BPW)
Download Guide
# !pip install huggingface_hub hf_transfer
import os
os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"
from huggingface_hub import snapshot_download
snapshot_download(
repo_id = "bobchenyx/DeepSeek-R1-0528-MLA-GGUF",
local_dir = "bobchenyx/DeepSeek-R1-0528-MLA-GGUF",
allow_patterns = ["*Q2_K*"],
)