README.md

metadata

title: DeepSeek-Math-V2 GGUF Converter
emoji: 🧮
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
app_port: 7860

DeepSeek-Math-V2 → GGUF Q8_0 Converter

Converts deepseek-ai/DeepSeek-Math-V2 from FP8 safetensors to a Q8_0 GGUF ready for use with llama.cpp and compatible runtimes.

Pipeline overview

HuggingFace FP8 safetensors
         │
         ▼  (1) huggingface-cli download
    /workspace/model-fp8/
         │
         ▼  (2) fp8_cast_bf16.py  (dequantise FP8 → BF16)
    /workspace/model-bf16/
         │
         ▼  (3) llama.cpp convert_hf_to_gguf.py  (--outtype bf16)
    model-bf16.gguf
         │
         ▼  (4) llama-quantize Q8_0
    deepseek-math-v2-q8_0.gguf  ← final output

Storage requirements

Stage	Approximate size
FP8 safetensors download	~380 GB
BF16 dequantised	~760 GB
GGUF BF16 (intermediate)	~760 GB
GGUF Q8_0 (final)	~380 GB
Peak concurrent disk	~1.5 TB

Recommended Space: HuggingFace ZeroGPU or a dedicated Persistent Storage Space with ≥2 TB disk and ≥800 GB RAM (or use NVMe swap).

Environment variables

Set these as Space Secrets in the HF Space settings:

Variable	Description	Default
HF_TOKEN	HuggingFace token (needed to access the model)	–
SKIP_DOWNLOAD	Set 1 if weights are already in /workspace/model-fp8	0
SKIP_FP8_CAST	Set 1 if BF16 weights are already in /workspace/model-bf16	0
SKIP_GGUF_CONVERT	Set 1 if model-bf16.gguf already exists	0
GGUF_Q8	Override output path	/workspace/output/deepseek-math-v2-q8_0.gguf

Running the conversion

The Space runs convert.sh automatically on start. Monitor logs in the HF Space console.

To run manually inside the container:

docker build -t dsmath-convert .
docker run --rm -it \
  -e HF_TOKEN=hf_xxx \
  -v /mnt/storage:/workspace \
  dsmath-convert \
  bash /workspace/scripts/convert.sh

After conversion

Upload the resulting GGUF to a new HF repo:

huggingface-cli upload your-username/DeepSeek-Math-V2-GGUF \
    /workspace/output/deepseek-math-v2-q8_0.gguf \
    deepseek-math-v2-q8_0.gguf

Or run locally with llama.cpp:

llama-cli \
  -m deepseek-math-v2-q8_0.gguf \
  -n 2048 \
  -p "Prove that there are infinitely many primes."

Files

.
├── Dockerfile                          # Build + runtime environment
├── README.md                           # This file (HF Spaces metadata header)
└── scripts/
    ├── convert.sh                      # Main orchestration script
    ├── fp8_cast_bf16_fallback.py       # FP8→BF16 dequantiser (fallback)
    └── entrypoint.sh                   # Docker entrypoint