README.md

---
title: DeepSeek-Math-V2 GGUF Converter
emoji: 🧮
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
app_port: 7860
---

# DeepSeek-Math-V2 → GGUF Q8_0 Converter

Converts [deepseek-ai/DeepSeek-Math-V2](https://huggingface.co/deepseek-ai/DeepSeek-Math-V2)
from FP8 safetensors to a Q8_0 GGUF ready for use with **llama.cpp** and compatible runtimes.

## Pipeline overview

```
HuggingFace FP8 safetensors
         │
         ▼  (1) huggingface-cli download
    /workspace/model-fp8/
         │
         ▼  (2) fp8_cast_bf16.py  (dequantise FP8 → BF16)
    /workspace/model-bf16/
         │
         ▼  (3) llama.cpp convert_hf_to_gguf.py  (--outtype bf16)
    model-bf16.gguf
         │
         ▼  (4) llama-quantize Q8_0
    deepseek-math-v2-q8_0.gguf  ← final output
```

## Storage requirements

| Stage | Approximate size |
|-------|----------------|
| FP8 safetensors download | ~380 GB |
| BF16 dequantised | ~760 GB |
| GGUF BF16 (intermediate) | ~760 GB |
| **GGUF Q8_0 (final)** | **~380 GB** |
| Peak concurrent disk | ~1.5 TB |

> **Recommended Space**: HuggingFace ZeroGPU or a dedicated Persistent Storage Space with ≥2 TB disk and ≥800 GB RAM (or use NVMe swap).

## Environment variables

Set these as **Space Secrets** in the HF Space settings:

| Variable | Description | Default |
|----------|-------------|---------|
| `HF_TOKEN` | HuggingFace token (needed to access the model) | – |
| `SKIP_DOWNLOAD` | Set `1` if weights are already in `/workspace/model-fp8` | `0` |
| `SKIP_FP8_CAST` | Set `1` if BF16 weights are already in `/workspace/model-bf16` | `0` |
| `SKIP_GGUF_CONVERT` | Set `1` if `model-bf16.gguf` already exists | `0` |
| `GGUF_Q8` | Override output path | `/workspace/output/deepseek-math-v2-q8_0.gguf` |

## Running the conversion

The Space runs `convert.sh` automatically on start. Monitor logs in the HF Space console.

To run manually inside the container:

```bash
docker build -t dsmath-convert .
docker run --rm -it \
  -e HF_TOKEN=hf_xxx \
  -v /mnt/storage:/workspace \
  dsmath-convert \
  bash /workspace/scripts/convert.sh
```

## After conversion

Upload the resulting GGUF to a new HF repo:

```bash
huggingface-cli upload your-username/DeepSeek-Math-V2-GGUF \
    /workspace/output/deepseek-math-v2-q8_0.gguf \
    deepseek-math-v2-q8_0.gguf
```

Or run locally with llama.cpp:

```bash
llama-cli \
  -m deepseek-math-v2-q8_0.gguf \
  -n 2048 \
  -p "Prove that there are infinitely many primes."
```

## Files

```
.
├── Dockerfile                          # Build + runtime environment
├── README.md                           # This file (HF Spaces metadata header)
└── scripts/
    ├── convert.sh                      # Main orchestration script
    ├── fp8_cast_bf16_fallback.py       # FP8→BF16 dequantiser (fallback)
    └── entrypoint.sh                   # Docker entrypoint
```