Spaces:
Sleeping
Sleeping
File size: 2,919 Bytes
ee67c0c c41af55 ee67c0c c41af55 ee67c0c c41af55 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | ---
title: DeepSeek-Math-V2 GGUF Converter
emoji: 🧮
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
app_port: 7860
---
# DeepSeek-Math-V2 → GGUF Q8_0 Converter
Converts [deepseek-ai/DeepSeek-Math-V2](https://huggingface.co/deepseek-ai/DeepSeek-Math-V2)
from FP8 safetensors to a Q8_0 GGUF ready for use with **llama.cpp** and compatible runtimes.
## Pipeline overview
```
HuggingFace FP8 safetensors
│
▼ (1) huggingface-cli download
/workspace/model-fp8/
│
▼ (2) fp8_cast_bf16.py (dequantise FP8 → BF16)
/workspace/model-bf16/
│
▼ (3) llama.cpp convert_hf_to_gguf.py (--outtype bf16)
model-bf16.gguf
│
▼ (4) llama-quantize Q8_0
deepseek-math-v2-q8_0.gguf ← final output
```
## Storage requirements
| Stage | Approximate size |
|-------|----------------|
| FP8 safetensors download | ~380 GB |
| BF16 dequantised | ~760 GB |
| GGUF BF16 (intermediate) | ~760 GB |
| **GGUF Q8_0 (final)** | **~380 GB** |
| Peak concurrent disk | ~1.5 TB |
> **Recommended Space**: HuggingFace ZeroGPU or a dedicated Persistent Storage Space with ≥2 TB disk and ≥800 GB RAM (or use NVMe swap).
## Environment variables
Set these as **Space Secrets** in the HF Space settings:
| Variable | Description | Default |
|----------|-------------|---------|
| `HF_TOKEN` | HuggingFace token (needed to access the model) | – |
| `SKIP_DOWNLOAD` | Set `1` if weights are already in `/workspace/model-fp8` | `0` |
| `SKIP_FP8_CAST` | Set `1` if BF16 weights are already in `/workspace/model-bf16` | `0` |
| `SKIP_GGUF_CONVERT` | Set `1` if `model-bf16.gguf` already exists | `0` |
| `GGUF_Q8` | Override output path | `/workspace/output/deepseek-math-v2-q8_0.gguf` |
## Running the conversion
The Space runs `convert.sh` automatically on start. Monitor logs in the HF Space console.
To run manually inside the container:
```bash
docker build -t dsmath-convert .
docker run --rm -it \
-e HF_TOKEN=hf_xxx \
-v /mnt/storage:/workspace \
dsmath-convert \
bash /workspace/scripts/convert.sh
```
## After conversion
Upload the resulting GGUF to a new HF repo:
```bash
huggingface-cli upload your-username/DeepSeek-Math-V2-GGUF \
/workspace/output/deepseek-math-v2-q8_0.gguf \
deepseek-math-v2-q8_0.gguf
```
Or run locally with llama.cpp:
```bash
llama-cli \
-m deepseek-math-v2-q8_0.gguf \
-n 2048 \
-p "Prove that there are infinitely many primes."
```
## Files
```
.
├── Dockerfile # Build + runtime environment
├── README.md # This file (HF Spaces metadata header)
└── scripts/
├── convert.sh # Main orchestration script
├── fp8_cast_bf16_fallback.py # FP8→BF16 dequantiser (fallback)
└── entrypoint.sh # Docker entrypoint
```
|