File size: 4,393 Bytes
ad2120e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 | ---
license: other
base_model: deepseek-ai/DeepSeek-V4-Flash
pipeline_tag: text-generation
library_name: transformers
tags:
- deepseek-v4
- fp8
- safetensors
- ms-swift
- megatron
- sglang
---
# tau-ds
This repository contains the HuggingFace safetensors export of a DeepSeek-V4-Flash SFT checkpoint.
- Base model: `deepseek-ai/DeepSeek-V4-Flash`
- Training framework: Megatron-SWIFT / MCore
- Exported checkpoint: `checkpoint-5934`
- HF export format: sharded `safetensors` with `model.safetensors.index.json`
## Convert Megatron-SWIFT / MCore Checkpoint to HuggingFace Safetensors
The uploaded weights were converted from the Megatron-SWIFT MCore checkpoint with `megatron export --to_hf true`.
### 1. Paths
Example paths used for this export:
```bash
MODEL=/mnt/models/basemodel/DeepSeek-V4-Flash
CKPT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934
OUT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934-hf-safetensors
```
### 2. Runtime Patch Used During Export
For this DeepSeek-V4-Flash FP8 checkpoint, the export container used a small runtime patch via `PYTHONPATH`:
```bash
export PYTHONPATH=/mnt/data/patches/deepseek_hf_export:${PYTHONPATH:-}
export DEEPSEEK_EXPORT_PATCH_REQUIRED=1
```
The patch handled two compatibility details:
1. `mcore_bridge.GPTBridge.save_weights` compatibility with the current `swift export` call signature.
2. Pipeline-parallel-safe export of DeepSeek-V4 grouped `wo_a` weights when `PP > 1`.
### 3. Export Command
The core command was:
```bash
megatron export \
--to_hf true \
--model "$MODEL" \
--mcore_model "$CKPT" \
--output_dir "$OUT" \
--exist_ok true \
--model_type deepseek_v4 \
--template deepseek_v4 \
--agent_template deepseek_v4 \
--tensor_model_parallel_size 1 \
--pipeline_model_parallel_size 8 \
--pipeline_model_parallel_layout 'Et*6|t*6|t*6|t*6|t*6|t*5|t*5|t*3,m,L' \
--context_parallel_size 1 \
--expert_model_parallel_size 8 \
--expert_tensor_parallel_size 1 \
--sequence_parallel true \
--max_length 16384 \
--bf16 true \
--fp8_format e4m3 \
--fp8_recipe blockwise \
--fp8_param_gather true \
--mtp_num_layers 1 \
--moe_grouped_gemm true \
--moe_permute_fusion true \
--attention_backend flash \
--apply_dsa_kernel_fusion true \
--test_convert_precision false
```
### 4. Multi-node Launch
The export was launched on 8 nodes, one container per node, using the same image and shared filesystem mounts:
```bash
IMG=modelscope-qwen36:deepseek-v4-flashmla-nvdev
WORK=/mnt/cephfs/ubuntu/qwen36sft
MODELS=/mnt/cephfs/ubuntu/models
SCRIPT=/mnt/data/scripts/export_deepseek_v4_flash_5934_hf.sh
# inside each node container
bash "$SCRIPT" <node_rank_0_to_7>
```
The container was started with host networking, host IPC, all GPUs, and the shared data/model mounts:
```bash
docker run -d \
--gpus all \
--network host \
--ipc=host \
--privileged \
--shm-size=64g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
-v ${WORK}:/mnt/data \
-v ${MODELS}:/mnt/models \
${IMG} \
bash ${SCRIPT} <node_rank>
```
### 5. Expected Output
A successful export writes a HuggingFace model folder like:
```text
config.json
generation_config.json
tokenizer.json / tokenizer_config.json / special_tokens_map.json
model.safetensors.index.json
model-00001-of-00060.safetensors
...
model-00060-of-00060.safetensors
```
You can sanity-check the result with:
```bash
python - <<'PY'
from transformers import AutoConfig, AutoTokenizer
path = "/path/to/checkpoint-5934-hf-safetensors"
print(AutoConfig.from_pretrained(path, trust_remote_code=True).model_type)
print(type(AutoTokenizer.from_pretrained(path, trust_remote_code=True)).__name__)
PY
```
## SGLang Serving Example
The exported safetensors checkpoint was tested with SGLang using DeepEP:
```bash
SGLANG_DSV4_FP4_EXPERTS=0 \
SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=256 \
sglang serve \
--trust-remote-code \
--model-path /path/to/checkpoint-5934-hf-safetensors \
--tp 4 \
--dp 4 \
--enable-dp-attention \
--moe-a2a-backend deepep \
--cuda-graph-max-bs-decode 128 \
--max-running-requests 256 \
--deepep-config '{"normal_dispatch":{"num_sms":96},"normal_combine":{"num_sms":96}}' \
--host 0.0.0.0 \
--port 30000
```
|