README.md

---
license: other
base_model: deepseek-ai/DeepSeek-V4-Flash
pipeline_tag: text-generation
library_name: transformers
tags:
- deepseek-v4
- fp8
- safetensors
- ms-swift
- megatron
- sglang
---

# tau-ds

This repository contains the HuggingFace safetensors export of a DeepSeek-V4-Flash SFT checkpoint.

- Base model: `deepseek-ai/DeepSeek-V4-Flash`
- Training framework: Megatron-SWIFT / MCore
- Exported checkpoint: `checkpoint-5934`
- HF export format: sharded `safetensors` with `model.safetensors.index.json`

## Convert Megatron-SWIFT / MCore Checkpoint to HuggingFace Safetensors

The uploaded weights were converted from the Megatron-SWIFT MCore checkpoint with `megatron export --to_hf true`.

### 1. Paths

Example paths used for this export:

```bash
MODEL=/mnt/models/basemodel/DeepSeek-V4-Flash
CKPT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934
OUT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934-hf-safetensors
```

### 2. Runtime Patch Used During Export

For this DeepSeek-V4-Flash FP8 checkpoint, the export container used a small runtime patch via `PYTHONPATH`:

```bash
export PYTHONPATH=/mnt/data/patches/deepseek_hf_export:${PYTHONPATH:-}
export DEEPSEEK_EXPORT_PATCH_REQUIRED=1
```

The patch handled two compatibility details:

1. `mcore_bridge.GPTBridge.save_weights` compatibility with the current `swift export` call signature.
2. Pipeline-parallel-safe export of DeepSeek-V4 grouped `wo_a` weights when `PP > 1`.

### 3. Export Command

The core command was:

```bash
megatron export \
  --to_hf true \
  --model "$MODEL" \
  --mcore_model "$CKPT" \
  --output_dir "$OUT" \
  --exist_ok true \
  --model_type deepseek_v4 \
  --template deepseek_v4 \
  --agent_template deepseek_v4 \
  --tensor_model_parallel_size 1 \
  --pipeline_model_parallel_size 8 \
  --pipeline_model_parallel_layout 'Et*6|t*6|t*6|t*6|t*6|t*5|t*5|t*3,m,L' \
  --context_parallel_size 1 \
  --expert_model_parallel_size 8 \
  --expert_tensor_parallel_size 1 \
  --sequence_parallel true \
  --max_length 16384 \
  --bf16 true \
  --fp8_format e4m3 \
  --fp8_recipe blockwise \
  --fp8_param_gather true \
  --mtp_num_layers 1 \
  --moe_grouped_gemm true \
  --moe_permute_fusion true \
  --attention_backend flash \
  --apply_dsa_kernel_fusion true \
  --test_convert_precision false
```

### 4. Multi-node Launch

The export was launched on 8 nodes, one container per node, using the same image and shared filesystem mounts:

```bash
IMG=modelscope-qwen36:deepseek-v4-flashmla-nvdev
WORK=/mnt/cephfs/ubuntu/qwen36sft
MODELS=/mnt/cephfs/ubuntu/models
SCRIPT=/mnt/data/scripts/export_deepseek_v4_flash_5934_hf.sh

# inside each node container
bash "$SCRIPT" <node_rank_0_to_7>
```

The container was started with host networking, host IPC, all GPUs, and the shared data/model mounts:

```bash
docker run -d \
  --gpus all \
  --network host \
  --ipc=host \
  --privileged \
  --shm-size=64g \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  -v ${WORK}:/mnt/data \
  -v ${MODELS}:/mnt/models \
  ${IMG} \
  bash ${SCRIPT} <node_rank>
```

### 5. Expected Output

A successful export writes a HuggingFace model folder like:

```text
config.json
generation_config.json
tokenizer.json / tokenizer_config.json / special_tokens_map.json
model.safetensors.index.json
model-00001-of-00060.safetensors
...
model-00060-of-00060.safetensors
```

You can sanity-check the result with:

```bash
python - <<'PY'
from transformers import AutoConfig, AutoTokenizer
path = "/path/to/checkpoint-5934-hf-safetensors"
print(AutoConfig.from_pretrained(path, trust_remote_code=True).model_type)
print(type(AutoTokenizer.from_pretrained(path, trust_remote_code=True)).__name__)
PY
```

## SGLang Serving Example

The exported safetensors checkpoint was tested with SGLang using DeepEP:

```bash
SGLANG_DSV4_FP4_EXPERTS=0 \
SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=256 \
sglang serve \
  --trust-remote-code \
  --model-path /path/to/checkpoint-5934-hf-safetensors \
  --tp 4 \
  --dp 4 \
  --enable-dp-attention \
  --moe-a2a-backend deepep \
  --cuda-graph-max-bs-decode 128 \
  --max-running-requests 256 \
  --deepep-config '{"normal_dispatch":{"num_sms":96},"normal_combine":{"num_sms":96}}' \
  --host 0.0.0.0 \
  --port 30000
```