README.md

---
license: other
license_name: deepseek
license_link: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark/blob/main/LICENSE
base_model: deepseek-ai/DeepSeek-V4-Flash-DSpark
language:
  - en
  - zh
tags:
  - deepseek
  - deepseek-v4
  - moe
  - mxfp4
  - mla
  - speculative-decoding
  - dspark
  - amd-npu
  - xdna2
  - ryzen-ai
  - fst
pipeline_tag: text-generation
library_name: fst
---

# Model Card for DeepSeek-V4-Flash-DSpark-FST

This is the **FaStar `.fst` container** of DeepSeek V4 Flash (DSpark), a 284-billion-parameter
Mixture-of-Experts model. It is a **format/quantization conversion** of
[`deepseek-ai/DeepSeek-V4-Flash-DSpark`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark),
not a retrained or fine-tuned model. The `.fst` format packs the model for execution on the
**AMD Ryzen AI 9 365 XDNA2 NPU** via the [FaStar](https://github.com/RaffaelloMolinari/FaStar)
inference engine, which spills the ~150 GB of expert weights across **SSD → RAM → NPU scratch
buffers** using *expert virtual memory*.

## Model Details

### Model Description

- **Developed by:** Original model by [DeepSeek-AI](https://github.com/deepseek-ai); `.fst`
  conversion and FaStar inference engine by Raffaello Molinari.
- **Funded by [optional]:** Independent research/engineering project.
- **Shared by [optional]:** Raffaello Molinari.
- **Model type:** Decoder-only Transformer with Mixture-of-Experts (MoE) and Multi-head Latent
  Attention (MLA), MXFP4-quantized, with a DSpark draft model for speculative decoding.
- **Language(s) (NLP):** English, Chinese (DeepSeek V4 is multilingual; primarily en/zh).
- **License:** DeepSeek Model License (inherited from the upstream checkpoint — see `license_link`).
- **Finetuned from model [optional]:** Not fine-tuned. Converted (quantized + repackaged) from
  `deepseek-ai/DeepSeek-V4-Flash-DSpark`.

### Model Sources [optional]

- **Repository:** https://github.com/RaffaelloMolinari/FaStar
- **Paper [optional]:** N/A (engineering showcase; see the upstream DeepSeek V4 / DSpark
  materials).
- **Demo [optional]:** FaStar ships a built-in HTTP chat UI (`ds4_npu_engine --serve`); see the
  repository README.

## Uses

### Direct Use

On-device research inference of DeepSeek V4 Flash on an AMD Ryzen AI 9 365 laptop NPU, using the
FaStar engine. Suitable for studying expert virtual memory, on-NPU MLA/FFN via IRON-generated
MLIR-AIE kernels, and consumer-NPU execution of a 284B model. Generation modes: one-shot,
interactive multi-turn, and a streaming web chat UI.

### Downstream Use [optional]

As a reference format/checkpoint for porting other DeepSeek MoE models to the `.fst` container
and XDNA2 target, or as a substrate for experimenting with NPU kernel fusion, expert prefetch
strategies, and speculative-decoding acceptance tuning.

### Out-of-Scope Use

- **Not a production serving system.** Decode throughput is ~0.05 tokens/sec (see
  [Bias, Risks, and Limitations]); do not use for latency-sensitive or multi-user serving.
- **Not fine-tuned for any downstream task** and not safety-aligned — outputs are raw
  completions from the converted base model.
- **Not for deployment on non-XDNA2 hardware.** The `.fst` container, sidecars, and bundled
  `.xclbin` kernels target the AMD Ryzen AI NPU; they are not a drop-in replacement for the
  upstream HuggingFace checkpoint on CPU/GPU.
- Do not use to generate misleading, harmful, or disallowed content; the upstream DeepSeek
  acceptable-use terms apply.

## Bias, Risks, and Limitations

- **Throughput: ~0.05 tokens/sec.** Decode is NPU-compute-bound per dispatch. Dispatch-count
  reduction (op-replication packing) was proven correct on silicon but does not move tok/s; a
  packed N-copy blob does N× the NPU compute. Reaching >1 tok/s requires IRON-level kernel
  fusion (future work).
- **First-token latency.** The first token of a turn takes ~2 minutes (prefill of a 43-layer,
  284B model on this NPU); subsequent tokens stream at ~0.05 tok/s.
- **Numerical residual.** A recurrent hidden-state drift (~±40 RMS at deep layers) is intrinsic
  to the MXFP4 / MLA numerics of this checkpoint — it is reproduced in a pure-PyTorch HF
  ground-truth reference and is not a FaStar bug. Output remains coherent English with greedy
  prefill argmax matching the HF ground truth.
- **Memory footprint.** ~150 GB of expert weights; requires 64 GB RAM and ~200 GB NVMe SSD via
  expert paging. Expert cache thrash drops throughput if the working set exceeds RAM.
- **Inherited biases.** All biases, risks, and limitations of the upstream DeepSeek V4 Flash
  base model carry over unchanged — this is a lossy-format conversion, not an alignment
  intervention.

### Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations
inherited from the upstream DeepSeek model, the very low decode throughput, and the on-device
NPU-only target. Verify the DeepSeek license terms before redistribution or commercial use.

## How to Get Started with the Model

Requirements: AMD Ryzen AI 9 365 (XDNA2 NPU), 64 GB RAM, ~200 GB NVMe SSD, Ubuntu 24.04 with
XRT, AIEBU, and the AMDXDNA driver.

```bash
# Build the FaStar engine (CMake fetches header-only deps on first configure)
git clone https://github.com/RaffaelloMolinari/FaStar.git
cd FaStar
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

# XRT lives in /usr on Ubuntu, not /opt/xilinx
export XILINX_XRT=/usr

# One-shot generation (greedy, deterministic):
./build/ds4_npu_engine --model deepseek_v4_dspark.fst \
                       --prompt "Explain quantum computing" --tokens 128 --temp 0.0

# Speculative decoding with the DSpark draft model:
./build/ds4_npu_engine --model deepseek_v4_dspark.fst --draft_model dspark_draft.fst \
                       --prompt "Hello world" --tokens 256

# Web UI / chat server:
./build/ds4_npu_engine --model deepseek_v4_dspark.fst --serve --port 8080
```

This HuggingFace repo must contain: the `.fst` file, its `.fst.hc` / `.fst.norm` /
`.fst.tid2eid` sidecars, `tokenizer.json`, and (for speculative decoding) `dspark_draft.fst`.
The `.fst.norm` and `.fst.hc` sidecars are required for coherent output.

## Training Details

This model was **not trained** — it is a conversion of an existing checkpoint. Training details
below describe the *source* model and the conversion procedure.

### Training Data

Inherited from the upstream `deepseek-ai/DeepSeek-V4-Flash-DSpark` checkpoint. See that model
card for the original training data.

### Training Procedure

#### Preprocessing [optional]

Conversion to the `.fst` container (no weight fine-tuning):

```bash
python3 scripts/fst_converter.py --model deepseek-ai/DeepSeek-V4-Flash-DSpark \
                                 --output deepseek_v4_dspark.fst
```

The `.fst` format stores a page-aligned config header, shared tensors (attention, router, norms)
in Q8_0 / BF16, and expert blocks as dense DS4 MXFP4 (17-byte blocks: 1 e8m0 scale + 16 FP4
nibbles). Verify integrity with `scripts/verify_fst.py` and `scripts/check_fst.py`.

#### Training Hyperparameters

- **Training regime:** N/A — not trained. Inference weights are MXFP4 (experts) + Q8_0/BF16
  (shared tensors), dequantized to BF16 on-NPU for compute.

#### Speeds, Sizes, Times [optional]

- **Parameters:** ~284B (MoE).
- **Layers:** 43.
- **Experts:** 1376 total (~32 per layer) + 1 shared expert per layer, routed with top-k = 6
  and hash routing over the first 3 layers (route_scale = 1.5).
- **Hidden size:** 4096; expert intermediate: 2048; vocabulary: 129,280.
- **Attention:** MLA — Q latent 1024, KV latent 512.
- **Decode throughput:** ~0.05 tokens/sec on Ryzen AI 9 365 NPU.
- **Prefill latency:** ~2 minutes for the first token of a turn.
- **Container size:** ~150 GB on disk (paged across SSD/RAM/NPU).

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

Numerical correctness was verified against a pure-PyTorch HuggingFace ground-truth reference
(`hf_bos_ref.py`, `hf_gen_ref.py`) using the real DeepSeek weights, and qualitative coherence
was checked with free-form prompts (e.g. *"The importance of NPU…"*).

#### Factors

Per-layer cosine similarity vs. the HF reference (MLA projections, FFN GEMM, dequantization);
greedy prefill argmax token match; multi-token speculative-decoding acceptance rate; output
coherence over a 43-layer prefill.

#### Metrics

- GEMM / dequant cosine similarity vs. reference: ~0.998 (MLA), ~0.998 (FFN),
  ~0.999998 (fused FFN gate, deterministic).
- Greedy prefill argmax: matches HF ground truth (e.g. argmax 671 = "The").
- Speculative-decoding acceptance: ~0.5–0.67 depending on configuration.

### Results

The engine produces coherent English, and greedy prefill argmax matches the HF ground truth.
A deep-layer hidden-state residual (~±40 RMS at layer 42) is reproduced identically in the
pure-PyTorch HF reference and is intrinsic to this checkpoint's MXFP4/MLA numerics, not a
FaStar defect.

#### Summary

FaStar is mathematically faithful to the HuggingFace reference for this checkpoint, at the cost
of very low on-NPU throughput (~0.05 tok/s). Correctness holds; throughput is the open problem.

## Model Examination [optional]

Per-layer cosine audits of MLA (`wq_a`, `wkv_a`, `wo_a`, output projection), FFN
(gate/up/down GEMM), and MXFP4 dequantization are built into the repository
(`scripts/fst_layer_benchmark.py`, `scripts/verify_fst_weights.py`,
`scripts/verify_fused_ffn.py`, plus `tools/` probes). These localize divergence to the
intrinsic deep-layer residual noted above.

## Environmental Impact

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute)
presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). This is an
**inference-only** artifact; the dominant carbon cost was paid upstream during DeepSeek's
original training and is not attributable to this conversion.

- **Hardware Type:** AMD Ryzen AI 9 365 laptop (XDNA2 NPU), 64 GB RAM, NVMe SSD.
- **Hours used:** Conversion + on-device verification only (single laptop, no cluster training).
- **Cloud Provider:** None — entirely on-device.
- **Compute Region:** Local consumer hardware.
- **Carbon Emitted:** Negligible for this artifact (inference/conversion on one laptop);
  upstream training emissions belong to DeepSeek-AI.

## Technical Specifications [optional]

### Model Architecture and Objective

Decoder-only MoE Transformer with Multi-head Latent Attention (Q latent 1024, KV latent 512),
per-layer shared + routed experts (top-k = 6, hash-routed over the first 3 layers with
route_scale = 1.5, swiglu_limit = 10), MXFP4 expert weights with e8m0 scales, Q8_0/BF16 shared
tensors, and a V4 KV-compressor that streams 512-dim compressed KV rows. A small DSpark draft
model enables speculative decoding. Objective: next-token text generation.

### Compute Infrastructure

#### Hardware

- APU: AMD Ryzen AI 9 365 (XDNA2 NPU is the compute target; iGPU unused).
- RAM: 64 GB minimum (~54 GB peak RSS observed).
- Storage: NVMe SSD, ~200 GB free (experts live on SSD; read latency dominates miss cost).
- GPU: not required.

#### Software

- Ubuntu 24.04; XRT (Xilinx Runtime) at `/usr`; AIEBU assembler; AMDXDNA kernel driver.
- FaStar engine (`ds4_npu_engine`, C++17, CMake build).
- NPU kernels generated with AMD IRON / MLIR-AIE; prebuilt `.xclbin` + `_insts.bin` ship in the
  repo `kernels/` directory.
- Python 3 + `tokenizers` for the HF BPE tokenizer bridge.

## Citation [optional]

If you use this work, please cite the upstream DeepSeek model and the FaStar engine.

**BibTeX:**

```bibtex
@misc{deepseek-v4-flash-dspark,
  author       = {{DeepSeek-AI}},
  title        = {DeepSeek V4 Flash (DSpark)},
  howpublished = {\url{https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark}}
}

@misc{fastar,
  author       = {Raffaello Molinari},
  title        = {FaStar: Expert Virtual Memory Inference Engine for DeepSeek V4 Flash on AMD Ryzen AI NPU},
  howpublished = {\url{https://github.com/RaffaelloMolinari/FaStar}}
}
```

**APA:**

DeepSeek-AI. (n.d.). *DeepSeek V4 Flash (DSpark)*. HuggingFace. https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark

Molinari, R. (n.d.). *FaStar: Expert virtual memory inference engine for DeepSeek V4 Flash on AMD Ryzen AI NPU*. GitHub. https://github.com/RaffaelloMolinari/FaStar

## Glossary [optional]

- **`.fst`:** FaStar's page-aligned model container (config header + shared tensors in
  Q8_0/BF16 + dense DS4 MXFP4 expert blocks).
- **Expert virtual memory:** MoE experts treated as virtual-memory pages — stored on SSD,
  cached in RAM (`ExpertPager` LRU), uploaded to NPU scratch on demand.
- **MXFP4:** Microscaling FP4 — 16 FP4 nibbles per block plus one shared e8m0 scale (17 bytes
  per expert block).
- **MLA:** Multi-head Latent Attention — compresses Q (1024-d) and KV (512-d) into latent
  projections; the KV cache stores compressed latents + positional embeddings.
- **DSpark:** Draft-model speculative decoding — a small model proposes tokens that the main
  model verifies in a block.
- **IRON / MLIR-AIE:** AMD's compiler toolchain that generates the NPU kernels (`.xclbin`).
- **hw_context:** An AMDXDNA device context bound to an xclbin; the driver caps simultaneous
  contexts at 9 (managed by `AiebuKernelCache`).

## More Information [optional]

See the [FaStar repository README](https://github.com/RaffaelloMolinari/FaStar) for the full
architecture overview, CLI flags, web-UI API, kernel rebuild instructions, and known
limitations.

## Model Card Authors [optional]

Raffaello Molinari (`.fst` conversion + model card). Model weights and architecture by
DeepSeek-AI.

## Model Card Contact

Open an issue on the [FaStar GitHub repository](https://github.com/RaffaelloMolinari/FaStar).
````