Instructions to use Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3
# Run inference directly in the terminal:
llama cli -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3
# Run inference directly in the terminal:
llama cli -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3
# Run inference directly in the terminal:
./llama-cli -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3

Use Docker

docker model run hf.co/Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3

LM Studio
Jan
Ollama
How to use Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 with Ollama:
```
ollama run hf.co/Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3
```

Unsloth Studio

How to use Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 to start chatting

Docker Model Runner
How to use Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 with Docker Model Runner:
```
docker model run hf.co/Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3
```

Lemonade

How to use Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Lucebox/DeepSeek-V4-Flash-0731-ROCmFP3

Run and chat with the model

lemonade run user.DeepSeek-V4-Flash-0731-ROCmFP3-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

DeepSeek-V4-Flash-0731-ROCmFP3 / README.md

davide221

Simplify model card around quality results

39745d3 verified 1 day ago

preview code

Raw

History Blame Contribute Delete

1.95 kB

	---
	license: mit
	base_model: deepseek-ai/DeepSeek-V4-Flash-0731
	tags: [gguf, rocmfpx, strix-halo, mixed-precision, quantization]
	---

	# DeepSeek-V4-Flash-0731 for Strix Halo

	82/92 quality in one 98.29 GB GGUF file.

	Built for 128 GB AMD Strix Halo systems. It loads unsplit on the Radeon 8060S
	iGPU and needs no sidecar files.

	## Quality

	\| Test \| Score \|
	\|---\|---:\|
	\| Full 92-question evaluation \| 82/92 \|
	\| COMPSEC-17 \| 17/17 \|

	The full score matches the published reference. This file averages 2.766 bits
	per model weight, about 4% less than the 2.88-bit reference.

	None of the 92 test questions were used while preparing this file. The
	published reference used 75 of them during its own preparation. Both results
	used the same grader and reasoning allowance.

	## Quality or speed

	\| Mode \| Options \| Decode speed \| COMPSEC-17 \|
	\|---\|---\|---:\|---:\|
	\| Quality (default) \| No extra flags \| 18.1 tok/s \| 17/17 \|
	\| Faster \| `--ds4-expert-top-k 4 --ds4-fused-decode` \| 22.3 tok/s \| 16/17 \|

	Quality mode is the recommended setting. Faster mode is 23% quicker but misses
	one additional COMPSEC question. The full 82/92 evaluation was run only in
	quality mode.

	The current DSpark helper model makes this file slower overall, so it is not
	recommended yet.

	## Download

	`DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf`

	- Size: 98,294,917,184 bytes
	- One file, no sidecars
	- SHA-256: `7c0789d190fdd2acad93255825822ca276f29d13f9410f2ac65f5f7a542b0a38`

	## Run

	```bash
	dflash_server DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf \
	--target-device hip:0 \
	--max-ctx 8192
	```

	Use the normal automatic memory settings on Strix Halo. If the machine also
	has a discrete GPU, expose only the iGPU with `HIP_VISIBLE_DEVICES`.

	Until support reaches the main dflash release, use the
	`feat/qtype106-down-surface` branch of `GeometricAGI/lucebox-hub`.

	Artifact and evaluation by Geometric-AI. Mirrored byte-for-byte by Lucebox.