Instructions to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Use Docker

docker model run hf.co/ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Ollama
How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with Ollama:
```
ollama run hf.co/ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M
```
Unsloth Desktop

How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with Docker Model Runner:
```
docker model run hf.co/ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M
```

Lemonade

How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.JackOD-9B-Coder-MTP-DFlash-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

JackOD-9B-Coder — GGUF (DFlash)

ManniX-ITA/JackOD-9B-Coder v3 quantized for DFlash speculative decoding: every quant tier of JackOD-9B-Coder-MTP-GGUF with the built-in MTP (NextN) layer removed, plus a separate DFlash drafter that does the drafting instead.

Use this repo if your runtime drafts with DFlash. Use JackOD-9B-Coder-MTP-GGUF if it drafts with the model's own MTP head, or if you do not use speculative decoding at all. The two repos hold the same quantized weights; only the speculative-decoding path differs.

JackOD-9B-Coder is a merge over Qwen/Qwen3.5-9B built for multi-turn agentic coding — DeltaCoder-9B 0.55 / Ornith-1.5-9B 0.30 / Coder-Flash 0.15, omnimerge_v2. The recipe, the evaluations and the full description are on the model card.

What is in this repo

27 quant tiers, JackOD-9B-Coder-MTP-DFlash-<TIER>.gguf. Each is the v3 -MTP-GGUF file of the same tier with the NextN block stripped (omnimergekit/scripts/gguf_strip_nextn.py): 442 → 427 tensors, block_count 33 → 32, nextn_predict_layers removed, the per-layer arrays truncated to 32. Every kept tensor was verified byte-identical to the -MTP-GGUF file it came from (--verify), so the quantized weights — and the v3 imatrix they were calibrated with, sha256 feb286c42be30760e92fbb218dfc1ceb4be07a1cb01634993072d1f2ecc9dd80 — are exactly those of the MTP repo. Every file has a .sha256 sidecar.
The drafter, Qwen3.5-9B-DFlash.Q8_0.gguf (1,383,768,224 bytes), redistributed unmodified from AtomicChat/Qwen3.5-9B-DFlash-GGUF at c7c69923d77a9aa0004b410d3a7a3dbb64591d41 (apache-2.0; base z-lab/Qwen3.5-9B-DFlash), sha256 27b9d18e605aea9c50ef506e4b63921e0dc624cc91c63c60617a8295f153d436 — the provenance line is in Qwen3.5-9B-DFlash.Q8_0.gguf.verified. It is a drafter for the Qwen3.5-9B family; JackOD shares that base's tokenizer and vocabulary.

Runtime support

runtime	DFlash drafting
llama.cpp	yes — upstream since `b9831` ("spec : add DFlash support", #22105)
opencoti-llamafile	yes
xollama	yes
stock Ollama	no — its bundled engine has no `draft-dflash`. Use the `-MTP-GGUF` files there.

Sampler

temperature 0.4 · top_p 0.95 · top_k 20 · min_p 0 · repeat_penalty 1.002 · repeat_last_n 112 · presence_penalty 1.05

The seed is not fixed. Worth trying: seed 42. In the author's testing it behaves better — no loops, and stellar on some tasks — but it is not reliable: it still fails on others. Treat it as something to test, not as part of the recommended setup.

Running it (llama.cpp)

llama-server -m JackOD-9B-Coder-MTP-DFlash-Q4_K_M.gguf \
  --spec-type draft-dflash -md Qwen3.5-9B-DFlash.Q8_0.gguf --spec-draft-n-max 4 \
  --jinja -c 32768 -ngl 99 \
  --temp 0.4 --top-p 0.95 --top-k 20 --min-p 0 \
  --repeat-penalty 1.002 --repeat-last-n 112 --presence-penalty 1.05

--jinja is mandatory for tool calling. Without it the tool grammar never comes from the model's chat template and every tool scenario degrades to prose.

Draft acceptance

Measured on every tier with llama.cpp 311d4211bf, --spec-type draft-dflash --spec-draft-n-max 4, on a 192-token and (for three tiers) a 768-token generation. These are greedy (temperature 0) measurements, taken before the sampler above was settled: acceptance under temperature 0.4 will be lower, and these numbers are a comparison between tiers, not a speedup promise. Draft acceptance depends on the content being generated.

tier	size	accept rate (192 tok)	mean accept len	accept rate (768 tok)	mean accept len
`Q8_0`	9.53 GB	95.0%	4.78	81.5%	4.26
`Q6_K_L`	7.85 GB	93.8%	4.75	—	—
`Q6_K`	7.36 GB	93.8%	4.75	80.6%	4.21
`CD-Q6_K`	7.33 GB	94.4%	4.78	—	—
`Q5_K_L`	7.10 GB	90.9%	4.63	—	—
`Q5_K_M`	6.47 GB	90.9%	4.63	—	—
`CD-Q5_K_M`	6.45 GB	91.5%	4.66	—	—
`Q4_K_L`	6.38 GB	90.9%	4.63	—	—
`Q5_K_S`	6.31 GB	91.5%	4.66	—	—
`Q4_K_M`	5.63 GB	92.6%	4.66	75.4%	3.99
`CD-Q4_K_M`	5.61 GB	91.5%	4.66	—	—
`Q3_K_XL`	5.51 GB	93.8%	4.75	—	—
`IQ4_NL`	5.42 GB	90.3%	4.55	—	—
`Q4_K_S`	5.35 GB	88.1%	4.52	—	—
`IQ4_XS`	5.20 GB	91.5%	4.66	—	—
`Q3_K_L`	4.93 GB	93.8%	4.75	—	—
`CD-Q3_K_L`	4.92 GB	93.8%	4.75	—	—
`Q2_K_L`	4.82 GB	84.0%	4.34	—	—
`Q3_K_M`	4.62 GB	93.8%	4.75	—	—
`IQ3_M`	4.42 GB	92.6%	4.66	—	—
`Q3_K_S`	4.26 GB	75.3%	3.98	—	—
`IQ3_XS`	4.24 GB	92.6%	4.66	—	—
`IQ3_XXS`	3.94 GB	92.0%	4.66	—	—
`CD-Q2_K`	3.81 GB	92.6%	4.66	—	—
`IQ2_M`	3.61 GB	86.0%	4.42	—	—
`IQ2_S`	3.43 GB	86.6%	4.44	—	—
`IQ2_XS`	3.29 GB	84.0%	4.34	—	—

Q3_K_S is the outlier (75.3% on the short run); every other tier accepts 84–95% of the drafted tokens on the short run.

Quants

The tiers, their sizes relative to the MTP files (each is 0.15–0.20 GB smaller: the NextN block is gone), the KLD-vs-F16 measurements and the ContribDynamic (CD-) bit-width maps are described on the -MTP-GGUF card. There is no F16 here; use the MTP repo's F16.

Verifying a download

sha256sum -c JackOD-9B-Coder-MTP-DFlash-Q4_K_M.gguf.sha256
sha256sum Qwen3.5-9B-DFlash.Q8_0.gguf   # 27b9d18e605aea9c50ef506e4b63921e0dc624cc91c63c60617a8295f153d436

Downloads last month: 11,790

GGUF

Model size

9B params

Architecture

qwen35

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF

Base model

Qwen/Qwen3.5-9B-Base

Finetuned

Qwen/Qwen3.5-9B

Finetuned

ManniX-ITA/JackOD-9B-Coder

Quantized

(4)

this model