Upload README.md with huggingface_hub

b497cd9 verified 11 days ago

13.8 kB

---
language:
  - en
  - zh
base_model: openpangu/openPangu-R-7B-Diffusion
tags:
  - gguf
  - llama.cpp
  - diffusion
  - block-diffusion
  - openPangu
license: other
---
# openPangu-R-7B-Diffusion — Q4_K_M GGUF (llama.cpp)

Quantized **Q4_K_M** GGUF of [`openpangu/openPangu-R-7B-Diffusion`](https://huggingface.co/openpangu/openPangu-R-7B-Diffusion),
together with a llama.cpp patch that adds native support for the model's
**context-causal block-diffusion** decoding.

> `openPangu-R-7B-Diffusion` is a 7B block-diffusion LLM. Generation is *not* plain
> next-token autoregression: a block of tokens (default length **32**) is denoised in
> parallel with **full (bi-directional) attention inside the block** and **causal
> attention for the preceding context**, revealing the most confident tokens first.
> The model can also be run in ordinary autoregressive mode.

Upstream llama.cpp (as of tag `b11719`) ships the `PanguEmbedded` architecture
(used by the autoregressive `openPangu-Embedded`) but does **not** implement the
diffusion decoding this model needs, hence the patch.

## Files

| File | Description |
|---|---|
| `pangu-diffusion-Q4_K_M.gguf` | Q4_K_M quantized weights (~4.6 GB), ready to run |
| `pangu-diffusion-llamacpp.patch` | llama.cpp patch adding openPangu-R-Diffusion support |
| `build_llamacpp_pangu_diffusion.sh` | one-click script: clone llama.cpp → apply patch → build |

## Quick start

### 1. Build llama.cpp with the patch (one-click)

```bash
bash build_llamacpp_pangu_diffusion.sh
# binaries end up in ./pangu-diffusion-build/llama.cpp/build/bin/
```

Or manually:

```bash
git clone --depth 1 --branch b11719 https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git apply /path/to/pangu-diffusion-llamacpp.patch
cmake -B build -DGGML_NATIVE=ON -DLLAMA_CURL=OFF
cmake --build build -j --target llama-diffusion-cli llama-cli llama-quantize
```

### 2. Diffusion decoding (recommended)

`llama-diffusion-cli` applies the model's chat template automatically: pass the **raw
user message**, not a pre-formatted prompt. The rendered prompt is

```
[unused9]系统：<system>[unused10][unused9]用户：<user>[unused10][unused9]助手：
```

```bash
./build/bin/llama-diffusion-cli \
  -m pangu-diffusion-Q4_K_M.gguf \
  -p "Which is bigger, 9.11 or 9.9?" \
  -ub 256 \
  --diffusion-block-length 32 \
  --diffusion-small-block 4 \
  --diffusion-algorithm 1 \
  --temp 0 -t 16 --seed 42
```

Recommended sampling (from the official model card):
`alg = entropy`, `num_small_blocks = 8` → `--diffusion-small-block 4` (with block length 32),
`top_p = 1`, `temperature = 0`. For coding tasks the authors suggest
`alg = entropy, num_small_blocks = 32 (→ --diffusion-small-block 1), top_p = 0.8, temperature = 1`.

**Fast mode** (higher throughput): reveal every masked token whose confidence exceeds a
threshold in a single denoising step, with one small block per block:

```bash
./build/bin/llama-diffusion-cli \
  -m pangu-diffusion-Q4_K_M.gguf \
  -p "<question>" \
  -ub 256 --diffusion-block-length 32 --diffusion-small-block 32 \
  --diffusion-confidence-threshold 0.9 --diffusion-algorithm 4 \
  --temp 0 -t 16 --seed 42
```

`--diffusion-n-seq N` runs N sequences through the block decoder in a single
(micro-)batch (the prompt is replicated; see *Batched decoding* below).

`--diffusion-algorithm` values: `0` origin, `1` entropy, `2` margin, `3` random, `4` confidence.
`-ub N` is the maximum sequence length; the tail that does not fill a whole block is dropped.

### 3. Autoregressive decoding

```bash
./build/bin/llama-cli -m pangu-diffusion-Q4_K_M.gguf \
  -p "[unused9]系统：[unused10][unused9]用户：Hi![unused10][unused9]助手：" -n 128 -t 16
```

## Example

Prompt (translated): *“Which is bigger, 9.11 or 9.9?”* — command as in §2, fast mode
(`-ub 256`, `--diffusion-confidence-threshold 0.9`, Q4_K_M):

```
I need to figure out which is bigger between 9.11 and 9.9. Both are decimals, so I should compare them directly.
First, let's write them with the same number of decimal places to make it easier. 9.11 has two decimal places, and 9.9 has one. So, I can write 9.9 as 9.90 to make it two decimal places.
So, 9.11 vs 9.90.
Now, comparing the tenths place: both have 9 in the tenths place. So, that's the same.
Next, the hundredths place: 9.11 has 1 in the hundredths place, and 9.90 has 0 in the hundredths place. Since 1 is greater than 0, ...
I can think of them as fractions. 9.11 is 911/100, and 9.9 is 990/100.
Since the denominator is the same, I can compare the numerators: 911 vs 990.
911 is less than 990, so 911/100 is less than 990/100, which ...
```

For a short prompt the output is reproduced **token-for-token identically to the Hugging
Face reference** (see *Numerical parity* below), e.g. `-p "hello"` (f16 GGUF):
`[unused16]Okay, the user just said "hello". That's a simple greeting. I should respond
politely and warmly. Maybe they're just starting a conversation`.

## Speed when you actually use it (measured)

Test environment — the resources **actually usable inside the container (docker)**:

| Resource | Value |
|---|---|
| CPU | Intel Xeon Platinum 8559C |
| Effective cores | **16 vCPU** (cgroup `cpu.max` = `1600000 100000`) |
| Memory (cgroup limit) | **≈ 97 GiB** |
| GPU | **none usable** — `/dev/nvidia*` nodes exist but `cuInit()` returns `CUDA_ERROR_NO_DEVICE` and NVML init fails |
| Build / runtime | CPU-only, `-t 16` |

These are **end-user speeds** measured from real runs of the CLI on
`pangu-diffusion-Q4_K_M.gguf` (4.59 GiB, 8.03 B params), i.e. what you actually
experience while using it:

| What the user experiences | Mode | Speed |
|---|---|---|
| **Input** — waiting for the prompt to be read before output starts | both | **≈ 78 t/s** short prompt; **≈ 132 t/s** for longer prompts (`llama-bench` pp512) |
| **Output** — text generated (autoregressive, streamed live) | `llama-cli` | **11.9 t/s** (llama-bench tg128: 11.1 t/s) |
| **Output** — block diffusion (quality: block 32 / small-block 4, `alg=entropy`, `temp=0`) | `llama-diffusion-cli` | **≈ 4.4 t/s** |
| **Output** — block diffusion (fast: `--diffusion-confidence-threshold 0.9`, small-block 32, `alg=confidence` — the official throughput setting) | `llama-diffusion-cli` | **≈ 5 t/s** |

Notes:
- Input (prompt processing) is the same causal prefill for both paths; it is what you wait
  for before the answer appears.
- The AR output speed (≈ 12 t/s) is what `llama-cli` prints as `Generation: X t/s`; the
  diffused text is only flushed at the end of a run (unless `--diffusion-visual`).
- Block diffusion normally reveals one token per denoising step, i.e. up to `block_length`
  (= 32) forward passes per block. With `--diffusion-confidence-threshold T` every masked
  token whose confidence exceeds `T` is revealed in the same step (`transfer = argmax(conf)
  | (conf > T)`), which raises throughput — the same mechanism the authors report as giving
  up to 2.5× over autoregressive decoding on Ascend NPUs. A lower `T` is faster but less
  accurate; measured on this CPU (235 tokens, block 32 / small-block 32, `alg=confidence`,
  `temp=0`):

  | `--diffusion-confidence-threshold` | speed |
  |---|---|
  | `0.9` (default) | 4.9 t/s |
  | `0.7` | 7.4 t/s |
  | `0.5` | 8.7 t/s |

- The AR figures are the best reference for raw model throughput on this hardware.

## What the patch does

1. **`src/llama-arch.cpp`** — mark `LLM_ARCH_PANGU_EMBED` as a diffusion architecture
   (`llm_arch_is_diffusion()`), so `llama-diffusion-cli` accepts the model.
2. **`examples/diffusion/diffusion.{h,cpp}`** — add a *context-causal block-diffusion*
   decoder (`diffusion.block_attention`):
   - prefill the prompt with a causal forward (kept in the KV cache);
   - denoise each block with `causal_attn = false` → full attention over the cached
     context **plus** the current block (the KV-cache kq mask keeps every non-empty
     cached cell when non-causal);
   - re-decode the block each step, dropping its stale keys (`llama_memory_seq_rm`),
     reveal the most-confident masked token of the current small block, left to right;
   - commit the block with a causal forward so the KV cache holds correct causal
     context, then sample the first token of the next block from the last position;
   - `diffusion_generate_nseq()` generalises this to `n_seq` sequences sharing one
     micro-batch (see `--diffusion-n-seq`).
3. **`examples/diffusion/diffusion-cli.cpp`** — read `diffusion.block_attention`, add
   `--diffusion-small-block/-confidence-threshold/-n-seq`, size the context for the
   batched prefill, and use greedy sampling when `temperature == 0`.
4. **`gguf-py` + `conversion/pangu.py`** — write the new `diffusion.block_attention`
   metadata key for diffusion checkpoints (alongside `diffusion.shift_logits` and
   `tokenizer.ggml.mask_token_id`), and set `tokenizer.ggml.add_space_prefix = false`
   (see *Tokenizer fix* below).
### Model metadata in the GGUF

```
general.architecture            = pangu-embedded
diffusion.shift_logits          = true
diffusion.block_attention       = true
tokenizer.ggml.mask_token_id    = 45830
tokenizer.ggml.add_space_prefix = false
```

### Tokenizer fix

The upstream conversion honours `add_prefix_space` from `tokenizer_config.json`, but the
slow `PanguTokenizer` (sentencepiece, `legacy: true`) does **not** actually prepend a space.
With the flag left on, llama.cpp inserted spurious space tokens after every special token
(`[unused9]系统：…` became `[unused9]␠系统：…`), which changed the input and the generation.
The patch forces `tokenizer.ggml.add_space_prefix = false`; the tokenizer output then
matches the reference exactly (a 22-token prompt becomes 22 tokens, was 37).

### Numerical parity

Validated against the Hugging Face reference (`inference/generation_utils.py`,
`alg=confidence`, `num_small_blocks=1`, `threshold=0.9`, `temperature=0`) on the **f16**
GGUF: for `-p "hello"` the generated token IDs are **identical, token for token**, to the
reference. (The earlier mismatch was entirely the tokenizer issue above, not the
decoder.)

## Batched decoding (`--diffusion-n-seq`)

`--diffusion-n-seq N` runs N independent sequences through the block decoder inside a
single (micro-)batch. The example CLI replicates the prompt, so this is mainly a way to
measure batching; a real deployment would fill the rows with different prompts.

All sequences produce **byte-identical output** to the single-sequence run (verified for
N = 4 and N = 8), i.e. the batching is correct.

Measured on the CPU above (Q4_K_M, block 32 / small-block 32, `threshold 0.9`, greedy,
235 generated tokens):

| `--diffusion-n-seq` | wall time | aggregate | per sequence |
|---|---|---|---|
| 1 | 44.5 s | 5.28 t/s | 5.28 t/s |
| 4 | 162.4 s | 5.79 t/s | 1.45 t/s |
| 8 | 321.2 s | 5.85 t/s | 0.73 t/s |

**Review / honest verdict:** on this **CPU** batching gives essentially *no* throughput
gain — aggregate speed stays flat (≈5.3–5.9 t/s) and the time grows ~linearly with `N`,
so per-sequence speed drops 1/N. That is expected: a CPU decode step is compute-bound and
widens ~linearly with `n_seq` (there are no tensor cores to fill and no memory-bandwidth
headroom to exploit), so the fixed per-step work simply multiplies. Batching is where the
official 2.5× over autoregressive decoding comes from **on Ascend NPUs/GPUs**, not here.
We kept it because it is correct, cheap to carry, and a prerequisite for any GPU
deployment — but do not expect a CPU speed-up from it.

## Notes / limitations

- CPU-only, `-t 16`, ≈ 4.6 GB file: a 256-token block-diffusion run takes tens of seconds
  to a few minutes (`block_length` forwards per block). Throughput can be improved with
  `alg = confidence_threshold, num_small_blocks = 1` (see the official README).
- The decoder reproduces the Hugging Face reference **token-for-token** on the f16 GGUF
  (see *Numerical parity*), and the block-diffusion path was cross-checked against the
  reference logits (`cos ≈ 0.996`, f16 vs bf16).
- **Quantization + the greedy fast mode can change the reasoning**: on the adversarial
  `9.11 vs 9.9` prompt the Q4_K_M fast mode first states the wrong intermediate answer and
  only corrects itself from the fraction comparison above, whereas the f16 model matches
  the reference. Use the f16 GGUF, or `alg = entropy`, `temp = 0`, when a case must be
  exactly reproduced.

## Supported features

Implemented (matching the reference `inference/generation_utils.py`):

| Feature | Status |
|---|---|
| Context-causal block diffusion — full attention inside a block, causal attention over the preceding context | ✓ |
| Block-by-block decoding (block length 32) | ✓ `--diffusion-block-length` |
| Committed-block KV cache with a causal mask; first token of the next block decoded from the last position | ✓ |
| Small blocks inside a block (`num_small_blocks`, optimum 4 or 8) | ✓ `--diffusion-small-block` |
| Variable-length inference / flexible context | ✓ |
| Logit shift (`diffusion.shift_logits`) | ✓ |
| Dedicated mask token (`tokenizer.ggml.mask_token_id = 45830`) | ✓ |
| Reveal order: `confidence`, `entropy`, `topk_margin` | ✓ `--diffusion-algorithm 4/1/2` |
| `confidence_threshold` fast mode (multi-token reveal per step) | ✓ `--diffusion-confidence-threshold` |
| EOS early stop | ✓ |
| Batched (multi-sequence) block diffusion | ✓ `--diffusion-n-seq` |
| Autoregressive decoding | ✓ `llama-cli` |

Not implemented:
- Vendor NPU kernels — this uses whatever backend llama.cpp was built with (CPU here).

## Original model

- Weights: [`openpangu/openPangu-R-7B-Diffusion`](https://huggingface.co/openpangu/openPangu-R-7B-Diffusion)
- Architecture details, benchmarks and citation: see the original model card.

	---
	language:
	- en
	- zh
	base_model: openpangu/openPangu-R-7B-Diffusion
	tags:
	- gguf
	- llama.cpp
	- diffusion
	- block-diffusion
	- openPangu
	license: other
	---

	# openPangu-R-7B-Diffusion — Q4_K_M GGUF (llama.cpp)

	Quantized Q4_K_M GGUF of [`openpangu/openPangu-R-7B-Diffusion`](https://huggingface.co/openpangu/openPangu-R-7B-Diffusion),
	together with a llama.cpp patch that adds native support for the model's
	context-causal block-diffusion decoding.

	> `openPangu-R-7B-Diffusion` is a 7B block-diffusion LLM. Generation is not plain
	> next-token autoregression: a block of tokens (default length 32) is denoised in
	> parallel with full (bi-directional) attention inside the block and **causal
	> attention for the preceding context**, revealing the most confident tokens first.
	> The model can also be run in ordinary autoregressive mode.

	Upstream llama.cpp (as of tag `b11719`) ships the `PanguEmbedded` architecture
	(used by the autoregressive `openPangu-Embedded`) but does not implement the
	diffusion decoding this model needs, hence the patch.

	## Files

	\| File \| Description \|
	\|---\|---\|
	\| `pangu-diffusion-Q4_K_M.gguf` \| Q4_K_M quantized weights (~4.6 GB), ready to run \|
	\| `pangu-diffusion-llamacpp.patch` \| llama.cpp patch adding openPangu-R-Diffusion support \|
	\| `build_llamacpp_pangu_diffusion.sh` \| one-click script: clone llama.cpp → apply patch → build \|

	## Quick start

	### 1. Build llama.cpp with the patch (one-click)

	```bash
	bash build_llamacpp_pangu_diffusion.sh
	# binaries end up in ./pangu-diffusion-build/llama.cpp/build/bin/
	```

	Or manually:

	```bash
	git clone --depth 1 --branch b11719 https://github.com/ggml-org/llama.cpp.git
	cd llama.cpp
	git apply /path/to/pangu-diffusion-llamacpp.patch
	cmake -B build -DGGML_NATIVE=ON -DLLAMA_CURL=OFF
	cmake --build build -j --target llama-diffusion-cli llama-cli llama-quantize
	```

	### 2. Diffusion decoding (recommended)

	`llama-diffusion-cli` applies the model's chat template automatically: pass the **raw
	user message**, not a pre-formatted prompt. The rendered prompt is

	```
	[unused9]系统：<system>[unused10][unused9]用户：<user>[unused10][unused9]助手：
	```

	```bash
	./build/bin/llama-diffusion-cli \
	-m pangu-diffusion-Q4_K_M.gguf \
	-p "Which is bigger, 9.11 or 9.9?" \
	-ub 256 \
	--diffusion-block-length 32 \
	--diffusion-small-block 4 \
	--diffusion-algorithm 1 \
	--temp 0 -t 16 --seed 42
	```

	Recommended sampling (from the official model card):
	`alg = entropy`, `num_small_blocks = 8` → `--diffusion-small-block 4` (with block length 32),
	`top_p = 1`, `temperature = 0`. For coding tasks the authors suggest
	`alg = entropy, num_small_blocks = 32 (→ --diffusion-small-block 1), top_p = 0.8, temperature = 1`.

	Fast mode (higher throughput): reveal every masked token whose confidence exceeds a
	threshold in a single denoising step, with one small block per block:

	```bash
	./build/bin/llama-diffusion-cli \
	-m pangu-diffusion-Q4_K_M.gguf \
	-p "<question>" \
	-ub 256 --diffusion-block-length 32 --diffusion-small-block 32 \
	--diffusion-confidence-threshold 0.9 --diffusion-algorithm 4 \
	--temp 0 -t 16 --seed 42
	```

	`--diffusion-n-seq N` runs N sequences through the block decoder in a single
	(micro-)batch (the prompt is replicated; see Batched decoding below).

	`--diffusion-algorithm` values: `0` origin, `1` entropy, `2` margin, `3` random, `4` confidence.
	`-ub N` is the maximum sequence length; the tail that does not fill a whole block is dropped.

	### 3. Autoregressive decoding

	```bash
	./build/bin/llama-cli -m pangu-diffusion-Q4_K_M.gguf \
	-p "[unused9]系统：[unused10][unused9]用户：Hi![unused10][unused9]助手：" -n 128 -t 16
	```

	## Example

	Prompt (translated): “Which is bigger, 9.11 or 9.9?” — command as in §2, fast mode
	(`-ub 256`, `--diffusion-confidence-threshold 0.9`, Q4_K_M):

	```
	I need to figure out which is bigger between 9.11 and 9.9. Both are decimals, so I should compare them directly.

	First, let's write them with the same number of decimal places to make it easier. 9.11 has two decimal places, and 9.9 has one. So, I can write 9.9 as 9.90 to make it two decimal places.

	So, 9.11 vs 9.90.

	Now, comparing the tenths place: both have 9 in the tenths place. So, that's the same.

	Next, the hundredths place: 9.11 has 1 in the hundredths place, and 9.90 has 0 in the hundredths place. Since 1 is greater than 0, ...

	I can think of them as fractions. 9.11 is 911/100, and 9.9 is 990/100.

	Since the denominator is the same, I can compare the numerators: 911 vs 990.
	911 is less than 990, so 911/100 is less than 990/100, which ...
	```

	For a short prompt the output is reproduced **token-for-token identically to the Hugging
	Face reference** (see Numerical parity below), e.g. `-p "hello"` (f16 GGUF):
	`[unused16]Okay, the user just said "hello". That's a simple greeting. I should respond
	politely and warmly. Maybe they're just starting a conversation`.

	## Speed when you actually use it (measured)

	Test environment — the resources actually usable inside the container (docker):

	\| Resource \| Value \|
	\|---\|---\|
	\| CPU \| Intel Xeon Platinum 8559C \|
	\| Effective cores \| 16 vCPU (cgroup `cpu.max` = `1600000 100000`) \|
	\| Memory (cgroup limit) \| ≈ 97 GiB \|
	\| GPU \| none usable — `/dev/nvidia*` nodes exist but `cuInit()` returns `CUDA_ERROR_NO_DEVICE` and NVML init fails \|
	\| Build / runtime \| CPU-only, `-t 16` \|

	These are end-user speeds measured from real runs of the CLI on
	`pangu-diffusion-Q4_K_M.gguf` (4.59 GiB, 8.03 B params), i.e. what you actually
	experience while using it:

	\| What the user experiences \| Mode \| Speed \|
	\|---\|---\|---\|
	\| Input — waiting for the prompt to be read before output starts \| both \| ≈ 78 t/s short prompt; ≈ 132 t/s for longer prompts (`llama-bench` pp512) \|
	\| Output — text generated (autoregressive, streamed live) \| `llama-cli` \| 11.9 t/s (llama-bench tg128: 11.1 t/s) \|
	\| Output — block diffusion (quality: block 32 / small-block 4, `alg=entropy`, `temp=0`) \| `llama-diffusion-cli` \| ≈ 4.4 t/s \|
	\| Output — block diffusion (fast: `--diffusion-confidence-threshold 0.9`, small-block 32, `alg=confidence` — the official throughput setting) \| `llama-diffusion-cli` \| ≈ 5 t/s \|

	Notes:
	- Input (prompt processing) is the same causal prefill for both paths; it is what you wait
	for before the answer appears.
	- The AR output speed (≈ 12 t/s) is what `llama-cli` prints as `Generation: X t/s`; the
	diffused text is only flushed at the end of a run (unless `--diffusion-visual`).
	- Block diffusion normally reveals one token per denoising step, i.e. up to `block_length`
	(= 32) forward passes per block. With `--diffusion-confidence-threshold T` every masked
	token whose confidence exceeds `T` is revealed in the same step (`transfer = argmax(conf)
	\| (conf > T)`), which raises throughput — the same mechanism the authors report as giving
	up to 2.5× over autoregressive decoding on Ascend NPUs. A lower `T` is faster but less
	accurate; measured on this CPU (235 tokens, block 32 / small-block 32, `alg=confidence`,
	`temp=0`):

	\| `--diffusion-confidence-threshold` \| speed \|
	\|---\|---\|
	\| `0.9` (default) \| 4.9 t/s \|
	\| `0.7` \| 7.4 t/s \|
	\| `0.5` \| 8.7 t/s \|

	- The AR figures are the best reference for raw model throughput on this hardware.

	## What the patch does

	1. `src/llama-arch.cpp` — mark `LLM_ARCH_PANGU_EMBED` as a diffusion architecture
	(`llm_arch_is_diffusion()`), so `llama-diffusion-cli` accepts the model.
	2. `examples/diffusion/diffusion.{h,cpp}` — add a context-causal block-diffusion
	decoder (`diffusion.block_attention`):
	- prefill the prompt with a causal forward (kept in the KV cache);
	- denoise each block with `causal_attn = false` → full attention over the cached
	context plus the current block (the KV-cache kq mask keeps every non-empty
	cached cell when non-causal);
	- re-decode the block each step, dropping its stale keys (`llama_memory_seq_rm`),
	reveal the most-confident masked token of the current small block, left to right;
	- commit the block with a causal forward so the KV cache holds correct causal
	context, then sample the first token of the next block from the last position;
	- `diffusion_generate_nseq()` generalises this to `n_seq` sequences sharing one
	micro-batch (see `--diffusion-n-seq`).
	3. `examples/diffusion/diffusion-cli.cpp` — read `diffusion.block_attention`, add
	`--diffusion-small-block/-confidence-threshold/-n-seq`, size the context for the
	batched prefill, and use greedy sampling when `temperature == 0`.
	4. `gguf-py` + `conversion/pangu.py` — write the new `diffusion.block_attention`
	metadata key for diffusion checkpoints (alongside `diffusion.shift_logits` and
	`tokenizer.ggml.mask_token_id`), and set `tokenizer.ggml.add_space_prefix = false`
	(see Tokenizer fix below).

	### Model metadata in the GGUF

	```
	general.architecture = pangu-embedded
	diffusion.shift_logits = true
	diffusion.block_attention = true
	tokenizer.ggml.mask_token_id = 45830
	tokenizer.ggml.add_space_prefix = false
	```

	### Tokenizer fix

	The upstream conversion honours `add_prefix_space` from `tokenizer_config.json`, but the
	slow `PanguTokenizer` (sentencepiece, `legacy: true`) does not actually prepend a space.
	With the flag left on, llama.cpp inserted spurious space tokens after every special token
	(`[unused9]系统：…` became `[unused9]␠系统：…`), which changed the input and the generation.
	The patch forces `tokenizer.ggml.add_space_prefix = false`; the tokenizer output then
	matches the reference exactly (a 22-token prompt becomes 22 tokens, was 37).

	### Numerical parity

	Validated against the Hugging Face reference (`inference/generation_utils.py`,
	`alg=confidence`, `num_small_blocks=1`, `threshold=0.9`, `temperature=0`) on the f16
	GGUF: for `-p "hello"` the generated token IDs are identical, token for token, to the
	reference. (The earlier mismatch was entirely the tokenizer issue above, not the
	decoder.)

	## Batched decoding (`--diffusion-n-seq`)

	`--diffusion-n-seq N` runs N independent sequences through the block decoder inside a
	single (micro-)batch. The example CLI replicates the prompt, so this is mainly a way to
	measure batching; a real deployment would fill the rows with different prompts.

	All sequences produce byte-identical output to the single-sequence run (verified for
	N = 4 and N = 8), i.e. the batching is correct.

	Measured on the CPU above (Q4_K_M, block 32 / small-block 32, `threshold 0.9`, greedy,
	235 generated tokens):

	\| `--diffusion-n-seq` \| wall time \| aggregate \| per sequence \|
	\|---\|---\|---\|---\|
	\| 1 \| 44.5 s \| 5.28 t/s \| 5.28 t/s \|
	\| 4 \| 162.4 s \| 5.79 t/s \| 1.45 t/s \|
	\| 8 \| 321.2 s \| 5.85 t/s \| 0.73 t/s \|

	Review / honest verdict: on this CPU batching gives essentially no throughput
	gain — aggregate speed stays flat (≈5.3–5.9 t/s) and the time grows ~linearly with `N`,
	so per-sequence speed drops 1/N. That is expected: a CPU decode step is compute-bound and
	widens ~linearly with `n_seq` (there are no tensor cores to fill and no memory-bandwidth
	headroom to exploit), so the fixed per-step work simply multiplies. Batching is where the
	official 2.5× over autoregressive decoding comes from on Ascend NPUs/GPUs, not here.
	We kept it because it is correct, cheap to carry, and a prerequisite for any GPU
	deployment — but do not expect a CPU speed-up from it.

	## Notes / limitations

	- CPU-only, `-t 16`, ≈ 4.6 GB file: a 256-token block-diffusion run takes tens of seconds
	to a few minutes (`block_length` forwards per block). Throughput can be improved with
	`alg = confidence_threshold, num_small_blocks = 1` (see the official README).
	- The decoder reproduces the Hugging Face reference token-for-token on the f16 GGUF
	(see Numerical parity), and the block-diffusion path was cross-checked against the
	reference logits (`cos ≈ 0.996`, f16 vs bf16).
	- Quantization + the greedy fast mode can change the reasoning: on the adversarial
	`9.11 vs 9.9` prompt the Q4_K_M fast mode first states the wrong intermediate answer and
	only corrects itself from the fraction comparison above, whereas the f16 model matches
	the reference. Use the f16 GGUF, or `alg = entropy`, `temp = 0`, when a case must be
	exactly reproduced.

	## Supported features

	Implemented (matching the reference `inference/generation_utils.py`):

	\| Feature \| Status \|
	\|---\|---\|
	\| Context-causal block diffusion — full attention inside a block, causal attention over the preceding context \| ✓ \|
	\| Block-by-block decoding (block length 32) \| ✓ `--diffusion-block-length` \|
	\| Committed-block KV cache with a causal mask; first token of the next block decoded from the last position \| ✓ \|
	\| Small blocks inside a block (`num_small_blocks`, optimum 4 or 8) \| ✓ `--diffusion-small-block` \|
	\| Variable-length inference / flexible context \| ✓ \|
	\| Logit shift (`diffusion.shift_logits`) \| ✓ \|
	\| Dedicated mask token (`tokenizer.ggml.mask_token_id = 45830`) \| ✓ \|
	\| Reveal order: `confidence`, `entropy`, `topk_margin` \| ✓ `--diffusion-algorithm 4/1/2` \|
	\| `confidence_threshold` fast mode (multi-token reveal per step) \| ✓ `--diffusion-confidence-threshold` \|
	\| EOS early stop \| ✓ \|
	\| Batched (multi-sequence) block diffusion \| ✓ `--diffusion-n-seq` \|
	\| Autoregressive decoding \| ✓ `llama-cli` \|

	Not implemented:
	- Vendor NPU kernels — this uses whatever backend llama.cpp was built with (CPU here).

	## Original model

	- Weights: [`openpangu/openPangu-R-7B-Diffusion`](https://huggingface.co/openpangu/openPangu-R-7B-Diffusion)
	- Architecture details, benchmarks and citation: see the original model card.