Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,165 @@
+---
+language:
+  - en
+  - zh
+base_model: openpangu/openPangu-R-7B-Diffusion
+tags:
+  - gguf
+  - llama.cpp
+  - diffusion
+  - block-diffusion
+  - openPangu
+license: other
+---
+
+# openPangu-R-7B-Diffusion — Q4_K_M GGUF (llama.cpp)
+
+Quantized **Q4_K_M** GGUF of [`openpangu/openPangu-R-7B-Diffusion`](https://huggingface.co/openpangu/openPangu-R-7B-Diffusion),
+together with a llama.cpp patch that adds native support for the model's
+**context-causal block-diffusion** decoding.
+
+> `openPangu-R-7B-Diffusion` is a 7B block-diffusion LLM. Generation is *not* plain
+> next-token autoregression: a block of tokens (default length **32**) is denoised in
+> parallel with **full (bi-directional) attention inside the block** and **causal
+> attention for the preceding context**, revealing the most confident tokens first.
+> The model can also be run in ordinary autoregressive mode.
+
+Upstream llama.cpp (as of tag `b11719`) ships the `PanguEmbedded` architecture
+(used by the autoregressive `openPangu-Embedded`) but does **not** implement the
+diffusion decoding this model needs, hence the patch.
+
+## Files
+
+| File | Description |
+|---|---|
+| `pangu-diffusion-Q4_K_M.gguf` | Q4_K_M quantized weights (~4.6 GB), ready to run |
+| `pangu-diffusion-llamacpp.patch` | llama.cpp patch adding openPangu-R-Diffusion support |
+| `build_llamacpp_pangu_diffusion.sh` | one-click script: clone llama.cpp → apply patch → build |
+
+## Quick start
+
+### 1. Build llama.cpp with the patch (one-click)
+
+```bash
+bash build_llamacpp_pangu_diffusion.sh
+# binaries end up in ./pangu-diffusion-build/llama.cpp/build/bin/
+```
+
+Or manually:
+
+```bash
+git clone --depth 1 --branch b11719 https://github.com/ggml-org/llama.cpp.git
+cd llama.cpp
+git apply /path/to/pangu-diffusion-llamacpp.patch
+cmake -B build -DGGML_NATIVE=ON -DLLAMA_CURL=OFF
+cmake --build build -j --target llama-diffusion-cli llama-cli llama-quantize
+```
+
+### 2. Diffusion decoding (recommended)
+
+The chat format is
+
+```
+[unused9]系统：<system>[unused10][unused9]用户：<user>[unused10][unused9]助手：
+```
+
+```bash
+./build/bin/llama-diffusion-cli \
+  -m pangu-diffusion-Q4_K_M.gguf \
+  -p "[unused9]系统：[unused10][unused9]用户：Which is bigger, 9.11 or 9.9?[unused10][unused9]助手：" \
+  -ub 256 \
+  --diffusion-block-length 32 \
+  --diffusion-small-block 4 \
+  --diffusion-algorithm 1 \
+  --temp 0 -t 16 --seed 42
+```
+
+Recommended sampling (from the official model card):
+`alg = entropy`, `num_small_blocks = 8` → `--diffusion-small-block 4` (with block length 32),
+`top_p = 1`, `temperature = 0`. For coding tasks the authors suggest
+`alg = entropy, num_small_blocks = 32 (→ --diffusion-small-block 1), top_p = 0.8, temperature = 1`.
+
+`--diffusion-algorithm` values: `0` origin, `1` entropy, `2` margin, `3` random, `4` confidence.
+`-ub N` is the maximum sequence length; the tail that does not fill a whole block is dropped.
+
+### 3. Autoregressive decoding
+
+```bash
+./build/bin/llama-cli -m pangu-diffusion-Q4_K_M.gguf \
+  -p "[unused9]系统：[unused10][unused9]用户：Hi![unused10][unused9]助手：" -n 128 -t 16
+```
+
+## Example
+
+Prompt (translated): *“Which is bigger, 9.11 or 9.9?”* — command as in §2:
+
+```
+# Which is bigger, 9.11 or 9.9?
+
+I need to compare 9.11 and 9.9 to see which one is bigger. Both numbers are
+between 9 and 10, so they're both greater than 9 but less than 10.
+
+Let me think about this carefully. 9.11 means 9 and 11 hundredths, which is
+9 + 11/100. Similarly, 9.9 means 9 and 9 tenths, which is 9 + 9/10.
+
+To compare them, I should convert them to the same format. Let me convert both
+to hundredths.
+
+9.9 = 9.90 (adding a zero to make it 9.90)
+
+Now I have:
+- 9.11
+- 9.90
+
+Since both have the same number of decimal places, I can compare them directly.
+
+The first digit after the decimal is 1 for 9.11 and 9 for 9.90. Since 9 > 1,
+9.90 is larger than 9.11.
+
+So, 9.9 is bigger than 9.11.
+```
+
+## What the patch does
+
+1. **`src/llama-arch.cpp`** — mark `LLM_ARCH_PANGU_EMBED` as a diffusion architecture
+   (`llm_arch_is_diffusion()`), so `llama-diffusion-cli` accepts the model.
+2. **`examples/diffusion/diffusion.{h,cpp}`** — add a *context-causal block-diffusion*
+   decoder (`diffusion.block_attention`):
+   - prefill the prompt with a causal forward (kept in the KV cache);
+   - denoise each block with `causal_attn = false` → full attention over the cached
+     context **plus** the current block (the KV-cache kq mask keeps every non-empty
+     cached cell when non-causal);
+   - re-decode the block each step, dropping its stale keys (`llama_memory_seq_rm`),
+     reveal the most-confident masked token of the current small block, left to right;
+   - commit the block with a causal forward so the KV cache holds correct causal
+     context, then sample the first token of the next block from the last position.
+3. **`examples/diffusion/diffusion-cli.cpp`** — read `diffusion.block_attention`,
+   add `--diffusion-small-block`, use greedy sampling when `temperature == 0`.
+4. **`gguf-py` + `conversion/pangu.py`** — write the new `diffusion.block_attention`
+   metadata key for diffusion checkpoints (alongside `diffusion.shift_logits` and
+   `tokenizer.ggml.mask_token_id`).
+
+### Model metadata in the GGUF
+
+```
+general.architecture            = pangu-embedded
+diffusion.shift_logits          = true
+diffusion.block_attention       = true
+tokenizer.ggml.mask_token_id    = 45830
+```
+
+## Notes / limitations
+
+- CPU-only, `-t 16`, ≈ 4.6 GB file: a 256-token block-diffusion run takes a few minutes
+  (`block_length` forwards per block). Throughput can be improved with
+  `alg = confidence_threshold, num_small_blocks = 1` (see the official README).
+- The committed-block KV-cache path was numerically cross-checked against the
+  Hugging Face reference (`cos ≈ 0.996`, f16 vs bf16), and the same binaries reproduce
+  the model's reference-quality output above.
+- Quantization may slightly affect the tricky reasoning cases; use `alg = entropy`,
+  `temp = 0` for the most stable results.
+
+## Original model
+
+- Weights: [`openpangu/openPangu-R-7B-Diffusion`](https://huggingface.co/openpangu/openPangu-R-7B-Diffusion)
+- Architecture details, benchmarks and citation: see the original model card.