Blame for README.md

---
license: gpl-3.0
library_name: llama.cpp
tags:
  - openpangu
  - llama.cpp
  - moe
  - gguf
  - cpu-inference
---

# openPangu-2.0-Flash support for llama.cpp v0.7.0

This repository contains a patch that adds support for **openPangu-2.0-Flash**
(92B MoE, ~6B active parameters, 512K context) to the latest llama.cpp release
**v0.7.0** (commit `620d2225cc436ab0b001e3200e590e3b286ef9ab`, 2026-10-15).

The support was ported from
[mrexodia/llama.cpp-openPangu-2.0-Flash](https://github.com/mrexodia/llama.cpp-openPangu-2.0-Flash)
(which tracks an older llama.cpp base) and rebased on top of v0.7.0.

Files:

- `llama.cpp-openpangu-v0.7.0.diff` — the full patch (48 files, +3340 / −368)
- `patch.sh` — downloads the pinned llama.cpp commit, applies the patch and
  prints build/run instructions
- `PROMPT_REPRODUCE.md` — task brief for reproducing this project with a
  coding agent (port + A/B experiment protocol + validation + upload)
- `LICENSE` — GPL-3.0 (this project) with the references/licenses of the
  upstream projects it is built on

## 中文說明

本 repo 是將 openPangu-2.0-Flash（92B MoE、約 6B 激活參數、512K 上下文）
的 llama.cpp 支援，從 mrexodia 的 fork 移植到最新的 llama.cpp **v0.7.0** 上。

用法：

1. 執行 `./patch.sh`（會自動下載 v0.7.0 原始碼並套用 patch）
2. `cd llama.cpp-620d2225cc436ab0b001e3200e590e3b286ef9ab`
3. `cmake -B build -S . -DCMAKE_BUILD_TYPE=Release && cmake --build build -j`
4. 下載 GGUF 模型（見下方連結）後即可執行 `llama-cli` / `llama-server`

## What is included

- The `openpangu-v2` architecture: absorbed MLA attention, DSA sparse attention
  (lightning-indexer top-2048 selection on the global layers), per-layer
  sliding-window attention via a hybrid iSWA cache, 4-stream manifold
  hyper-connections (mHC), MoME causal convolutions with recurrent cross-batch
  state, and 128 learned cache-resident attention sinks per layer
- Fused CPU/CUDA kernels for the mHC mixing chain (`GGML_OP_SINKHORN`,
  `GGML_OP_HC_MIX`), with automatic fallback to the unfused ops on backends
  without the kernels
- Gather-based sparse attention at decode: DSA layers attend only the top-2048
  selected tokens (plus sinks), with a fused indexer-scoring kernel
  (`GGML_OP_DSA_SCORE`)
- Fused DSA indexer scoring at prefill (tiled kernel that skips the multi-GB
  scoring intermediate) plus a radix-select top-k for `GGML_OP_TOP_K`
- Multi-token-prediction self-speculative decoding (`--spec-type draft-mtp`,
  with the split MTP draft GGUF)
- Chat support: tool calling and ` thinking` reasoning parsing for the openPangu
  template
- HF → GGUF conversion (`conversion/pangu.py`) and a base/MTP split tool
  (`conversion/split_pangu_mtp.py`)

Ported and adapted to v0.7.0 APIs, plus fixes on top of the original fork:

- adapted to the v0.7.0 common chat parser API (`build_chat_peg_parser`),
  per-layer `n_ff_exp()`, and the new `build_attn_mha(..., n_kv_max, ...)`
  signature
- CPU AMX fix: the AMX weight packer only implements 2D layouts; batched
  (3D/4D) quantized weights — e.g. the MLA `wk_b`/`wv_b` tensors — were
  corrupted when routed through the AMX extra buffer, producing gibberish
  output. The AMX backend now rejects batched tensors and they run on the
  plain CPU path
- the absorbed-MLA `wk_b`/`wv_b` inputs are kept contiguous so they are also
  correct with weight repacking on other backends

## Requirements

- A C/C++ build environment (CMake ≥ 3.14, C++17 compiler)
- No GPU required (CPU-only works); CUDA/ Metal/ Vulkan also supported
- Get the GGUF models at
  [mrexodia/openPangu-2.0-Flash-GGUF](https://huggingface.co/mrexodia/openPangu-2.0-Flash-GGUF):
  - `openPangu-2.0-Flash-base-Q3_K_M.gguf` — 40.7 GiB
  - `openPangu-2.0-Flash-base-Q4_K_M.gguf` — 51.5 GiB
  - `openPangu-2.0-Flash-base-Q8_0.gguf` — 90.8 GiB
  - `openPangu-2.0-Flash-base-BF16.gguf` — 170.7 GiB
  - `openPangu-2.0-Flash-mtp-Q8_0.gguf` — 9.2 GiB (optional MTP head)

## Quick start

```sh
./patch.sh
cd llama.cpp-620d2225cc436ab0b001e3200e590e3b286ef9ab
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-cli llama-server
```

CUDA build:

```sh
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
```

## Run

```sh
./build/bin/llama-cli -m openPangu-2.0-Flash-base-Q3_K_M.gguf -c 65536 --jinja -t $(nproc)

./build/bin/llama-server -m openPangu-2.0-Flash-base-Q3_K_M.gguf \
    -c 65536 --jinja --host 0.0.0.0 --port 8080
```

Context scales up to `-c 524288`; the compressed MLA KV cache needs only ~12 GB
at the full 512K.

### Multi-token prediction (self-speculative decoding)

```sh
./build/bin/llama-cli -m openPangu-2.0-Flash-base-Q3_K_M.gguf \
    --spec-draft-model openPangu-2.0-Flash-mtp-Q8_0.gguf \
    --spec-type draft-mtp -c 65536 --jinja
```

Notes:

- The MTP draft adds ~10 GB of RAM usage on top of the base model
- On CPUs with AMX, weight repacking (`--repack`, enabled by default) costs
  ~16 GiB of extra RAM; if the combined footprint does not fit, run with `-nr`
  (no repack)
- On CPU-only setups the draft overhead can make generation slower than plain
  decoding — measure on your hardware (`llama-bench` does not support draft
  models; use `llama-cli` and compare the reported t/s)

## Test environment（測試環境）

All performance numbers below were measured on a real machine:

- **CPU**: Intel Xeon Platinum 8559C (Emerald Rapids, supports AMX); the test
  container is capped at **16 vCPUs** (host has 2 sockets × 48 cores × 2 SMT)
- **RAM**: 2.0 TiB host; the test environment enforces a **64 GiB per-process
  memory cap**, which is why all testing used the 40.7 GiB `Q3_K_M` quant
- **GPU**: none — CPU-only inference, 16 threads (`-t 16`)
- **OS/toolchain**: Linux, gcc 14.2.0, CMake 3.31.6
- **Model**: `openPangu-2.0-Flash-base-Q3_K_M.gguf` (40.72 GiB, 91.6 B params)

Measured resident memory of `llama-cli` with this model (`-c 512`, `-n 1`):

| configuration | peak RSS |
|---|---|
| default (AMX weight repacking enabled) | ~57.1 GiB |
| `-nr` (no repack) | ~41.2 GiB |
| `-nr` + MTP draft (Q8_0) | ~52 GiB |
| default + MTP draft (Q8_0) | exceeds 64 GiB — could not run under the test cap |

## Tests

- full `ctest` suite (61 tests): **61/61 pass** — including
  `test-generate-models` and `test-llama-context`, which now cover
  openpangu-v2 as a first-class arch (synthetic model fixtures, KV-cache
  cases) and round-trip its K-sink-prefix metadata through the model saver
- `test-backend-ops -b CPU -o SINKHORN -o HC_MIX -o DSA_SCORE -o TOP_K`:
  534/534 tests pass
- `test-chat`, `test-chat-template`, `test-chat-peg-parser`,
  `test-chat-auto-parser`: all pass

## Comparison with the original fork

Measured against the original
[mrexodia/llama.cpp-openPangu-2.0-Flash](https://github.com/mrexodia/llama.cpp-openPangu-2.0-Flash)
(HEAD `76c92350b`, llama.cpp base `6f4f53f2b` from 2026-06-29), same machine,
same model, same flags.

### Compatibility on this machine (AMX CPU)

| | original fork | this port (v0.7.0) |
|---|---|---|
| default settings | **aborts** — `GGML_ASSERT(*cur_backend_id != -1)` during `sched_reserve` (AMX weight repacking) | runs |
| `-nr` (weight repacking disabled) | runs | runs |

The original fork cannot be used with default settings on a CPU with AMX.
Its `llama-bench` has no option to disable repacking at all — the original
numbers below required a local source edit. This port fixes the underlying
AMX packing bug (the AMX packer silently corrupted batched/3D weights).

### llama-bench, `Q3_K_M`, `-t 16 -r 3` (2026-10-16)

Both builds with weight repacking disabled (`-nr`, the only common
runnable configuration on this machine):

| test | original fork | this port | change |
|------|--------------:|----------:|-------:|
| pp128 | 41.51 ± 2.21 | 60.47 ± 4.63 | **+46 %** |
| pp512 | 42.62 ± 0.83 | 64.75 ± 2.26 | **+52 %** |
| tg32  | 11.65 ± 0.19 | 11.52 ± 0.35 | ≈ |
| tg128 | 11.47 ± 0.21 | 11.82 ± 0.72 | ≈ |

This port with the default AMX weight repacking (original fork cannot run
this configuration):

| test | this port (default) |
|------|--------------------:|
| pp128 | 56.84 ± 1.32 |
| pp512 | 62.65 ± 0.54 |
| tg32  | 11.94 ± 0.71 |
| tg128 | 12.02 ± 0.30 |

Reading these numbers:

- Prefill is ~1.5× faster than the original fork, mostly from the v0.7.0
  base (3.5 months of upstream CPU optimizations). Generation is on par.
- On this CPU, AMX weight repacking is performance-neutral and costs
  ~16 GiB of extra RAM, so `-nr` is usually the better choice.
- The host is shared and measurements vary between runs (see the ±).

### Real inference (llama-cli end-to-end)

Actual generation with a real prompt (3,271 tokens of English text),
`llama-cli -c 8192 -n 128 -t 16 --no-warmup -st`, two runs each
(prompt eval / generation speed as reported by llama-cli):

| configuration | prompt eval | generation |
|---------------|------------:|-----------:|
| original fork, `-nr` | 29.7 / 28.9 t/s | 6.1 / 6.1 t/s |
| this port, `-nr` | 41.0 / 41.0 t/s | 6.0 / 6.0 t/s |
| this port, default (AMX repack) | 37.7 / 38.1 t/s | 5.6 / 5.5 t/s |

- real prompt processing is ~**+40 %** faster than the original fork
  (+29 % in the default AMX repack configuration)
- generation with a ~3K-token context is ~6 t/s in all configurations
  (the DSA indexer adds per-token work over the cached context, so this is
  lower than the tg128 llama-bench numbers on a fresh 128-token run)
- run-to-run variance on this host is small for these end-to-end runs

### MTP (self-speculative decoding)

Same prompt, `llama-cli -n 128 -nr`:

| | generation |
|---|---|
| original fork, MTP draft | 4.9 t/s |
| this port, MTP draft | 4.7 t/s |
| either fork, plain decoding (`llama-bench` tg128) | 11.5–12.0 t/s |

On CPU the MTP draft head is a net loss in both forks; it is included for
completeness and may pay off on GPUs where the draft forward pass is cheap.

## CPU optimization experiments（CPU 優化實驗記錄）

Upstream CPU-speedup candidates (open pull requests of llama.cpp) are
benchmarked one by one on the machine described above. Because this shared
host is noisy, each candidate is measured with an **alternating A/B
protocol**: two binaries built from the same source tree (baseline vs
baseline + candidate patch) are run interleaved in the same session
(`llama-bench -t 16 -r 3 -p 512 -n 128`, default settings, 3–8 rounds each);
the table reports the medians over all rounds. Every number is
**incremental**: each candidate is compared against the **previous merged
state**, not against the original port.

> Methodology fix (2026-10-16): a first attempt measured `-nr` via the
> `LLAMA_ARG_REPACK=0` env var. `llama-bench` ignores that variable (it has
> its own parser; only common-arg tools such as `llama-cli` /
> `llama-perplexity` honor it), so those runs were silently default-mode.
> The `-nr` entry for #27590 was re-measured with `llama-bench --repack 0`,
> the flag the tool actually supports.

Only candidates that win beyond the run-to-run noise are merged into the
patch (✅ 有幫助); candidates without a real gain are recorded here with
their data and are **not** merged (❌ 沒幫助).

| candidate | content | verdict | pp512 t/s (median) | tg128 t/s (median) |
|---|---|---:|---:|---:|
| upstream PR [#27024](https://github.com/ggml-org/llama.cpp/pull/27024) by [@jinzihao](https://github.com/jinzihao) | ggml-cpu/amx: fix the (dead) `block_q8_K` VNNI quantization and enable the VNNI path | ✅ 有幫助 — merged | 57.25 → 59.20 (**+3.4 %**) | 10.94 → 11.24 (**+2.7 %**) |
| upstream PR [#22525](https://github.com/ggml-org/llama.cpp/pull/22525) by [@HyeongiJeon](https://github.com/HyeongiJeon) | ggml-cpu: optimize `ggml_gemm_q4_K_8x8_q8_K` interleaving/staging for AVX-512 (and AVX2) | ✅ 有幫助（prefill）— merged | 55.78 → 57.42 (**+2.9 %**) | 10.70 → 10.87 (+1.5 %, neutral) |
| upstream PR [#19706](https://github.com/ggml-org/llama.cpp/pull/19706) by [@Manogna-Sree](https://github.com/Manogna-Sree) | Q6_K block interleaving for x86 SIMD (AVX-512/AVX2) | ❌ 沒幫助 — not merged | 58.16 → 58.34 (+0.3 %) | 11.06 → 11.07 (+0.1 %) |
| upstream PR [#27590](https://github.com/ggml-org/llama.cpp/pull/27590) by [@SongXiaoXi](https://github.com/SongXiaoXi) | ggml-cpu: add AVX-512/VNNI paths for Q5_K/Q6_K dot products | ✅ 有幫助（`-nr`）— merged | `-nr`: 65.91 → 66.35 (+0.7 %) | `-nr`: 11.16 → 11.32 (**+1.4 %**) |
| upstream PR [#18698](https://github.com/ggml-org/llama.cpp/pull/18698) by [@zzjianhui](https://github.com/zzjianhui) | ggml-cpu: repack weights in parallel (OpenMP, NUMA-local thread placement) | ❌ 沒幫助（推論退化）— not merged | 56.93 → 54.58 (**−4.1 %**) | 10.67 → 10.45 (**−2.1 %**) |
| draft PR [#27032](https://github.com/ggml-org/llama.cpp/pull/27032) by [@lexasub](https://github.com/lexasub) | CPU optimizations (GET_ROWS+ADD fusion, `get_rows` fast paths, tokenizer, KV-cell rework) — only the ggml-kernel subset is portable onto this tree | ❌ 沒幫助（不適用於本模型）— not merged | 56.64 → 56.88 (+0.4 %) | 10.82 → 10.73 (−0.8 %) |
| draft PR [#27396](https://github.com/ggml-org/llama.cpp/pull/27396) by [@lexasub](https://github.com/lexasub) | speculative draft-model prefill (zero-copy embedding, event-based sync) for EAGLE3/DFlash/GLM4-style draft heads | ❌ 沒幫助（不適用；未移植）— not merged | — | — |

Round-by-round for #27024 (candidate vs baseline, t/s):

| round | pp512 | tg128 |
|---|---:|---:|
| 1 | 57.66 vs 57.69 | 11.09 vs 10.57 |
| 2 | 59.20 vs 57.25 | 11.05 vs 10.94 |
| 3 | 58.66 vs 56.57 | 11.47 vs 11.27 |
| 4 | 62.13 vs 55.67 | 11.62 vs 10.83 |
| 5 | 61.76 vs 65.98 | 11.24 vs 12.01 |

The candidate won 4 of 5 pairwise rounds on both metrics; the single losing
round (r5) is a transient quiet window on the host (both absolute values
spike upward). Median deltas stay positive (+3.4 % / +2.7 %), so the change
is merged. All values t/s; measured 2026-10-16.

Round-by-round for #22525 (candidate vs baseline, t/s), default settings.
Note: `Q3_K_M` does contain 134 `Q4_K` tensors (the MoE expert weights),
so the Q4_K GEMM path this PR optimizes is actually exercised:

| round | pp512 | tg128 |
|---|---:|---:|
| 1 | 56.69 vs 57.18 | 10.97 vs 10.97 |
| 2 | 57.30 vs 57.15 | 10.88 vs 10.78 |
| 3 | 57.30 vs 55.90 | 10.30 vs 10.94 |
| 4 | 56.26 vs 55.66 | 10.85 vs 9.51 |
| 5 | 57.91 vs 55.07 | 10.55 vs 10.62 |
| 6 | 57.55 vs 56.81 | 10.60 vs 10.19 |
| 7 | 57.84 vs 55.52 | 11.04 vs 10.43 |
| 8 | 59.24 vs 55.06 | 10.97 vs 11.05 |

The candidate won 6 of 8 pairwise rounds on prefill (1 loss, 1 tie within
0.5 %); decode is neutral (4 W / 3 L / 1 T). The PR only changes the AVX-512
interleaving/staging of the Q4_K GEMM (the prefill path), so the measured
profile matches the expectation. Merged. Measured 2026-10-16.

Round-by-round for #19706 (candidate vs baseline, t/s), default settings:

| round | pp512 | tg128 |
|---|---:|---:|
| 1 | 58.42 vs 53.67 | 11.67 vs 11.45 |
| 2 | 59.46 vs 58.68 | 12.04 vs 10.84 |
| 3 | 54.57 vs 60.04 | 10.54 vs 11.82 |
| 4 | 62.32 vs 57.06 | 10.95 vs 11.16 |
| 5 | 57.66 vs 58.59 | 11.10 vs 10.86 |
| 6 | 57.26 vs 57.92 | 11.05 vs 11.17 |
| 7 | 59.71 vs 56.95 | 10.98 vs 10.96 |
| 8 | 58.25 vs 58.41 | 11.11 vs 10.83 |

**Verdict ❌ 沒幫助 (no measurable gain).** Medians differ by +0.3 %
(prefill) and +0.1 % (decode), with a 4 W / 3 L / 1 T split on both — pure
noise. The base `Q3_K_M` model contains only **one** single `Q6_K` tensor,
so a faster Q6_K kernel has almost no weight to act on. The PR is *not*
merged; the data is recorded here for the record. Measured 2026-10-16.

Round-by-round for #27590 (candidate vs baseline, t/s), `-nr` mode
(`llama-bench --repack 0`). The Q5_K/Q6_K dots this PR changes are used on
the plain CPU path selected by `-nr`; with default repacking those weights go
through the repack gemv/gemm kernels instead (default-mode incremental
measured ~neutral, within noise):

| round | pp512 | tg128 |
|---|---:|---:|
| 1 | 67.88 vs 67.80 | 11.17 vs 11.15 |
| 2 | 66.08 vs 66.46 | 11.46 vs 11.18 |
| 3 | 66.01 vs 65.34 | 11.32 vs 11.19 |
| 4 | 66.63 vs 65.36 | 11.33 vs 11.12 |

**Verdict ✅ 有幫助 (`-nr` 模式, +0.7 % prefill / +1.4 % decode medians;
decode won 4 of 4 rounds).** In this model the PR affects Q5_K (2 tensors,
the dense `ffn_down` of layers 0–1) and Q6_K (1 tensor — `output.weight`,
the 388 M-param head that is read on every token), so the consistent decode
win matches the expectation. One caveat: the upstream PR is *closed but never
merged* (only an automated AI-content flag, no review), so it received extra
local validation before merging:

- perplexity on a fixed corpus: 1.0354 → 1.0354 (identical)
- `test-backend-ops -b CPU -o MUL_MAT -p 'type=q5_K|type=q6_K'` passes

Merged. Re-measured 2026-10-16 with the correct `--repack 0` flag (see
methodology note above).

Round-by-round for #18698 (candidate vs baseline = current HEAD i.e. all
merged patches, t/s), default settings. This PR repacks the quantized
weights in parallel with OpenMP and pins each repack thread NUMA-local.
(Ported by hand: the year-old PR conflicts with the current tree and depends
on the never-merged PR #18239.)

| round | pp512 | tg128 |
|---|---:|---:|
| 1 | 54.24 vs 56.40 | 10.48 vs 10.35 |
| 2 | 55.50 vs 57.92 | 10.37 vs 10.67 |
| 3 | 54.58 vs 56.93 | 10.45 vs 10.87 |

**Verdict ❌ 沒幫助 (clear prefill regression, not merged).** Medians:
prefill −4.1 % (0 W / 3 L) and decode −2.1 % (1 W / 2 L); an earlier 5-round
block against the same effective baseline agreed on prefill (−2.7 %,
1 W / 4 L) with noisy decode. Side effect worth recording: the parallel
repack cuts model **load** wall time (load + tiny bench, warm page cache)
from 24 s (2/2 runs) to 10 s (2/2 runs) — but the inference phase itself
gets slower, so per the merge rule (real tok/s gains) the candidate is
**not** merged. Correctness: perplexity 1.0361 → 1.0361 (identical), i.e.
the parallel repack produces identical weights; the regression is presumably
a NUMA page-placement effect, not a data error. Measured 2026-10-16.

Round-by-round for draft #27032 (candidate vs baseline = current HEAD, t/s),
default settings. The draft bundles several optimizations (GET_ROWS+ADD
fusion, `get_rows` fast paths, tokenizer, KV-cell rework). The KV/batch and
tokenizer parts are either **not portable** (rooted in a pre-refactor
KV-cell design that conflicts with this tree) or not measurable with this
protocol; only the disjoint ggml CPU-kernel subset could be ported and
tested:

| round | pp512 | tg128 |
|---|---:|---:|
| 1 | 57.49 vs 56.64 | 10.73 vs 10.82 |
| 2 | 56.88 vs 58.68 | 10.85 vs 10.75 |
| 3 | 55.71 vs 56.50 | 10.35 vs 10.95 |

**Verdict ❌ 沒幫助 (not applicable to this model, not merged).** ≈±1 % on
both metrics, PPL identical (1.0361). The reason follows from the graph and
kernel dispatch: (1) the GET_ROWS+ADD fusion needs an ADD node immediately
after a GET_ROWS node — this model's graph has no such pattern; (2) the
optimized Q4_K dequant path is not used (`token_embd` is Q3_K); (3) the
remaining `get_rows` cache-hit path only helps repeated 1-row gathers, which
this workload barely has. Recorded for the record. Measured 2026-10-16.

Recorded verdict for draft #27396 (not ported, not measured): the draft
optimizes the *speculative draft-model prefill* path (zero-copy embedding,
event-based sync) for EAGLE3 / DFlash / GLM4-style draft heads, and it
touches core files (`llama-batch`, `llama-context`, `llama-graph`, ...) that
have diverged from its base — 7 of 17 files conflict and a faithful port
would require redesigning the speculative prefill plumbing. This CPU setup
does not use speculative decoding at all — the bundled MTP head measured
net-negative on CPU (see the inference section above) — so the draft has no
applicable surface here. Recorded 2026-10-16.

### Combined effect of the merged set

Interleaved A/B of the shipped tree (original port + all merged candidates)
against the original port, same session, 3 rounds each, medians:

| config | pp512 | tg128 |
|---|---:|---:|
| default, original port → shipped | 55.65 → 56.30 (**+1.2 %**, won 3/3) | 10.50 → 10.86 (**+3.4 %**, won 3/3) |
| `-nr`, original port → shipped | 60.96 → 64.87 (**+6.4 %**, won 3/3) | 10.20 → 10.26 (+0.6 %, noisy) |

The per-mechanism incremental numbers in the sections above are the
authoritative records; this table is the same data rolled up against the
original-port baseline in a single session. Note the separate, much larger
`-nr` vs default finding below.

### `-nr` vs default on this machine

Side observation from the re-anchoring session: on this AMX machine the
plain-CPU (`-nr`) path is clearly faster than the default repack path.
HEAD (all merged patches): pp512 56.93 → 66.35 (**+16.6 %**) and tg128
10.67 → 11.32 (**+6.1 %**) with `--repack 0` (3–4 interleaved rounds each,
2026-10-16). The repack kernels used by default have no AMX path, while the
plain Q4_K×Q8_K kernels use AMX (boosted further by #27024). On AMX-class
machines it is worth testing `-nr` for ~+15 % prefill.

## License

This project (the patch set, `patch.sh` and documentation) is licensed
under **GPL-3.0** — see [`LICENSE`](LICENSE). The patch is a derivative of
llama.cpp; the MIT-licensed portions remain under their original licenses,
with the references listed in the `LICENSE` header and repeated here:

- [llama.cpp](https://github.com/ggml-org/llama.cpp) — MIT — © ggml-org
  contributors (base commit `620d2225cc436ab0b001e3200e590e3b286ef9ab`)
- [mrexodia/llama.cpp-openPangu-2.0-Flash](https://github.com/mrexodia/llama.cpp-openPangu-2.0-Flash)
  — MIT — original openPangu-2.0-Flash port
- upstream PR [#27024](https://github.com/ggml-org/llama.cpp/pull/27024)
  by [@jinzihao](https://github.com/jinzihao) — MIT (merged)
- upstream PR [#22525](https://github.com/ggml-org/llama.cpp/pull/22525)
  by [@HyeongiJeon](https://github.com/HyeongiJeon) — MIT (merged)
- upstream PR [#27590](https://github.com/ggml-org/llama.cpp/pull/27590)
  by [@SongXiaoXi](https://github.com/SongXiaoXi) — MIT (merged; closed
  upstream unmerged, verified locally)
- [ggml](https://github.com/ggml-org/ggml) — MIT (bundled with llama.cpp)