Add Tess-4.5-27B (dense) speed comparison beside MiniCPM

14b5896 verified 11 days ago

16.5 kB

---
license: mit
library_name: llama.cpp
tags:
- llama.cpp
- cuda
- volta
- v100
- sm_70
- gguf
---
# llama.cpp for NVIDIA Volta (sm_70)
Linux x86-64 binaries of llama.cpp built with CUDA 12.9 for NVIDIA Tesla V100 GPUs. CUDA 13.0 removed offline compilation and CUDA library support for Volta, so this build uses the CUDA 12.x toolchain. See the [CUDA 13.0 release notes](https://docs.nvidia.com/cuda/archive/13.0.0/cuda-toolkit-release-notes/).
**Status: released 2026-10-20 at the maintainer's decision, with one open caveat. All three fresh kernel suites (29,352 cases, 0 failures) and the model-agreement gate pass. One Q5_1 `MUL_MAT` failure from the first validation pass (NMSE 1.48× the limit) could not be replayed, because its input was not saved. Our internal requirement to capture and explain every failure is therefore not met, and no `BUILD-OK` marker was issued; see the Q5_1 caveat below.** The copies of this README and the release notes inside the archive were written at review time and call the build a review candidate. This page supersedes only that status line; the archive itself is unchanged.
## Source and build identity
The source is upstream [f74e2fa6a31cc33ab4ee4ae516641cfe62df3924](https://github.com/ggml-org/llama.cpp/commit/f74e2fa6a31cc33ab4ee4ae516641cfe62df3924), with two local patches. The build includes the upstream `qwen35moe` architecture.
- `source-local.patch` skips LoRA adapter matrix operations when the adapter's effective scale is zero, in ordinary, indexed/MoE, and embedding adapter paths in `src/llama-graph.cpp`.
- `sm70-flash-attention.patch` corrects CUDA flash-attention dispatch in `ggml/src/ggml-cuda/fattn.cu`. Volta MMA kernels require at least 32 columns; the patch prevents selection of the unsupported 16-column variant and falls through to the existing 32-column kernel. The triggering case has F16 K/V, head size 128, GQA ratio 8, KV length 5003, and query batch 4. The exact regression passed again, and all 12,012 supported Flash Attention cases across the three fresh full runs passed. This is a candidate upstream fix.
- Reported version: `0.7.0-dev (build 1, commit f74e2fa-modified)`.
- Build number `1` comes from the shallow source checkout's commit count; it is not an upstream release number.
- `source-local.patch` SHA256: `f1886f55c4bc527b6c801001f4d22d661fc7023d10a22c1eb3299e2680bfdee2`.
- `sm70-flash-attention.patch` SHA256: `fd61cc17fd0c902052900a576344ea6cfbb729280b838f6400466163f8be2cfa`.
- The initial source transfer was verified by identical relative paths and SHA256 hashes for all 3,825 files, excluding `.git` and directories named `build*`. This comparison included the LoRA patch and preceded the additional Volta dispatch patch.
- Initial source manifest SHA256 (before the Volta dispatch patch): `61baf1ca095793b2f32dece65dd4473d3dd849cf06ac687a300f629e9c69ce8f`.
- Compiler: GNU C/C++ 13.3.0 and NVIDIA CUDA compiler 12.9.86.
- Embedded CUDA machine code: `sm_70` only.

## Requirements

- Linux x86-64 with glibc 2.38 or newer and a compatible libstdc++ exposing `GLIBCXX_3.4.32` or newer.
- CPU support for SSE4.2, AVX, AVX2, FMA, F16C, and BMI2. `GGML_NATIVE=OFF` avoids host-specific tuning but does not disable these instruction sets.
- A Volta GPU with sufficient VRAM for the selected model and context. The numerical validation target is a Tesla V100 PCIe 32 GB.
- For a toolkit-matched deployment, use a Volta-compatible NVIDIA driver at least `575.57.08`, the Linux driver baseline listed for [CUDA 12.9 Update 1](https://docs.nvidia.com/cuda/archive/12.9.1/cuda-toolkit-release-notes/index.html). Validation uses `580.178.04`. NVIDIA identifies R580 as the last branch supporting Volta; a newer branch is not automatically compatible. See [NVIDIA's CUDA 13.0 guidance](https://developer.nvidia.com/blog/whats-new-and-important-in-cuda-toolkit-13-0/).
- External CUDA 12 runtime and cuBLAS libraries, preferably those supplied with CUDA 12.9: `libcudart.so.12`, `libcublas.so.12`, and its dependency `libcublasLt.so.12`. The NVIDIA driver supplies `libcuda.so.1`. These libraries must be discoverable by the system loader; CUDA 13 libraries do not replace the required CUDA 12 libraries.
- Standard Linux runtime libraries, including libgomp for OpenMP.

NCCL is disabled. OpenSSL was unavailable during compilation, so HTTPS support is disabled. Use local model files; the examples below bind the HTTP server to the local machine.

## Contents and use

The package layout is `bin/` for executables and `lib/` for llama.cpp shared libraries, plus license notices. It includes `llama-server`, `llama-cli`, `llama-bench`, `llama-quantize`, `llama-perplexity`, and `test-backend-ops`. Keep `bin/` and `lib/` together when relocating the package.

To inspect the review archive and verify its checksum:

```bash
sha256sum -c SHA256SUMS
tar --zstd -xf llama.cpp-f74e2fa-cuda12.9-sm70-linux-x64.tar.zst
./bin/llama-server --version
```

Place a separately obtained model in `./models/`. For example:

```bash
CUDA_VISIBLE_DEVICES=0 ./bin/llama-server \
  -m ./models/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
  -ngl 99 -c 4096 -fa on -ctk f16 -ctv f16 \
  --host 127.0.0.1 --port 8080
```

Or run a text prompt:

```bash
CUDA_VISIBLE_DEVICES=0 ./bin/llama-cli \
  -m ./models/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
  -ngl 99 -c 4096 -fa on -ctk f16 -ctv f16 \
  --temp 0 --seed 20261016 -n 64 \
  -p 'Explain why the sky appears blue.'
```

Select the intended GPU with `CUDA_VISIBLE_DEVICES`. The model is not included in the binary package. These examples use text input; no image, audio, video, or projector validation is claimed.

## Validation

**Fresh numerical tests pass. One historical Q5_1 failure remains unexplained (see the Q5_1 caveat).**

The tested model is `MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf`, the Q4_K_M quantization of MiniCPM-V 4.7 35B-A3B. Its size is 21,166,255,008 bytes and its freshly verified SHA256 is `9ecf91e814d7e9adc64325b1b65f8628a9b1bb8da4f61a92869ca7489bfd888e`, matching the [publisher’s file checksum](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf/blob/main/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf). Its GGUF architecture is `qwen35moe`. These checks cover text inference only; they do not establish compatibility with other models or multimodal inputs.

| Model agreement check | Measured result | Acceptance limit | Result |
| --- | ---: | ---: | --- |
| Mean KLD | **0.000597** | ≤ 0.001 | Pass |
| 99th-percentile KLD | **0.016200** | Report only | Informational |
| Same top token (“same top p”) | **99.510%**; 1,015/1,020 positions | ≥ 99.0% | Pass |
| CPU reference perplexity | **1.155534** | Reference | — |
| V100 perplexity | **1.154694** | Compare with CPU | — |
| Absolute relative PPL difference | **0.072694%** | ≤ 0.5% | Pass |

The comparison uses the included deterministic original English text: four 512-token windows and 1,020 scored positions, batch 512, microbatch 128, Flash Attention enabled, F16 K/V caches, and seed 20261016. The CPU reference disables GPU and operation offload; the V100 offloads all 41/41 layers. The base file is generated by `--kl-divergence-base` and compared using `--kl-divergence`. Its scaled 16-bit log-probability format and the upstream KLD estimator are described in `REVALIDATION.md`. This small, low-perplexity corpus is a numerical check, not a model-quality benchmark. The initial four-decimal PPL display gave a 0.0692% difference; the values above use the more precise output from the paired KLD run.

Three complete fresh backend runs used the same operation-family filter and no parameter filter. Each executed 9,784 supported cases, with zero failures and 1,793 unsupported combinations. The interrupted partial attempts are excluded. The table totals all three complete runs; fusion variants are grouped with their corresponding operation.

| CUDA operation family | Passed | Failed | Unsupported/skipped |
| --- | ---: | ---: | ---: |
| MUL_MAT | 11,754 | 0 | 1,260 |
| MUL_MAT_ID | 2,886 | 0 | 42 |
| FLASH_ATTN_EXT | 12,012 | 0 | 4,017 |
| SSM_CONV | 405 | 0 | 0 |
| GATED_DELTA_NET | 144 | 0 | 0 |
| RMS_NORM | 159 | 0 | 0 |
| ROPE | 1,350 | 0 | 60 |
| SOFT_MAX | 642 | 0 | 0 |
| **Total** | **29,352** | **0** | **5,379** |

The fresh MUL_MAT failure fraction is **0/11,754**; the Q5_1 subset is **0/168**. Unsupported combinations are not counted as passes or executed-case denominator entries. Each run uses fresh `std::random_device` tensor values. Identical CSV hashes across clean runs reflect the fact that CSV records parameters and outcomes, not random values. The separate capture harness links the unchanged release libraries and inserts capture calls only when a comparison fails. No new failures occurred, so there were no new inputs requiring replay.

**Q5_1 caveat.** The initial run recorded one Q5_1/F32 `MUL_MAT` failure at `m=16, n=1, k=256`: NMSE **0.000740110** against **0.000500000**, or **1.48022×** the limit. Including the initial full run and the three fresh runs gives **1/15,672** supported MUL_MAT cases, below the proposed pooled 0.1% limit. That does not prove its cause: the original random input was not saved and cannot be replayed exactly on CPU. CPU/CUDA quantization arithmetic differences are plausible contributors, but input conditioning has not been established for that failure. The earlier 64 captured diagnostics all passed. Separately reported Q5_1 repetitions passed 1,120 V100 cases and 280 sm_86 cases; those reports are not pooled into the fresh counts above. The tested model contains Q4_K, Q6_K, and F32 tensors, with no Q5_1 weights. The written requirement to retain and explain every failure therefore remains unmet; passing random retries cannot repair the missing original evidence.
**Greedy caveat.** The initial 64-token prompt comparisons first diverged at token **26** for reasoning and **12** for prose; the code prompt matched all **64** tokens. Positions are one-based and informational, not a gate. CPU and GPU floating-point summation order can change near-tied token choices, so greedy sequences can diverge even when aggregate agreement remains within tolerance.
The exact former Flash Attention crash case passed again using the unpacked release executable, and all 12,012 supported cases of the full family passed across the three fresh runs. The patch keeps Volta away from an unsupported 16-column MMA dispatch and uses the existing supported 32-column path.
All 18 unpacked ELF files match the tested build. All six tools resolve dependencies and pass their supported version or usage probes with `LD_LIBRARY_PATH` unset and no CUDA compiler in `PATH`. The machine has a toolkit installed, so this verifies relocation and runtime-only dependencies, not a separately provisioned runtime-only OS image. The package contains no CUDA runtime libraries, model weights, private execution logs, or lease records.
`REVALIDATION.json` and `REVALIDATION.md` contain the current results and protocol. `VALIDATION.json` preserves the initial results, including the superseded greedy-equality decision. The internal release gate remains false because the original Q5_1 input is unavailable. The build was published at the maintainer's decision with that caveat stated; no success marker is bundled or implied.
## Speed benchmarks
Measured on one Tesla V100 PCIe 32 GB (driver 580.178.04), with the card reserved for the run. The model is MiniCPM-V 4.7 35B-A3B Q4_K_M with all layers offloaded and Flash Attention on.
`llama-bench -ngl 99 -fa 1 -r 3`, in tokens/s:
| Test | Tokens/s |
| --- | ---: |
| pp512 | 837.51 ± 3.49 |
| pp2048 | 827.25 ± 5.39 |
| pp8192 | 813.15 ± 4.07 |
| tg128 | 110.86 ± 0.88 |
`llama-server` `/completion` with `n_predict` 512, `ignore_eos`, and the prompt cache off. Values are prompt processing / generation in tokens/s; the prompt size is the actual token count:
| Prompt | No draft | With DSpark draft | Draft acceptance |
| --- | ---: | ---: | ---: |
| 714 tokens | 625.7 / 102.4 | 572.4 / 121.0 | 49.8% (mean length 2.49) |
| 5,614 tokens | 898.9 / 100.2 | 840.3 / 106.1 | 40.9% (mean length 2.22) |
| 16,814 tokens | 888.3 / 96.9 | 837.8 / 86.2 | 29.3% (mean length 1.88) |
Peak VRAM was 21.4 GB without a draft and 23.7 GB with it; the peak GPU temperature was 51 °C. The draft file is the publisher's `dspark-MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf`, passed with `-md … -ngld 99`; the server auto-detects it as `draft-dspark` block drafting. `--spec-type draft-mtp` is rejected for this model because it has no MTP layers. The draft helps at short contexts and costs throughput at long ones.
### Second model: Tess-4.5-27B (dense) on the same card
Same card, binaries and method. The model is the publisher's `Tess-4.5-27B-Q4_K_M.gguf` from [migtissera/Tess-4.5-27B-GGUF](https://huggingface.co/migtissera/Tess-4.5-27B-GGUF) (Apache-2.0), a dense `qwen35` 27B. Its draft is the publisher's MTP head, `mtp-Tess-4.5-27B-Q4_K_M.gguf`, run with `--spec-type draft-mtp -md … -ngld 99`. It is a speed comparison only; the numerical validation above covers MiniCPM.
`llama-bench`, in tokens/s, with MiniCPM for comparison:
| Test | Tess-4.5-27B (dense) | MiniCPM-V 4.7 35B-A3B (MoE) | Tess / MiniCPM |
| --- | ---: | ---: | ---: |
| pp512 | 859.73 ± 30.46 | 837.51 ± 3.49 | 1.03 |
| pp2048 | 854.14 ± 6.84 | 827.25 ± 5.39 | 1.03 |
| pp8192 | 806.09 ± 2.32 | 813.15 ± 4.07 | 0.99 |
| tg128 | 33.04 ± 0.33 | 110.86 ± 0.88 | 0.30 |
`llama-server` `/completion` for Tess, prompt processing / generation in tokens/s:
| Prompt | No draft | With MTP draft | MTP acceptance |
| --- | ---: | ---: | ---: |
| 714 tokens | 582.2 / 32.7 | 469.5 / 66.1 | 88.3% (mean length 3.65) |
| 5,614 tokens | 797.8 / 31.1 | 737.0 / 69.0 | 96.7% (mean length 3.90) |
| 16,814 tokens | 715.3 / 26.7 | 695.8 / 63.0 | 90.3% (mean length 3.70) |
Peak VRAM was 18.5 GB without a draft and 22.0 GB with it; the peak temperature was 71 °C. Prompt processing is similar for the two models. Generation is about 3.3× slower for the dense model, which reads all 27B weights per token, while the MoE reads about 3B active ones. Tess's MTP head raises its generation 2.0-2.4×, which closes part of that gap.
## Build configuration
The following reproduces the compiler settings, using generic paths. Apply both `source-local.patch` and `sm70-flash-attention.patch` to the exact upstream commit before configuring. `$src` must resolve to that source directory.
```bash
src="$(pwd)"
cuda=/usr/local/cuda-12.9
export PATH="$cuda/bin:$PATH"
export CUDACXX="$cuda/bin/nvcc"
export CUDA_HOME="$cuda"
export CUDA_VISIBLE_DEVICES=''
nice -n 15 cmake -S "$src" -B "$src/build-volta" -G Ninja \
  -DCMAKE_CUDA_COMPILER="$cuda/bin/nvcc" -DCUDAToolkit_ROOT="$cuda" \
  -DGGML_CUDA=ON -DGGML_CUDA_NCCL=OFF -DCMAKE_CUDA_ARCHITECTURES=70 \
  -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF \
  -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DGGML_CCACHE=OFF \
  -DLLAMA_BUILD_NUMBER=1 -DLLAMA_BUILD_COMMIT=f74e2fa-modified \
  '-DCMAKE_BUILD_RPATH=$ORIGIN;$ORIGIN/../lib' \
  '-DCMAKE_INSTALL_RPATH=$ORIGIN;$ORIGIN/../lib' \
  -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=OFF \
  "-DCMAKE_C_FLAGS=-ffile-prefix-map=$src=llama.cpp" \
  "-DCMAKE_CXX_FLAGS=-ffile-prefix-map=$src=llama.cpp" \
  "-DCMAKE_CUDA_FLAGS=-Xcompiler=-ffile-prefix-map=$src=llama.cpp"
nice -n 15 cmake --build "$src/build-volta" --parallel 40 --target \
  llama-server llama-cli llama-bench llama-quantize llama-perplexity test-backend-ops
```
`LLAMA_CURL=OFF` is retained from the requested build configuration; this source revision deprecates that option. HTTPS availability is controlled by OpenSSL detection, as noted above. Prefix mapping removes the original source location from compiler-generated file strings. Build and install RPATH both use `$ORIGIN` and `$ORIGIN/../lib`.
## License
llama.cpp is distributed under the MIT License, copyright (c) 2023-2026 The ggml authors. Include and retain the complete upstream `LICENSE` with copies or substantial portions of the software. The software is provided without warranty under that license.
Bundled third-party components retain their respective licenses and notices; preserve the accompanying `THIRD-PARTY-NOTICES`. NVIDIA runtime libraries and separately downloaded models are governed by their own licenses and are not relicensed by this package.
GitHub release: https://github.com/Atlas3DSS/llama.cpp/releases/tag/sm70-cuda12.9-f74e2fa

	---
	license: mit
	library_name: llama.cpp
	tags:
	- llama.cpp
	- cuda
	- volta
	- v100
	- sm_70
	- gguf
	---

	# llama.cpp for NVIDIA Volta (sm_70)

	Linux x86-64 binaries of llama.cpp built with CUDA 12.9 for NVIDIA Tesla V100 GPUs. CUDA 13.0 removed offline compilation and CUDA library support for Volta, so this build uses the CUDA 12.x toolchain. See the [CUDA 13.0 release notes](https://docs.nvidia.com/cuda/archive/13.0.0/cuda-toolkit-release-notes/).

	Status: released 2026-10-20 at the maintainer's decision, with one open caveat. All three fresh kernel suites (29,352 cases, 0 failures) and the model-agreement gate pass. One Q5_1 `MUL_MAT` failure from the first validation pass (NMSE 1.48× the limit) could not be replayed, because its input was not saved. Our internal requirement to capture and explain every failure is therefore not met, and no `BUILD-OK` marker was issued; see the Q5_1 caveat below. The copies of this README and the release notes inside the archive were written at review time and call the build a review candidate. This page supersedes only that status line; the archive itself is unchanged.

	## Source and build identity

	The source is upstream [f74e2fa6a31cc33ab4ee4ae516641cfe62df3924](https://github.com/ggml-org/llama.cpp/commit/f74e2fa6a31cc33ab4ee4ae516641cfe62df3924), with two local patches. The build includes the upstream `qwen35moe` architecture.

	- `source-local.patch` skips LoRA adapter matrix operations when the adapter's effective scale is zero, in ordinary, indexed/MoE, and embedding adapter paths in `src/llama-graph.cpp`.
	- `sm70-flash-attention.patch` corrects CUDA flash-attention dispatch in `ggml/src/ggml-cuda/fattn.cu`. Volta MMA kernels require at least 32 columns; the patch prevents selection of the unsupported 16-column variant and falls through to the existing 32-column kernel. The triggering case has F16 K/V, head size 128, GQA ratio 8, KV length 5003, and query batch 4. The exact regression passed again, and all 12,012 supported Flash Attention cases across the three fresh full runs passed. This is a candidate upstream fix.

	- Reported version: `0.7.0-dev (build 1, commit f74e2fa-modified)`.
	- Build number `1` comes from the shallow source checkout's commit count; it is not an upstream release number.
	- `source-local.patch` SHA256: `f1886f55c4bc527b6c801001f4d22d661fc7023d10a22c1eb3299e2680bfdee2`.
	- `sm70-flash-attention.patch` SHA256: `fd61cc17fd0c902052900a576344ea6cfbb729280b838f6400466163f8be2cfa`.
	- The initial source transfer was verified by identical relative paths and SHA256 hashes for all 3,825 files, excluding `.git` and directories named `build*`. This comparison included the LoRA patch and preceded the additional Volta dispatch patch.
	- Initial source manifest SHA256 (before the Volta dispatch patch): `61baf1ca095793b2f32dece65dd4473d3dd849cf06ac687a300f629e9c69ce8f`.
	- Compiler: GNU C/C++ 13.3.0 and NVIDIA CUDA compiler 12.9.86.
	- Embedded CUDA machine code: `sm_70` only.

	## Requirements

	- Linux x86-64 with glibc 2.38 or newer and a compatible libstdc++ exposing `GLIBCXX_3.4.32` or newer.
	- CPU support for SSE4.2, AVX, AVX2, FMA, F16C, and BMI2. `GGML_NATIVE=OFF` avoids host-specific tuning but does not disable these instruction sets.
	- A Volta GPU with sufficient VRAM for the selected model and context. The numerical validation target is a Tesla V100 PCIe 32 GB.
	- For a toolkit-matched deployment, use a Volta-compatible NVIDIA driver at least `575.57.08`, the Linux driver baseline listed for [CUDA 12.9 Update 1](https://docs.nvidia.com/cuda/archive/12.9.1/cuda-toolkit-release-notes/index.html). Validation uses `580.178.04`. NVIDIA identifies R580 as the last branch supporting Volta; a newer branch is not automatically compatible. See [NVIDIA's CUDA 13.0 guidance](https://developer.nvidia.com/blog/whats-new-and-important-in-cuda-toolkit-13-0/).
	- External CUDA 12 runtime and cuBLAS libraries, preferably those supplied with CUDA 12.9: `libcudart.so.12`, `libcublas.so.12`, and its dependency `libcublasLt.so.12`. The NVIDIA driver supplies `libcuda.so.1`. These libraries must be discoverable by the system loader; CUDA 13 libraries do not replace the required CUDA 12 libraries.
	- Standard Linux runtime libraries, including libgomp for OpenMP.

	NCCL is disabled. OpenSSL was unavailable during compilation, so HTTPS support is disabled. Use local model files; the examples below bind the HTTP server to the local machine.

	## Contents and use

	The package layout is `bin/` for executables and `lib/` for llama.cpp shared libraries, plus license notices. It includes `llama-server`, `llama-cli`, `llama-bench`, `llama-quantize`, `llama-perplexity`, and `test-backend-ops`. Keep `bin/` and `lib/` together when relocating the package.

	To inspect the review archive and verify its checksum:

	```bash
	sha256sum -c SHA256SUMS
	tar --zstd -xf llama.cpp-f74e2fa-cuda12.9-sm70-linux-x64.tar.zst
	./bin/llama-server --version
	```

	Place a separately obtained model in `./models/`. For example:

	```bash
	CUDA_VISIBLE_DEVICES=0 ./bin/llama-server \
	-m ./models/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
	-ngl 99 -c 4096 -fa on -ctk f16 -ctv f16 \
	--host 127.0.0.1 --port 8080
	```

	Or run a text prompt:

	```bash
	CUDA_VISIBLE_DEVICES=0 ./bin/llama-cli \
	-m ./models/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
	-ngl 99 -c 4096 -fa on -ctk f16 -ctv f16 \
	--temp 0 --seed 20261016 -n 64 \
	-p 'Explain why the sky appears blue.'
	```

	Select the intended GPU with `CUDA_VISIBLE_DEVICES`. The model is not included in the binary package. These examples use text input; no image, audio, video, or projector validation is claimed.

	## Validation

	Fresh numerical tests pass. One historical Q5_1 failure remains unexplained (see the Q5_1 caveat).

	The tested model is `MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf`, the Q4_K_M quantization of MiniCPM-V 4.7 35B-A3B. Its size is 21,166,255,008 bytes and its freshly verified SHA256 is `9ecf91e814d7e9adc64325b1b65f8628a9b1bb8da4f61a92869ca7489bfd888e`, matching the [publisher’s file checksum](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf/blob/main/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf). Its GGUF architecture is `qwen35moe`. These checks cover text inference only; they do not establish compatibility with other models or multimodal inputs.

	\| Model agreement check \| Measured result \| Acceptance limit \| Result \|
	\| --- \| ---: \| ---: \| --- \|
	\| Mean KLD \| 0.000597 \| ≤ 0.001 \| Pass \|
	\| 99th-percentile KLD \| 0.016200 \| Report only \| Informational \|
	\| Same top token (“same top p”) \| 99.510%; 1,015/1,020 positions \| ≥ 99.0% \| Pass \|
	\| CPU reference perplexity \| 1.155534 \| Reference \| — \|
	\| V100 perplexity \| 1.154694 \| Compare with CPU \| — \|
	\| Absolute relative PPL difference \| 0.072694% \| ≤ 0.5% \| Pass \|

	The comparison uses the included deterministic original English text: four 512-token windows and 1,020 scored positions, batch 512, microbatch 128, Flash Attention enabled, F16 K/V caches, and seed 20261016. The CPU reference disables GPU and operation offload; the V100 offloads all 41/41 layers. The base file is generated by `--kl-divergence-base` and compared using `--kl-divergence`. Its scaled 16-bit log-probability format and the upstream KLD estimator are described in `REVALIDATION.md`. This small, low-perplexity corpus is a numerical check, not a model-quality benchmark. The initial four-decimal PPL display gave a 0.0692% difference; the values above use the more precise output from the paired KLD run.

	Three complete fresh backend runs used the same operation-family filter and no parameter filter. Each executed 9,784 supported cases, with zero failures and 1,793 unsupported combinations. The interrupted partial attempts are excluded. The table totals all three complete runs; fusion variants are grouped with their corresponding operation.

	\| CUDA operation family \| Passed \| Failed \| Unsupported/skipped \|
	\| --- \| ---: \| ---: \| ---: \|
	\| MUL_MAT \| 11,754 \| 0 \| 1,260 \|
	\| MUL_MAT_ID \| 2,886 \| 0 \| 42 \|
	\| FLASH_ATTN_EXT \| 12,012 \| 0 \| 4,017 \|
	\| SSM_CONV \| 405 \| 0 \| 0 \|
	\| GATED_DELTA_NET \| 144 \| 0 \| 0 \|
	\| RMS_NORM \| 159 \| 0 \| 0 \|
	\| ROPE \| 1,350 \| 0 \| 60 \|
	\| SOFT_MAX \| 642 \| 0 \| 0 \|
	\| Total \| 29,352 \| 0 \| 5,379 \|

	The fresh MUL_MAT failure fraction is 0/11,754; the Q5_1 subset is 0/168. Unsupported combinations are not counted as passes or executed-case denominator entries. Each run uses fresh `std::random_device` tensor values. Identical CSV hashes across clean runs reflect the fact that CSV records parameters and outcomes, not random values. The separate capture harness links the unchanged release libraries and inserts capture calls only when a comparison fails. No new failures occurred, so there were no new inputs requiring replay.

	Q5_1 caveat. The initial run recorded one Q5_1/F32 `MUL_MAT` failure at `m=16, n=1, k=256`: NMSE 0.000740110 against 0.000500000, or 1.48022× the limit. Including the initial full run and the three fresh runs gives 1/15,672 supported MUL_MAT cases, below the proposed pooled 0.1% limit. That does not prove its cause: the original random input was not saved and cannot be replayed exactly on CPU. CPU/CUDA quantization arithmetic differences are plausible contributors, but input conditioning has not been established for that failure. The earlier 64 captured diagnostics all passed. Separately reported Q5_1 repetitions passed 1,120 V100 cases and 280 sm_86 cases; those reports are not pooled into the fresh counts above. The tested model contains Q4_K, Q6_K, and F32 tensors, with no Q5_1 weights. The written requirement to retain and explain every failure therefore remains unmet; passing random retries cannot repair the missing original evidence.

	Greedy caveat. The initial 64-token prompt comparisons first diverged at token 26 for reasoning and 12 for prose; the code prompt matched all 64 tokens. Positions are one-based and informational, not a gate. CPU and GPU floating-point summation order can change near-tied token choices, so greedy sequences can diverge even when aggregate agreement remains within tolerance.

	The exact former Flash Attention crash case passed again using the unpacked release executable, and all 12,012 supported cases of the full family passed across the three fresh runs. The patch keeps Volta away from an unsupported 16-column MMA dispatch and uses the existing supported 32-column path.

	All 18 unpacked ELF files match the tested build. All six tools resolve dependencies and pass their supported version or usage probes with `LD_LIBRARY_PATH` unset and no CUDA compiler in `PATH`. The machine has a toolkit installed, so this verifies relocation and runtime-only dependencies, not a separately provisioned runtime-only OS image. The package contains no CUDA runtime libraries, model weights, private execution logs, or lease records.

	`REVALIDATION.json` and `REVALIDATION.md` contain the current results and protocol. `VALIDATION.json` preserves the initial results, including the superseded greedy-equality decision. The internal release gate remains false because the original Q5_1 input is unavailable. The build was published at the maintainer's decision with that caveat stated; no success marker is bundled or implied.

	## Speed benchmarks

	Measured on one Tesla V100 PCIe 32 GB (driver 580.178.04), with the card reserved for the run. The model is MiniCPM-V 4.7 35B-A3B Q4_K_M with all layers offloaded and Flash Attention on.

	`llama-bench -ngl 99 -fa 1 -r 3`, in tokens/s:

	\| Test \| Tokens/s \|
	\| --- \| ---: \|
	\| pp512 \| 837.51 ± 3.49 \|
	\| pp2048 \| 827.25 ± 5.39 \|
	\| pp8192 \| 813.15 ± 4.07 \|
	\| tg128 \| 110.86 ± 0.88 \|

	`llama-server` `/completion` with `n_predict` 512, `ignore_eos`, and the prompt cache off. Values are prompt processing / generation in tokens/s; the prompt size is the actual token count:

	\| Prompt \| No draft \| With DSpark draft \| Draft acceptance \|
	\| --- \| ---: \| ---: \| ---: \|
	\| 714 tokens \| 625.7 / 102.4 \| 572.4 / 121.0 \| 49.8% (mean length 2.49) \|
	\| 5,614 tokens \| 898.9 / 100.2 \| 840.3 / 106.1 \| 40.9% (mean length 2.22) \|
	\| 16,814 tokens \| 888.3 / 96.9 \| 837.8 / 86.2 \| 29.3% (mean length 1.88) \|

	Peak VRAM was 21.4 GB without a draft and 23.7 GB with it; the peak GPU temperature was 51 °C. The draft file is the publisher's `dspark-MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf`, passed with `-md … -ngld 99`; the server auto-detects it as `draft-dspark` block drafting. `--spec-type draft-mtp` is rejected for this model because it has no MTP layers. The draft helps at short contexts and costs throughput at long ones.

	### Second model: Tess-4.5-27B (dense) on the same card

	Same card, binaries and method. The model is the publisher's `Tess-4.5-27B-Q4_K_M.gguf` from [migtissera/Tess-4.5-27B-GGUF](https://huggingface.co/migtissera/Tess-4.5-27B-GGUF) (Apache-2.0), a dense `qwen35` 27B. Its draft is the publisher's MTP head, `mtp-Tess-4.5-27B-Q4_K_M.gguf`, run with `--spec-type draft-mtp -md … -ngld 99`. It is a speed comparison only; the numerical validation above covers MiniCPM.

	`llama-bench`, in tokens/s, with MiniCPM for comparison:

	\| Test \| Tess-4.5-27B (dense) \| MiniCPM-V 4.7 35B-A3B (MoE) \| Tess / MiniCPM \|
	\| --- \| ---: \| ---: \| ---: \|
	\| pp512 \| 859.73 ± 30.46 \| 837.51 ± 3.49 \| 1.03 \|
	\| pp2048 \| 854.14 ± 6.84 \| 827.25 ± 5.39 \| 1.03 \|
	\| pp8192 \| 806.09 ± 2.32 \| 813.15 ± 4.07 \| 0.99 \|
	\| tg128 \| 33.04 ± 0.33 \| 110.86 ± 0.88 \| 0.30 \|

	`llama-server` `/completion` for Tess, prompt processing / generation in tokens/s:

	\| Prompt \| No draft \| With MTP draft \| MTP acceptance \|
	\| --- \| ---: \| ---: \| ---: \|
	\| 714 tokens \| 582.2 / 32.7 \| 469.5 / 66.1 \| 88.3% (mean length 3.65) \|
	\| 5,614 tokens \| 797.8 / 31.1 \| 737.0 / 69.0 \| 96.7% (mean length 3.90) \|
	\| 16,814 tokens \| 715.3 / 26.7 \| 695.8 / 63.0 \| 90.3% (mean length 3.70) \|

	Peak VRAM was 18.5 GB without a draft and 22.0 GB with it; the peak temperature was 71 °C. Prompt processing is similar for the two models. Generation is about 3.3× slower for the dense model, which reads all 27B weights per token, while the MoE reads about 3B active ones. Tess's MTP head raises its generation 2.0-2.4×, which closes part of that gap.

	## Build configuration

	The following reproduces the compiler settings, using generic paths. Apply both `source-local.patch` and `sm70-flash-attention.patch` to the exact upstream commit before configuring. `$src` must resolve to that source directory.

	```bash
	src="$(pwd)"
	cuda=/usr/local/cuda-12.9
	export PATH="$cuda/bin:$PATH"
	export CUDACXX="$cuda/bin/nvcc"
	export CUDA_HOME="$cuda"
	export CUDA_VISIBLE_DEVICES=''

	nice -n 15 cmake -S "$src" -B "$src/build-volta" -G Ninja \
	-DCMAKE_CUDA_COMPILER="$cuda/bin/nvcc" -DCUDAToolkit_ROOT="$cuda" \
	-DGGML_CUDA=ON -DGGML_CUDA_NCCL=OFF -DCMAKE_CUDA_ARCHITECTURES=70 \
	-DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF \
	-DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DGGML_CCACHE=OFF \
	-DLLAMA_BUILD_NUMBER=1 -DLLAMA_BUILD_COMMIT=f74e2fa-modified \
	'-DCMAKE_BUILD_RPATH=$ORIGIN;$ORIGIN/../lib' \
	'-DCMAKE_INSTALL_RPATH=$ORIGIN;$ORIGIN/../lib' \
	-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=OFF \
	"-DCMAKE_C_FLAGS=-ffile-prefix-map=$src=llama.cpp" \
	"-DCMAKE_CXX_FLAGS=-ffile-prefix-map=$src=llama.cpp" \
	"-DCMAKE_CUDA_FLAGS=-Xcompiler=-ffile-prefix-map=$src=llama.cpp"

	nice -n 15 cmake --build "$src/build-volta" --parallel 40 --target \
	llama-server llama-cli llama-bench llama-quantize llama-perplexity test-backend-ops
	```

	`LLAMA_CURL=OFF` is retained from the requested build configuration; this source revision deprecates that option. HTTPS availability is controlled by OpenSSL detection, as noted above. Prefix mapping removes the original source location from compiler-generated file strings. Build and install RPATH both use `$ORIGIN` and `$ORIGIN/../lib`.

	## License

	llama.cpp is distributed under the MIT License, copyright (c) 2023-2026 The ggml authors. Include and retain the complete upstream `LICENSE` with copies or substantial portions of the software. The software is provided without warranty under that license.

	Bundled third-party components retain their respective licenses and notices; preserve the accompanying `THIRD-PARTY-NOTICES`. NVIDIA runtime libraries and separately downloaded models are governed by their own licenses and are not relicensed by this package.

	GitHub release: https://github.com/Atlas3DSS/llama.cpp/releases/tag/sm70-cuda12.9-f74e2fa