llama.cpp f74e2fa for NVIDIA Volta (sm_70), CUDA 12.9 — released 2026-10-20 with the Q5_1 caveat stated
0b864c5 verified | # Re-validation protocol | |
| `REVALIDATION.json` records the current decision. `VALIDATION.json` preserves the | |
| initial run, including its unsaved Q5_1 failure and its superseded greedy-equality | |
| gate. Neither record omits a failed case. | |
| The model is MiniCPM-V 4.7 35B-A3B Q4_K_M, SHA256 | |
| `9ecf91e814d7e9adc64325b1b65f8628a9b1bb8da4f61a92869ca7489bfd888e`. | |
| The fixed original-text corpus has SHA256 | |
| `68c3808a113f74773f6c3c278173171358f2402d4e38e116ae6a03aca0263055`. | |
| This is a numerical comparison over four 512-token windows, scoring 255 positions | |
| per window, or 1,020 positions total. It is not a representative model-quality | |
| benchmark and covers text inference only. | |
| ## Model agreement | |
| Use the same executable, model, corpus, context, batch, microbatch, cache formats, | |
| and Flash Attention setting for both commands. `$BUILD_DIR` is the build | |
| directory, `$MODEL` the local model file, and `$CPU_SET` an assigned set of at most | |
| 40 logical CPUs. The GPU command requires separately arranged GPU admission and | |
| an appropriate `CUDA_VISIBLE_DEVICES`. | |
| ```sh | |
| CUDA_VISIBLE_DEVICES='' nice -n 15 taskset -c "$CPU_SET" \ | |
| "$BUILD_DIR/bin/llama-perplexity" \ | |
| -m "$MODEL" -c 512 -b 512 -ub 128 -t 40 -tb 40 \ | |
| -fa on -ctk f16 -ctv f16 --fit off --seed 20261016 \ | |
| --no-context-shift --verbosity 4 \ | |
| -ngl 0 --device none --no-op-offload \ | |
| -f validation-corpus.txt --chunks 4 \ | |
| --kl-divergence-base "$BUILD_DIR/reference-logits.bin" | |
| nice -n 15 taskset -c "$CPU_SET" "$BUILD_DIR/bin/llama-perplexity" \ | |
| -m "$MODEL" -c 512 -b 512 -ub 128 -t 40 -tb 40 \ | |
| -fa on -ctk f16 -ctv f16 --fit off --seed 20261016 \ | |
| --no-context-shift --verbosity 4 -ngl 999 --device CUDA0 \ | |
| -f validation-corpus.txt --chunks 4 \ | |
| --kl-divergence-base "$BUILD_DIR/reference-logits.bin" --kl-divergence | |
| ``` | |
| The acceptance limits are mean KLD ≤ 0.001, same-top agreement ≥ 99.0%, and | |
| absolute relative perplexity difference ≤ 0.5%. The 99th-percentile KLD is | |
| reported without a separate cutoff. “Same top p” is the fraction of scored | |
| positions with the same highest-probability token, not a sampling top-p setting. | |
| The upstream base file stores scaled 16-bit log probabilities rather than | |
| uncompressed FP32 logits. Its KLD estimate includes base probabilities above | |
| `exp(-16)`; tiny negative values can result from representation and arithmetic | |
| error. Quantiles use the upstream tool's linear interpolation. The measured | |
| perplexity difference uses the base and candidate values from the same KLD run; | |
| it is more precise than subtracting the initial four-decimal PPL display. | |
| Greedy first-divergence positions from the initial three 64-token prompts remain | |
| informational: reasoning 26, prose 12, and no divergence for code. Positions are | |
| one-based. CPU and GPU floating-point summation order can change near-tied token | |
| choices, so greedy sequences can diverge despite passing aggregate limits. | |
| ## Kernel repetitions and failure evidence | |
| Run three complete repetitions with no parameter filter: | |
| ```sh | |
| test-backend-ops -b CUDA0 \ | |
| -o 'MUL_MAT.*,FLASH_ATTN_EXT,SSM_CONV.*,GATED_DELTA_NET.*,RMS_NORM,ROPE,SOFT_MAX' \ | |
| --output csv -j 1 | |
| ``` | |
| The re-validation uses a separate copy of this upstream test harness with capture | |
| calls inserted only at comparison failures. The production executables, shared | |
| libraries, CUDA kernels, random initialization, numerical comparisons, and | |
| tolerances remain unchanged. Each repetition has a distinct capture prefix. | |
| Source and binary hashes are recorded in the current result. | |
| Supported rows with no error are passes. Unsupported combinations are reported | |
| separately and excluded from the executed-case denominator. Unexpected process | |
| exits, missing families, incomplete output, nonfinite results, and failures | |
| outside quantized MUL_MAT edge cases fail the gate. Random inputs come from | |
| `std::random_device`; separate invocations generate fresh tensors. | |
| Any allowed numerical edge case must have NMSE below twice its own threshold, | |
| an observed MUL_MAT failure fraction below 0.001, retained exact operands, and a | |
| CPU replay with enough evidence to establish its cause. CPU replay alone does | |
| not prove input conditioning or rule out a CUDA-specific arithmetic defect. | |
| An independent high-precision calculation, and where applicable the same-input | |
| comparison on another architecture, are needed to support that attribution. | |
| Observed pooled failure fractions are sample statistics, not guarantees about | |
| every quantization, tensor shape, model, or input. | |
| The historical Q5_1 failure was 0.000740110 against 0.000500000, or 1.48022 times | |
| the threshold. Its original input was not saved. Passing subsequent random | |
| cases cannot retroactively reproduce or establish the cause of that exact | |
| failure. The release decision explicitly accounts for this limitation. | |
| ## Flash Attention and relocation | |
| The exact former crash case is: | |
| ```sh | |
| test-backend-ops -b CUDA0 -o FLASH_ATTN_EXT \ | |
| -p '^hsk=128,hsv=128,nh=1,nr23=\[8,1\],kv=5003,nb=4,' --output csv -j 1 | |
| ``` | |
| It is run with the unmodified packaged test executable. All three complete | |
| repetitions also include the entire FLASH_ATTN_EXT family. | |
| Relocation checks compare all 18 unpacked ELF files to the tested build, resolve | |
| dependencies for all six tools, and run their supported version or usage probes | |
| with `LD_LIBRARY_PATH` unset and a runtime-only `PATH`. The host has a CUDA | |
| toolkit installed; this verifies runtime library dependencies and independence | |
| from the original build directory, not a separately provisioned runtime-only OS | |
| image. No compiler is needed to launch the package. | |
| Raw execution logs, machine-specific plans, workboard records, and captured | |
| tensor files remain private and are not package contents. | |