hacnho/gguf-normalizer-lowercase-poc

GGUF normalizer.lowercase scout - 2026-06-28

Purpose:

• test whether tokenizer.ggml.normalizer.lowercase opens a fresh GGUF metadata lane outside the already-submitted tokenizer key families

Candidate

Proposed title:

• GGUF tokenizer.ggml.normalizer.lowercase silently retokenizes mixed-case prompts in llama.cpp

Scout command:

• python3 targets/huntr/poc/gguf_normalizer_lowercase_probe.py

Artifacts:

• summary: targets/huntr/evidence/gguf-normalizer-lowercase-scout-20260628/summary.json
• patched model: targets/huntr/evidence/gguf-normalizer-lowercase-scout-20260628/llama-spm-lowercase.gguf

Local result

Using:

• tokenizer binary: targets/huntr/work/llama.cpp/build-hunt/bin/llama-tokenize
• seed vocab-only model: targets/huntr/work/llama.cpp/models/ggml-vocab-llama-spm.gguf

Patch:

• tokenizer.ggml.normalizer.lowercase = true

Observed on prompt Hello World:

• control IDs: [1, 15043, 2787]
• malicious IDs: [1, 274, 28059]

Meaning:

• the malicious GGUF file silently retokenizes mixed-case prompt text
• the attacker does not need to change the visible prompt string
• the behavioral change comes from the boolean metadata field itself

Duplicate posture

Authenticated exact-term dashboard gate on 2026-06-28:

• NONE of ['GGUF tokenizer.ggml.normalizer.lowercase silently retokenizes mixed-case prompts in llama.cpp', 'normalizer.lowercase', 'llama-spm-lowercase.gguf'] found on dashboard

Operational meaning:

• this looks exact-title clean versus the current local GGUF tokenizer-key cluster