htdy7703/gguf-tokenizer-array-type-confusion-poc

GGUF tokenizer metadata array type confusion PoC

This repository proves a narrow claim: loading a crafted GGUF whose tokenizer.ggml.scores metadata is stored as a 1-byte u8[] array instead of float32[] triggers a heap-buffer-overflow during vocabulary loading in llama.cpp.

• What this PoC shows:

  • the attacker controls GGUF tokenizer metadata
  • the model reaches the normal vocabulary-loading path
  • the loader reads 4 bytes from a 1-byte heap buffer in native code

• What this PoC does not show:

  • code execution
  • arbitrary file read or write
  • a broader bug beyond tokenizer metadata arrays that are blindly cast to typed pointers

• Verified commit:

  • 95e5254c0ae93529b2f6c05e210b9cac5c0070fc

Files

• build_wrong_typed_tokenizer_scores.py: helper to generate wrong-typed-tokenizer-scores-u8.gguf from a local llama.cpp checkout
• poc_vocab_only_loader.cpp: minimal vocab-loading harness using product code
• build_stubs.cpp: small helper stubs needed by the reduced harness build

Generate the malicious GGUF

python3 build_wrong_typed_tokenizer_scores.py /path/to/llama.cpp ./wrong-typed-tokenizer-scores-u8.gguf

That helper imports gguf-py from the provided llama.cpp checkout and therefore expects its Python dependencies to be available. It writes a metadata-only GGUF with:

• general.architecture = "llama"
• tokenizer.ggml.model = "t5"
• tokenizer.ggml.tokens = ["A"]
• tokenizer.ggml.scores = [65] stored as arr[u8,1]

Reproduce

Build a small ASAN/UBSAN-instrumented loader from product sources together with poc_vocab_only_loader.cpp and build_stubs.cpp, then run:

ASAN_OPTIONS=detect_leaks=0 \
./poc_vocab_only_loader ./wrong-typed-tokenizer-scores-u8.gguf

The expected result is an AddressSanitizer report showing:

• ERROR: AddressSanitizer: heap-buffer-overflow
• READ of size 4
• a stack including llama_vocab::impl::load

Notes

The adjacent tokenizer.ggml.token_type path is affected by the same unsafe assumption, but this PoC intentionally stays on the narrower scores proof path.