tiny-kev-4b-for-testing-GGUF

This is a modified, damaged copy of jaredpalmer/kev-4b, made only to test software. Its output is gibberish. Do not use it for anything else.

GGUF of ggml-org-dev/tiny-kev-4b-for-testing merged into its base model ggml-org-dev/tiny-Qwen3.5-4B-Base-for-testing, converted with convert_hf_to_gguf.py of llama.cpp (F16, then quantized to Q4_0 with llama-quantize, token embedding included). Used by the llama.cpp server tests of the kev decision type.

The vocabulary of the GGUF is cut from 248320 to 3348 tokens (the HF slice keeps the full tokenizer): the special tokens, all single characters, the BPE path of the decision label words (A..Z, AA..ZZ, digits, yes/no) and the first 2000 merges are kept, the token ids are renumbered and the rows of the token embedding are sliced to match. Text is still fully encodable, but into more tokens than with the original tokenizer.

llama-server -hf ggml-org-dev/tiny-kev-4b-for-testing-GGUF:Q4_0

License

Derived from kev-4b by Jared Palmer, under the Apache License 2.0 (as is the base model Qwen3.5-4B-Base).

Downloads last month
-
GGUF
Model size
527k params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ggml-org-dev/tiny-kev-4b-for-testing-GGUF