ubs_autotest-4bit (GGUF Q4_K_M)

4-bit GGUF quantization of Catter58/ubs_autotest, a fine-tune of Qwen3.6-35B-A3B (MoE).

Files

File	Quant	Size
ubs_autotest-Q4_K_M.gguf	Q4_K_M	see repo

How it was made

python llama.cpp/convert_hf_to_gguf.py ./ubs_autotest --outfile ubs_autotest-bf16.gguf --outtype bf16
llama-quantize ubs_autotest-bf16.gguf ubs_autotest-Q4_K_M.gguf Q4_K_M

Architecture qwen35moe requires a recent llama.cpp build.

Usage

llama.cpp:

llama-cli -m ubs_autotest-Q4_K_M.gguf -p "Hello"

Ollama:

ollama run hf.co/Catter58/ubs_autotest-4bit:Q4_K_M

Note

The source model is multimodal (image-text-to-text). This GGUF contains the text weights only; the vision encoder is not included.