Xylolabs

GLM-4.6V-Flash-Text GGUF

Base model Formats License Code

GGUF quants of zai-org/GLM-4.6V-Flash for llama.cpp, on CUDA, Metal or CPU.

  • Files: IQ4_XS-imat.
  • IQ4_XS-imat: KL 0.0292 from BF16, top-1 91.79 %.
  • Which one to use: the smallest file whose KL you accept; see Files.

Files

  • GLM-4.6V-Flash-Text-<QUANT>.gguf is one file per quant.
  • Contents: Each file holds the full language model plus the original tokenizer and chat template.
Quant Size Mean KL vs BF16 Same top-1 vs BF16
IQ4_XS-imat 5.26 GB 0.0292 91.79%

How we measured:

  • Reference: our BF16 pass in transformers.
  • KL and top-1: 16,248 positions of English and Korean text.

How to run

Use llama.cpp commit cd959bae99ff4bed22bef2604f45241c09bbb8c8 or later.

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git checkout cd959bae99ff4bed22bef2604f45241c09bbb8c8
cmake -B build -DCMAKE_BUILD_TYPE=Release        # add -DGGML_CUDA=ON for NVIDIA GPUs
cmake --build build -j --target llama-cli llama-server

./build/bin/llama-cli -m GLM-4.6V-Flash-Text-IQ4_XS-imat.gguf --jinja -c 8192 -ngl 99

llama-server takes the same flags.

How these were made

  1. Convert the original checkpoint to a BF16 GGUF with convert_hf_to_gguf.py.
  2. Quantize each type with llama-quantize.
  3. The -imat files use an importance matrix.

License and notices

  • License: MIT License, copied in LICENSE.
  • What we changed: we converted the weights to GGUF and quantized them.
  • Notices: NOTICE names the source model and its copyright holder.
  • Xylolabs is not affiliated with or endorsed by the original model's authors.

Citation

The original model card recommends the GLM-V family paper:

@misc{glm_v_2025,
  title={GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning},
  author={V Team and others},
  year={2025},
  eprint={2507.01006},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2507.01006}
}
Downloads last month
604
GGUF
Model size
9B params
Architecture
glm4
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Xylolabs/GLM-4.6V-Flash-Text-GGUF

Quantized
(51)
this model

Collection including Xylolabs/GLM-4.6V-Flash-Text-GGUF

Paper for Xylolabs/GLM-4.6V-Flash-Text-GGUF