kongbai-84/animagguf-quantizer

Animagguf Quantizer

This repository currently contains the Windows GGUF quantization tool and a comparison image only. It does not include Anima GGUF model weights.

Files

• AnimaggufQuantizer.exe - a Windows single-file GGUF conversion and quantization tool.
• image (20).png - comparison / preview image.

Usage

Important: AnimaggufQuantizer.exe can take a long time to open, especially on first launch, because it is packaged as a large single-file executable and needs to load or unpack bundled runtime files. Please wait patiently after double-clicking it.

Basic workflow:

1. Run AnimaggufQuantizer.exe.
2. Select an input model. The input must be either an original non-GGUF checkpoint, such as .safetensors, or an unquantized GGUF.
3. Select an output directory and a GGUF quantization format.
4. If the input is not GGUF, the tool first creates a BF16 GGUF in the output directory, then quantizes that GGUF.
5. If the input is an unquantized GGUF, the tool quantizes it directly.
6. Already quantized GGUF inputs are rejected to avoid double quantization.

The Q2_K_S option is intentionally not provided because the current quantization backend rejects it without an importance matrix.

Forge Neo Loader

For Forge Neo loading compatibility with Anima/Cosmos GGUF checkpoints, use:

https://github.com/kongbai-0421/animagguf-loader

Notes

No model weights are redistributed in this repository. Use the tool only with model files that you have the right to convert or quantize, and follow the license terms of the original model you process.