How did you quantize this one?
#1
by
JohnCe - opened
I tried quantizing myself with convert to quant and the result was utterly unusable. What is the secret sauce if you don't mind me asking?
I am using my script.
https://github.com/bedovyy/comfy-dit-quantizer
INT8 Tensorwise model lose quality quite a lot, so I kept blocks.0 and mlp.layer2 to BF16.
You can reproduce by,
- cloning git
- activate ComfyUI's venv
- run as below
python quantize.py configs/anima-int8.json <MODEL_PATH>/anima-preview.safetensors <MODEL_PATH>/anima-preview-int8.safetensors -m mse
By the way, you can try INT8 Rowwise model I just uploaded. (or this one https://huggingface.co/bertbobson/Anima-INT8-QUIP )
You can use it with the below custom node. (Quip branch)
https://github.com/BobJohnson24/ComfyUI-Flux2-INT8/tree/Quip
The quality is far better than Tensorwise and the generation speed is also fast enough.
You can reproduce INT8 Rowwise model by
python quantize.py configs/anima-int8rowwise.json <MODEL_PATH>/anima-preview.safetensors <MODEL_PATH>/anima-preview-int8rowwise.safetensors -m percentile