NVFP4?
#1
by
Remile
- opened
Could you please provide the NVFP4 format? Much appreciated!
I tried, but the quality dropped significantly and the generation speed decreased. (I don't understand why, btw)
If you want to boost the generation speed, try "EasyCache" node.
This node really sped up my generation in some seeds, but EasyCache's speed boost is still nowhere near as good as nvfp4. The quality drop might just be because this model is currently only a preview version? Anyway, thanks for the quantized model and the node recommendation!
I have updated nvfp4mixed version. That was the limit I can acccept between quality and size.
And it is actually slower than fp8 version when you torch.compile.
Anyway, I tried my best.
Bedovyy
changed discussion status to
closed