Will nvfp4 be coming?
#11
by
Ozzkozz - opened
Would you please add the nvfp4 model?
Hello! I’d like to suggest adding a quantized version of the model to the project using paroquant (https://github.com/z-lab/paroquant). This would greatly reduce model size and make it easier to run on CPU / small GPUs. Thanks!