Efficient-Large-Model

/

NVILA-Lite-8B-hf-preview

like 0

Follow

Efficient-Large-Model 532

I'm looking into quantizing this model to int8 precision and I'm wondering if I should manually quantize the weights or use an automated technique like AWQ or bitsandbytes. Any recommendations on which method works best for this model? Thanks!