chris320211/flatquant-phi3-mini-4k-g52xlarge
Text Generation • Updated • 613
QuantPort library
Quantized LLMs made and measured with QuantPort (github.com/chris320211/QuantPort). Rankings live in its library/LIBRARY.md.
Note FlatQuant · microsoft/Phi-3-mini-4k-instruct · g5.2xlarge · WikiText-2 ppl_ratio=1.1676
chris320211/smoothquant-granite33-2b-g52xlarge
Text Generation • Updated • 542Note SmoothQuant · ibm-granite/granite-3.3-2b-instruct · g5.2xlarge · WikiText-2 ppl_ratio=1.0060
chris320211/smoothquant-qwen25-15b-g52xlarge
Text Generation • 2B • Updated • 523Note SmoothQuant · Qwen/Qwen2.5-1.5B-Instruct · g5.2xlarge · WikiText-2 ppl_ratio=1.0509
chris320211/smoothquant-gemma2-2b-g52xlarge
Text Generation • 3B • Updated • 371Note SmoothQuant · google/gemma-2-2b-it · g5.2xlarge · WikiText-2 ppl_ratio=1.0215
chris320211/smoothquant-qwen25-15b-g52xlarge-vllm
Text Generation • 2B • UpdatedNote vLLM build of chris320211/smoothquant-qwen25-15b-g52xlarge · SmoothQuant · Qwen/Qwen2.5-1.5B-Instruct · vLLM ppl_ratio=1.0148 on NVIDIA GeForce RTX 3090
chris320211/smoothquant-gemma2-2b-g52xlarge-vllm
Text Generation • 3B • UpdatedNote vLLM build of chris320211/smoothquant-gemma2-2b-g52xlarge · SmoothQuant · google/gemma-2-2b-it · vLLM ppl_ratio=1.0285 on NVIDIA GeForce RTX 3090