request fp8 quantized model
#25
by brotherb74939 - opened
Can you provide a fp8 quantized model for qwen3-reranker-0.6B? The inference time is still too slow.
Or can you provide a quantize script or tutorial?
Thank you!
Can you provide a fp8 quantized model for qwen3-reranker-0.6B? The inference time is still too slow.
Or can you provide a quantize script or tutorial?
Thank you!