FP8 quant?

#8
by gdevenyi - opened

Any plans to release an FP8 quant (or an NVFP4)?

We're working on the official quant, but meanwhile there is already a community version: https://huggingface.co/sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4.
Actually there are multiple 😀 https://huggingface.co/morosystems/ThinkingCap-Qwen3.6-27B-NVFP4

BottleCapAI org

Just published the FP8 Version.

Amazing! Thanks!

gdevenyi changed discussion status to closed

Sign up or log in to comment