BF16/FP8 version availability?
#77
by
JFLOR274
- opened
Hi Comunity,
We've successfully deployed the INT4 version but our infrastructure cannot handle 2TB for BF16 dequantization attempts.
Questions:
- Are there plans to release an official FP8 quantized version?
- Can BF16/FP32 checkpoints be made available for enterprise users?
Any guidance would be appreciated.
Thanks!
@JFLOR274 whats your deployment stack and hardware infra?
I think there are a lot of details from that question, so could you be more specific about the details you want?