What is difference between fp8 scaled and mxfp8?
#4
by shivshankar - opened
Do 4090 support both?
MXFP8 is Blackwell only.
MXFP8 is block wise scaled, specifically with hardware level support in latest NVIDIA GPUs (Blackwell), it's of better quality.
On a 4090 this would mean it has to be dequanted on the fly, making it slower to run, but you will get the quality benefits regardless.
On a 50xx GPU it would be around ~30% faster than bf16.
How quality difference is there? Identical quality?