fp8?
#2
by Joviex - opened
Making sure the links on the Model Card are supposed to work?
https://huggingface.co/shuttleai/shuttle-3.1-aesthetic-fp8
Doesnt point anywhere?
Cheers
Hello, we have not fully released the model yet, we are working on the fp8 and the gguf version! 😁
fp8 has been added, gguf will be soon
Hello,
Incredible work! Thanks!
How to use the fp8 with Diffusers? You have a working example?
I am also looking for FP8 with Diffusers demo. Not interested in Comfyi.
Hello @edtjulien @fullsoftwares
To run fp8 with diffusers, I usually just do
from optimum.quanto import freeze, quantize, qint8, qfloat8, qint4
quantize(
pipe.transformer,
# weights=qfloat8,
weights=qint8,
exclude=[
"*.norm", "*.norm1", "*.norm2", "*.norm2_context",
"proj_out", "x_embedder", "norm_out", "context_embedder",
],
)
freeze(pipe.transformer)
# pipe.enable_model_cpu_offload()
That's work like a charm. Thanks!
and with comfy, I just need to load ANY fp8 workflow and choose 4 steps in the sampler? That's all?