GGUF source model?

#8
by smhf72 - opened

Were the GGUFs created using the BF16 base model or the FP8 base model (I'm assuming BF16)?

Were the GGUFs created using the BF16 base model or the FP8 base model (I'm assuming BF16)?

BF16 of course, without the VAE and other extra layers, so it's just the transformer model.

'VAE' object has no attribute 'latent_frequency_bins' error using GGUF

'VAE' object has no attribute 'latent_frequency_bins' error using GGUF

@Kijai mentioned the GGUF is just the Transformer. You need to load the VAE separately.

And thanks for the info Kijai, just wanted to make sure Q8_0 would indeed be a precision upgrade from FP8.

smhf72 changed discussion status to closed

Sign up or log in to comment