Guidance embedding influence
#8
by sd0809 - opened
Thanks for the awesome work!!!
I want to know the image genertion ability with true cfg, so I tried to load the model with the model_config "guidence_embed": false to remove the effect of the guidance embedding. But raise error: 'CombinedTimestepTextProjEmbeddings' object has no attribute 'guidance_embedder'.
So how should I to inference the model without guidance embedding?
@sd0809 hey! You can use the code from ostris ai-toolkit to do it like this:
from functools import partial
def guidance_embed_bypass_forward(self, timestep, guidance, pooled_projection):
timesteps_proj = self.time_proj(timestep)
timesteps_emb = self.timestep_embedder(
timesteps_proj.to(dtype=pooled_projection.dtype)) # (N, D)
pooled_projections = self.text_embedder(pooled_projection)
conditioning = timesteps_emb + pooled_projections
return conditioning
def bypass_flux_guidance(transformer):
if hasattr(transformer.time_text_embed, '_bfg_orig_forward'):
return
# dont bypass if it doesnt have the guidance embedding
if not hasattr(transformer.time_text_embed, 'guidance_embedder'):
return
transformer.time_text_embed._bfg_orig_forward = transformer.time_text_embed.forward
transformer.time_text_embed.forward = partial(
guidance_embed_bypass_forward, transformer.time_text_embed
)
bypass_flux_guidance(pipeline.transformer)