Handle the color drift for multi-pass setting

#1
by MinhNH232331M - opened

While using the FlowUpscaler, I notice some problem about color drifting, as also happen with this demo space. The results are some what saturated and happening to have lower red and blue color level than the original image. After some engineering with the help of Claude Code, I've create a pipeline to resolve this problem:

  • Correct the low frequency band of the latent produced by FlowUpscaler by the low frequency band of the conditioning latent. This result in a sharper output, which can be interpolated to control the sharpness of the output. This is applied per pass.
  • Add another correction stage at the final decoding step in the pixel space, by correct the low band frequency of the output image in the Lab space.

Comparison with and without pixel-space color fix:

compare_color_fix

Comparison with and without latent transplant:

compare_latent_transplant

You can copy the pipeline from here: https://huggingface.co/MinhNH232331M/FlowUpscaler-diffusers

Note: The unet_upscaler in my repo has been modified for memory optimization, so the result won't be bit identical to the original implementation because of floating point truncation as well as operation orders.

MinhNH232331M changed discussion status to closed
MinhNH232331M changed discussion status to open

Sign up or log in to comment