How is tiling happening here?
#3
by
nived2
- opened
The code in this repo just resizes the control image and is passed to the model without any tiling process. But the ultimate sd upscaler script using this model performs the tiling and invokes controlnet for each of the tiles. Is the diffusers code for doing upscaling in this way?
For anyone who is curios, this is what controlnet-tile actually does:
- It take in an upsample image from off-the-shelf upscaler (maybe by interpolation, GAN-based upscaler, etc.) as the condition and reconstruct the image with the same size but better detail.
- How would you do super resolution with this? By tiling your image yourself and upsample the patches first, then pass them through tile controlnet to get the detail enhanced image and patching them together, you will basically have your image super resolution"ed".
- To train a controlnet-tile, in this case most likely the author only does normal downscale and upscale again to create upsamle artifact, and the original image is the goal.