| --- |
| license: cc-by-nc-4.0 |
| library_name: diffusers |
| --- |
| |
| It's simple upscaler using AsymmetricAutoencoderKL. I was playing around with code used for training in the middle of it a lot so it's nothing scientific. I was just pleased with results from something that easy to train. |
|
|
| For optimizers, training was done with AdEMAMix optimizer, dataset of ~4k images mostly including photos, digital art and small amount of PBR textures. I did some finetuning with same dataset, but Adopt optimizer with OrthoGrad from <a href="https://arxiv.org/abs/2501.04697" target="_blank"><i>Grokking at the Edge of Numerical Stability</i></a> (arXiv: 2501.04697). Model was trained at 96px x 96px resolution (so 192px x 192ox output). |
|
|
| For loss, I was using most of the time simple HSL loss (1 - cosine of difference between target and pred H and L1 loss for S and L channels), LPIPS+ and DISTS. |
|
|
| Model have issues with handling jpeg artifacts because I couldn't train it on random compression levels due to lack of support of ROCm by torchvision.transforms.v2.JPEG. In this case it's better to scale down image a bit before upscaling. |
|
|
| This is some proof of concept model. It can't be used commercially as is, but there is a chance that I'll train new version on some CC0 dataset with license permiting commercial usage and with better jpeg artifacts handling in future. |
|
|
| You can run model using code below |
|
|
| ``` |
| import torch |
| |
| from torchvision import transforms, utils |
| |
| import diffusers |
| from diffusers import AsymmetricAutoencoderKL |
| |
| from diffusers.utils import load_image |
| |
| def crop_image_to_nearest_divisible_by_8(img): |
| # Check if the image height and width are divisible by 8 |
| if img.shape[1] % 8 == 0 and img.shape[2] % 8 == 0: |
| return img |
| else: |
| # Calculate the closest lower resolution divisible by 8 |
| new_height = img.shape[1] - (img.shape[1] % 8) |
| new_width = img.shape[2] - (img.shape[2] % 8) |
| |
| # Use CenterCrop to crop the image |
| transform = transforms.Compose([transforms.CenterCrop((new_height, new_width)) , transforms.ToTensor]) |
| img = transform(img).to(torch.float32).clamp(-1, 1) |
| |
| return img |
| |
| vae = AsymmetricAutoencoderKL.from_pretrained("Heasterian/AsymmetricAutoencoderKLUpscaler", weight_dtype=torch.float32) |
| vae.requires_grad_(False) |
| |
| image = load_image(r"/home/heasterian/test/a/F8VlGmCWEAAUVpc (copy).jpeg") |
| |
| image = crop_image_to_nearest_divisible_by_8(image).unsqueeze(0) |
| |
| upscaled_image = vae(image).sample |
| # Save the reconstructed image |
| utils.save_image(upscaled_image, "test.png") |
| ``` |
