| # 🪄 Affordance-based Novel Concept Generator (Kandinsky-3 Fine-Tuned) | |
| This is a fine-tuned version of the **Kandinsky-3** text-to-image pipeline, designed to generate **novel object and furniture concepts** by combining affordance-driven functionalities (e.g., "sofa + bed + cargo + bicycle"). | |
| --- | |
| ## 🚀 How to Use | |
| ```python | |
| import os | |
| import sys | |
| import torch | |
| from kandinsky3 import get_T2I_pipeline, get_T2I_Flash_pipeline | |
| # Add kandinsky3 to Python path | |
| sys.path.append('..') | |
| # Set device and dtype maps | |
| device_map = torch.device('cuda:0') | |
| dtype_map = { | |
| 'unet': torch.float32, | |
| 'text_encoder': torch.float32, | |
| 'movq': torch.float32, | |
| } | |
| # Load the Flash text-to-image pipeline | |
| t2i_pipe = get_T2I_Flash_pipeline( | |
| device_map=device_map, | |
| dtype_map=dtype_map, | |
| cache_dir="./cache/" | |
| ) | |
| # Load fine-tuned UNet weights | |
| t2i_pipe.unet.load_state_dict(torch.load( | |
| "unet_model_checkpoint.pt", | |
| map_location=device_map | |
| )) | |
| # Generate image from prompt | |
| res = t2i_pipe( | |
| text="a new furniture design that has functions from sofa, bed, cargo, bicycle", | |
| steps=50 | |
| )[0] | |
| # Save the result | |
| res.save("generated_image.jpg") | |
| ``` |