import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("Kameshr/flux-pipeline", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]LoRA-Based Text-to-Image Diffusion Model
This model is a LoRA-based text-to-image diffusion model with quantization and is specifically optimized for environments with 16 GB RAM like Google Colab. It uses LoRA for lightweight fine-tuning and quantization to reduce memory demands.
Model Overview
- Model Type: Text-to-Image Diffusion
- Optimization: LoRA + Quantization
- Precision: Half-precision (float16) with 4-bit quantization to reduce memory footprint.
- Memory Requirements: Designed for 16 GB RAM with CPU offloading capabilities.
Key Features
- LoRA (Low-Rank Adaptation): Allows efficient fine-tuning without large memory overhead.
- 4-bit Quantization: Reduces memory usage while maintaining model quality.
- CPU Offloading: Enables stable performance within memory constraints by offloading parts of the model to the CPU.
Usage Instructions
- Environment: Use in Google Colab (16 GB RAM recommended).
- Inference: Run text-to-image generation using a simple text prompt.
- Memory Management: To prevent memory issues, utilize CPU offloading and periodically clear the cache.
This model setup is optimized for straightforward, memory-efficient inference on Colab. Ideal for users working in constrained environments.
Colab Notebook for Reference
To get started with the model, you can refer to this Colab Notebook for a full guide and hands-on demonstration.
- Downloads last month
- 10