--- license: mit tags: - image-colorization - gan - computer-vision - pytorch - onnx library_name: pytorch --- # DeOldify Model Weights This repository contains pretrained weights for **DeOldify**, a deep learning model for colorizing and restoring old black and white images and videos. **Original Repository**: [thookham/DeOldify](https://github.com/thookham/DeOldify) **Original Author**: Jason Antic ([jantic/DeOldify](https://github.com/jantic/DeOldify)) ## Model Overview DeOldify uses a Self-Attention Generative Adversarial Network (SAGAN) with a novel **NoGAN** training approach to achieve stable, high-quality colorization without the typical GAN artifacts. ### Three Specialized Models 1. **Artistic** - Highest quality with vibrant colors and interesting details - Best for: General images, historical photos - Backbone: ResNet34 U-Net - Training: 5 NoGAN cycles, 32% ImageNet 2. **Stable** - Best for portraits and landscapes, reduced artifacts - Best for: Faces, nature scenes - Backbone: ResNet101 U-Net - Training: 3 NoGAN cycles, 7% ImageNet 3. **Video** - Optimized for smooth, flicker-free video - Best for: Video colorization, consistency - Backbone: ResNet101 U-Net - Training: Initial cycle only, 2.2% ImageNet ## Available Files ### ONNX Models (Browser/Inference) | File | Size | Description | |------|------|-------------| | `deoldify-art.onnx` | 243 MB | Artistic model in ONNX format for browser use | | `deoldify-quant.onnx` | 61 MB | Quantized artistic model (75% smaller, slightly lower quality) | ### PyTorch Weights (Training & Inference) **Generator Weights** (Main): - `ColorizeArtistic_gen.pth` (243 MB) - `ColorizeStable_gen.pth` (834 MB) - `ColorizeVideo_gen.pth` (834 MB) **Critic Weights** (Main): - `ColorizeArtistic_crit.pth` (361 MB) - `ColorizeStable_crit.pth` (361 MB) - `ColorizeVideo_crit.pth` (361 MB) **PretrainOnly Weights** (For continued training): - `ColorizeArtistic_PretrainOnly_gen.pth` (729 MB) - `ColorizeArtistic_PretrainOnly_crit.pth` (1.05 GB) - `ColorizeStable_PretrainOnly_crit.pth` (1.05 GB) - `ColorizeVideo_PretrainOnly_crit.pth` (1.05 GB) > **Note**: Stable and Video PretrainOnly generators are split files hosted on [GitHub Releases](https://github.com/thookham/DeOldify/releases/tag/v2.0-models). ## Usage ### Browser (ONNX) ```html ``` ### PyTorch (Python) ```python from huggingface_hub import hf_hub_download import torch # Download model weights model_path = hf_hub_download( repo_id="thookham/DeOldify", filename="ColorizeArtistic_gen.pth" ) # Load weights (requires deoldify package installed) # See GitHub repository for full usage examples ``` ### Installation ```bash # Clone the main repository git clone https://github.com/thookham/DeOldify cd DeOldify # Install dependencies pip install -r requirements.txt # Download a model from huggingface_hub import hf_hub_download model = hf_hub_download(repo_id="thookham/DeOldify", filename="ColorizeStable_gen.pth") ``` ## Technical Details ### Architecture - **Generator**: U-Net with ResNet34/101 backbone, spectral normalization, self-attention layers - **Critic**: PatchGAN discriminator - **Loss**: Perceptual loss (VGG16) + GAN loss ### NoGAN Training A novel training approach that combines: 1. Generator pretraining with feature loss 2. Critic pretraining on generated images 3. Short GAN training (30-60 minutes) at inflection point 4. Optional cycle repeats for more colorful results This eliminates typical GAN artifacts while maintaining realistic colorization. ### Training Data - Dataset: ImageNet subsets (1-32% depending on model) - Resolution: 192px during training - Augmentation: Gaussian noise for video stability ## Model Card ### Model Details - **Developed by**: Jason Antic (original), Travis Hookham (modernization) - **Model type**: Conditional GAN for image-to-image translation - **Language(s)**: N/A (computer vision) - **License**: MIT - **Parent Model**: Based on FastAI U-Net and Self-Attention GAN papers ### Intended Use **Primary Use**: Colorizing black and white photographs and videos **Out-of-Scope**: Real-time processing, guaranteed historical accuracy ### Limitations - Colors may not be historically accurate - Performance degrades on very low quality/damaged images - Artistic model may require render_factor tuning - Video model trades some color vibrancy for consistency ## Related Models & Resources ### Similar Colorization Models on Hugging Face **GAN-based Colorization:** - [Hammad712/GAN-Colorization-Model](https://huggingface.co/Hammad712/GAN-Colorization-Model) - GAN model for grayscale to color transformation - [jessicanono/filparty_colorization](https://huggingface.co/jessicanono/filparty_colorization) - ResNet-based model for historical photos **Stable Diffusion-based:** - [rsortino/ColorizeNet](https://huggingface.co/rsortino/ColorizeNet) - ControlNet adaptation of SD 2.1 for colorization - [AlekseyCalvin/ColorizeTruer_KontextFluxVar6_BySAP](https://huggingface.co/AlekseyCalvin/ColorizeTruer_KontextFluxVar6_BySAP) - Advanced Flux-based colorization **Interactive Demos (Spaces):** - [aryadytm/Photo-Colorization](https://huggingface.co/spaces/aryadytm/Photo-Colorization) - [Shashank009/Black-And-White-Image-Colorization](https://huggingface.co/spaces/Shashank009/Black-And-White-Image-Colorization) - [CA611/Image-Colorization](https://huggingface.co/spaces/CA611/Image-Colorization) ### Why Choose DeOldify? DeOldify stands out for: - **NoGAN Training**: Unique approach eliminating typical GAN artifacts - **Specialized Models**: Three purpose-built models (Artistic, Stable, Video) - **Video Support**: Flicker-free temporal consistency - **Proven Track Record**: Powers MyHeritage InColor and widely adopted - **ONNX Support**: Browser-ready models for offline use ## Citation If you use these models, please cite: ```bibtex @misc{deoldify, author = {Antic, Jason}, title = {DeOldify}, year = {2019}, publisher = {GitHub}, url = {https://github.com/jantic/DeOldify} } ``` ## Links - **GitHub Repository**: https://github.com/thookham/DeOldify - **Original DeOldify**: https://github.com/jantic/DeOldify - **MyHeritage InColor** (Commercial version): https://www.myheritage.com/incolor - **Demo (Browser)**: See browser/ folder in GitHub repo ## License MIT License. See [LICENSE](https://github.com/thookham/DeOldify/blob/master/LICENSE) file.