README.md

---
license: apache-2.0
tags:
- anime
- diffusion
- text-to-image
- image-generation
library_name: diffusers
pipeline_tag: text-to-image
language:
- en
---
![Aniimage-2 Samples](https://huggingface.co/8BitStudio/Aniimage-2/resolve/main/assets/collage.png)

# Aniimage-2

Aniimage-2 is the second public model in the Aniimage series.
The model is a 512x512 anime image generation model trained from scratch using a UNet + VAE + CLIP architecture with 441.9 million parameters.
Aniimage-2 has been trained on 1.2M anime images from [Danbooru](https://danbooru.donmai.us/) and other sources.
It is not based on any existing models, the UNet is trained from scratch.

## Model Details

| | |
|---|---|
| **Resolution** | 512x512 |
| **Architecture** | Latent Diffusion (UNet + VAE + CLIP) |
| **Parameters** | 441.9M |
| **Training Steps** | 70,000 |
| **Batch Size** | 64 |
| **Dataset** | ~1.2 million curated anime images from Danbooru |
| **GPU** | NVIDIA RTX 5060 Ti 16GB |
| **Scheduler** | DPM++ SDE |
| **Step count** | 50 using DPM++ SDE|

## Requirements

- **GPU**: ~6 GB VRAM minimum (recommend 8+ GB)
- **CPU**: ~7 GB RAM. Image generation is extremely slow on CPU. (recommend 10+ GB)

## Prompting

Aniimage uses plain text captions as well as Danbooru tags.

An example of acceptable captions:

"A smiling anime girl with red hair and a school uniform"

"1girl, solo, smile, red_hair, school_uniform, anime_coloring"

Long prompts and enhanced prompts usually result in worse overall results. Try to keep your prompts short.

Although you can use Danbooru tags, you will get the best results with plain text captions

Adding a negative prompt is required for a good output. The one used during inference in the preview image is
"low quality, ugly, blurry, distorted, deformed, bad anatomy, bad proportions, extra limbs, missing limbs, watermark, text, signature, washed out, flat colors, manga panel, disfigured, poorly drawn, jpeg artifacts, cropped, out of frame"

For the best quality, use DPM++ SDE at 50 steps.

**Important:** If you do not include “NSFW” in the negative prompt, the model may generate NSFW images


## Run

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/#fileId=https://huggingface.co/8BitStudio/Aniimage-2/blob/main/Aniimage_2_Interactive_Colab.ipynb)

[![Download Generator](https://img.shields.io/badge/Download-generate__images.py-blue?style=for-the-badge)](https://huggingface.co/8BitStudio/Aniimage-2/resolve/main/generate_images.py)

after downloading, install the dependencies.

```bash
pip install torch torchvision diffusers transformers safetensors pillow huggingface_hub
python generate_hf.py
```

## Capabilities

Aniimage-2 has significantly higher capabilities than [Aniimage-1](https://huggingface.co/8BitStudio/Aniimage-1). Especially when it comes to backgrounds.

<table width="700">
  <tr>
    <td colspan="2">
      <img
        src="https://huggingface.co/8BitStudio/Aniimage-2/resolve/main/assets/Softly%20lit%20forest%20at%20midday%2C%20peaceful%20atmosphere%2C%20no%20people.png"
        width="700"
        alt="Forest comparison between Aniimage-1 and Aniimage-2"
      >
    </td>
  </tr>

  <tr>
    <td align="center" width="50%">
      <b>Aniimage-1</b>
    </td>
    <td align="center" width="50%">
      <b>Aniimage-2</b>
    </td>
  </tr>

  <tr>
    <td colspan="2">
      <sub>Prompt: Softly lit forest at midday, peaceful atmosphere, no people.</sub>
    </td>
  </tr>
</table>

<table width="700">
  <tr>
    <td colspan="2">
      <img
        src="https://huggingface.co/8BitStudio/Aniimage-2/resolve/main/assets/anime-boy.png"
        width="700"
        alt="Anime boy comparison between Aniimage-1 and Aniimage-2"
      >
    </td>
  </tr>

  <tr>
    <td align="center" width="50%">
      <b>Aniimage-1</b>
    </td>
    <td align="center" width="50%">
      <b>Aniimage-2</b>
    </td>
  </tr>

Aniimage-2 produces male characters much more reliably than Aniimage-1.

  <tr>
    <td colspan="2">
      <sub>Prompt: An anime boy with black hair and green eyes.</sub>
    </td>
  </tr>
</table>

## Limitations

Aniimage-2 is bad at generating hands. It may generate hands but they often melt into surfaces if they are similar colors and usually have more or fewer than five fingers.
The model generates male characters less reliably than female characters, it may take many images to find a good image like above.

## What's Next

Aniimage-3 is currently in the works and will have a much higher parameter count: 738.1M parameters compared to aniimage-2 and aniimage-1's 441.9M parameters.
The model will also have 1.5 million unique images, 25% more than the 1.2 million Aniimage-2 has.
Aniimage-2 used incorrect Min-SNR loss weighting with its v-prediction objective, causing near-pure-noise timesteps to receive excessive emphasis. This has been corrected for Aniimage-3.
Aniimage-3 also uses a much higher-quality dataset with improved filtering. More than 100,000 low-quality images from the Aniimage-2 dataset were removed.


## License

Apache 2.0