File size: 6,510 Bytes
5df6a54 4893387 5df6a54 4893387 5df6a54 4893387 5df6a54 4893387 ed9da68 5df6a54 4893387 5df6a54 ed9da68 5df6a54 4893387 5df6a54 ed9da68 05452d6 4893387 05452d6 b8c4b36 5df6a54 b8c4b36 06ee509 b8c4b36 4893387 c1b761b b8c4b36 4893387 b8c4b36 5df6a54 5ce1cca 4893387 5ce1cca 4893387 5ce1cca 5df6a54 5ce1cca e426b4c a26e48a 5df6a54 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | ---
base_model:
- circlestone-labs/Anima
license: other
license_name: circlestone-labs-non-commercial-license
license_link: LICENSE.md
tags:
- anima
- decoder
---
# Anima Pixel Decoder
A 25M-parameter decoder that reads the last-layer features of the
[Anima](https://huggingface.co/circlestone-labs/Anima) DiT at the final sampling step and
writes RGB pixels directly, in place of the VAE decode. The head and its training follow
[FastFrames](https://arxiv.org/html/2611.23226)' pixel head (a ConvNeXt readout on the diffusion transformer's features).
| | |
|---|---|
| File | `anima_pixel_head.safetensors` (fp32, 95 MB; shape in the file's metadata) |
| Input | the input of the DiT's `final_layer.linear` (2048-dim per token, after the adaLN-modulated LayerNorm) |
| Output | a 16×16 RGB patch per token, in [-1, 1] |
| Architecture | linear projection of the feature + linear embedding of the x_t patch, 2 ConvNeXt, one 2× pixel-shuffle upsample, 1 ConvNeXt, linear to 8×8×3 |
| DiT | `anima-base-v1.0` (28 blocks) |
| Target | base points: the original image, with LPIPS (weight 0.05); turbo point: VAE(x̂0) of the same forward, x̂0 = x_t − σ·v |
## Compatibility
Read the head only where it was trained:
- **Base DiT**: Euler, shift 3, 20 … 50 steps (last-step σ 0.058 … 0.136), read at CFG 4,
Spectrum included (its last step is an actual forward).
- **Turbo**: the 4-step turbo LoRA of
[anima-turbo-4step](https://huggingface.co/sorryhyun/anima-turbo-4step), Euler 4 steps,
shift 3, CFG 1 (last-step σ 0.5). Trained on `anima_turbo_4step_v4_longer.safetensors`.
The previous head is `old/anima_pixel_head.safetensors` (VAE(x̂0) target at every point, turbo
point on that repo's `old/anima_turbo_4step.safetensors`).
## Benchmark
The benchmarks, samples and held-out reads below were measured on the previous head
(`old/anima_pixel_head.safetensors`).
### In ComfyUI
Turbo 4 steps, CFG 1, RTX 5070 Ti, 16 prompts at ~1 MP, one seed
| | KSampler + VAE Decode | KSampler + Anima Pixel Decode |
|---|---|---|
| KSampler | 0.976 s | 0.998 s (with the tap) |
| decode | 151 ms | 7 ms |
| decode peak VRAM above the loaded models | 1.22 GB | 0.21 GB |
| PSNR against VAE Decode | — | 36.9 dB |
| **total per image** | **1.13 s** | **1.00 s (1.12×)** |
ComfyUI keeps the VAE on the card after its first decode, so the VAE column has no move cost.
### In anima_lora
The same turbo setting and prompts through anima_lora's `inference.py` (compiled DiT blocks);
medians. The VAE column is the usual interactive path: the VAE waits on the CPU and goes to the
card for the decode.
| | turbo + VAE | turbo + pixel head |
|---|---|---|
| DiT sampling | 0.97 s | 0.97 s |
| load VAE | 710 ms | — |
| VAE move to the card and back | 35 ms | — |
| decode | 141 ms | 7 ms |
| decode peak VRAM above the DiT | 1.38 GB | 0.24 GB |
| **total per image (DiT + decode)** | **1.86 s** | **0.98 s** |
## Samples
Turbo 4 steps, CFG 1, seed 42, 1024×1024, in ComfyUI (the workflow above); left VAE Decode,
right Anima Pixel Decode, decoded from the same latent. Prompt: `masterpiece, best quality,
score_7, safe. An anime girl wearing a black tank-top and denim shorts is standing outdoors.
She's holding a rectangular sign out in front of her that reads "ANIMA". She's looking at the
viewer with a smile. The background features some trees and blue sky with clouds.`; the
monochrome pair adds `monochrome, comic` after `safe`. Crops: 384×384 at native resolution,
x 192–576, y 512–896.
| | VAE Decode | pixel head |
|---|---|---|
| colour|  |  |
| crop |  |  |
| monochrome |  |  |
| crop |  |  |
On the monochrome crop the VAE keeps the screentone dots on the shorts; the head renders them as
flat grey (see Known limits).
## Held-out reads
Held-out reads during training (dB, colour / mono):
| decode point | colour | mono |
|---|---|---|
| turbo 4 steps | 35.90 | 29.01 |
| base 20 steps | 33.71 | 26.58 |
| base 28 steps | 33.83 | 26.38 |
| base 50 steps | 33.49 | 25.48 |
| base 28 + Spectrum | 34.37 | 26.00 |
## Known limits
- Screentone dots are smoothed into soft grey shading; textured regions (screentone,
dense hatching) come out about 1 – 2.5 levels darker than the VAE in linear light.
The lowest per-image PSNRs are manga pages and sketches (turbo 28.7 – 30.9 dB).
- Speed lines and footnote-size glyphs are slightly soft under zoom.
- One trained head, one seed; no run spread.
## Use
- ComfyUI: the [ComfyUI-Anima-PixelDecoder](https://github.com/sorryhyun/ComfyUI-Anima-PixelDecoder)
node pack (ComfyUI Manager: "Anima Pixel Decoder"). Put this file in `models/pixel_heads/`,
add **Anima Pixel Head Tap** on the MODEL before the KSampler (stock or Spectrum) and decode
the sampler's LATENT with **Anima Pixel Decode** instead of VAE Decode. Run the DiT on the
fp32 stream (Anima Adapter Loader, `stream = fp32`).
- [anima_lora](https://github.com/sorryhyun/anima_lora): `inference.py --decoder pixel_head`
(`make test PIXEL=1`, `make test-turbo PIXEL=1`); the module, loader and feature tap are in
`library/anima/pixel_head.py`.
- ×2 (test): `anima_pixel_head_x2_preview.safetensors` decodes and upscales 2× in one
readout (Anima Pixel Decode `scale = 2`, `inference.py --pixel_scale 2`). A test file; it will
be replaced.
## License
The weights in this repository are a **Derivative** of
[Anima](https://huggingface.co/circlestone-labs/Anima) and are distributed under the
**[CircleStone Labs Non-Commercial License](LICENSE.md)** (see [NOTICE.md](NOTICE.md)). Any
rights to use them are granted directly by CircleStone Labs LLC under that license.
- The model/weights may only be used for non-commercial purposes. Generated images (Outputs)
may be used commercially.
- Anima is itself a Derivative Model of NVIDIA Cosmos-Predict2-2B-Text2Image and is subject
to the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/)
insofar as it applies to Derivative Models.
- This is not an official CircleStone Labs product and is not endorsed by CircleStone Labs.
Built on NVIDIA Cosmos.
|