anima
decoder
File size: 6,510 Bytes
5df6a54
 
 
 
 
 
 
 
 
 
 
4893387
5df6a54
 
 
 
4893387
5df6a54
 
 
 
4893387
5df6a54
4893387
 
ed9da68
5df6a54
 
4893387
5df6a54
 
 
 
 
 
 
ed9da68
 
 
 
5df6a54
4893387
5df6a54
ed9da68
 
 
05452d6
 
4893387
05452d6
 
 
 
 
 
 
 
 
 
 
 
 
 
b8c4b36
 
5df6a54
b8c4b36
06ee509
b8c4b36
4893387
c1b761b
 
b8c4b36
4893387
b8c4b36
5df6a54
5ce1cca
 
 
 
 
 
 
 
 
 
 
 
4893387
5ce1cca
4893387
5ce1cca
 
 
 
 
 
 
5df6a54
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5ce1cca
 
e426b4c
 
 
 
 
 
a26e48a
 
 
5df6a54
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
---
base_model:
- circlestone-labs/Anima
license: other
license_name: circlestone-labs-non-commercial-license
license_link: LICENSE.md
tags:
- anima
- decoder
---

# Anima Pixel Decoder

A 25M-parameter decoder that reads the last-layer features of the
[Anima](https://huggingface.co/circlestone-labs/Anima) DiT at the final sampling step and
writes RGB pixels directly, in place of the VAE decode. The head and its training follow
[FastFrames](https://arxiv.org/html/2611.23226)' pixel head (a ConvNeXt readout on the diffusion transformer's features).

| | |
|---|---|
| File | `anima_pixel_head.safetensors` (fp32, 95 MB; shape in the file's metadata) |
| Input | the input of the DiT's `final_layer.linear` (2048-dim per token, after the adaLN-modulated LayerNorm) |
| Output | a 16×16 RGB patch per token, in [-1, 1] |
| Architecture | linear projection of the feature + linear embedding of the x_t patch, 2 ConvNeXt, one 2× pixel-shuffle upsample, 1 ConvNeXt, linear to 8×8×3 |
| DiT | `anima-base-v1.0` (28 blocks) |
| Target | base points: the original image, with LPIPS (weight 0.05); turbo point: VAE(x̂0) of the same forward, x̂0 = x_t − σ·v |


## Compatibility

Read the head only where it was trained:

- **Base DiT**: Euler, shift 3, 20 … 50 steps (last-step σ 0.058 … 0.136), read at CFG 4,
  Spectrum included (its last step is an actual forward).
- **Turbo**: the 4-step turbo LoRA of
  [anima-turbo-4step](https://huggingface.co/sorryhyun/anima-turbo-4step), Euler 4 steps,
  shift 3, CFG 1 (last-step σ 0.5). Trained on `anima_turbo_4step_v4_longer.safetensors`.

The previous head is `old/anima_pixel_head.safetensors` (VAE(x̂0) target at every point, turbo
point on that repo's `old/anima_turbo_4step.safetensors`).

## Benchmark

The benchmarks, samples and held-out reads below were measured on the previous head
(`old/anima_pixel_head.safetensors`).

### In ComfyUI

Turbo 4 steps, CFG 1, RTX 5070 Ti, 16 prompts at ~1 MP, one seed

| | KSampler + VAE Decode | KSampler + Anima Pixel Decode |
|---|---|---|
| KSampler | 0.976 s | 0.998 s (with the tap) |
| decode | 151 ms | 7 ms |
| decode peak VRAM above the loaded models | 1.22 GB | 0.21 GB |
| PSNR against VAE Decode | — | 36.9 dB |
| **total per image** | **1.13 s** | **1.00 s (1.12×)** |

ComfyUI keeps the VAE on the card after its first decode, so the VAE column has no move cost.

### In anima_lora

The same turbo setting and prompts through anima_lora's `inference.py` (compiled DiT blocks);
medians. The VAE column is the usual interactive path: the VAE waits on the CPU and goes to the
card for the decode.

| | turbo + VAE | turbo + pixel head |
|---|---|---|
| DiT sampling | 0.97 s | 0.97 s |
| load VAE | 710 ms | — |
| VAE move to the card and back | 35 ms | — |
| decode | 141 ms | 7 ms |
| decode peak VRAM above the DiT | 1.38 GB | 0.24 GB |
| **total per image (DiT + decode)** | **1.86 s** | **0.98 s** |


## Samples

Turbo 4 steps, CFG 1, seed 42, 1024×1024, in ComfyUI (the workflow above); left VAE Decode,
right Anima Pixel Decode, decoded from the same latent. Prompt: `masterpiece, best quality,
score_7, safe. An anime girl wearing a black tank-top and denim shorts is standing outdoors.
She's holding a rectangular sign out in front of her that reads "ANIMA". She's looking at the
viewer with a smile. The background features some trees and blue sky with clouds.`; the
monochrome pair adds `monochrome, comic` after `safe`. Crops: 384×384 at native resolution,
x 192–576, y 512–896.

| | VAE Decode | pixel head |
|---|---|---|
| colour| ![colour, VAE](samples/color_vae.png) | ![colour, pixel head](samples/color_head.png) |
| crop | ![colour crop, VAE](samples/color_vae_crop.png) | ![colour crop, pixel head](samples/color_head_crop.png) |
| monochrome | ![mono, VAE](samples/mono_vae.png) | ![mono, pixel head](samples/mono_head.png) |
| crop | ![mono crop, VAE](samples/mono_vae_crop.png) | ![mono crop, pixel head](samples/mono_head_crop.png) |

On the monochrome crop the VAE keeps the screentone dots on the shorts; the head renders them as
flat grey (see Known limits).

## Held-out reads

Held-out reads during training (dB, colour / mono):

| decode point | colour | mono |
|---|---|---|
| turbo 4 steps | 35.90 | 29.01 |
| base 20 steps | 33.71 | 26.58 |
| base 28 steps | 33.83 | 26.38 |
| base 50 steps | 33.49 | 25.48 |
| base 28 + Spectrum | 34.37 | 26.00 |

## Known limits

- Screentone dots are smoothed into soft grey shading; textured regions (screentone,
  dense hatching) come out about 1 – 2.5 levels darker than the VAE in linear light.
  The lowest per-image PSNRs are manga pages and sketches (turbo 28.7 – 30.9 dB).
- Speed lines and footnote-size glyphs are slightly soft under zoom.
- One trained head, one seed; no run spread.

## Use

- ComfyUI: the [ComfyUI-Anima-PixelDecoder](https://github.com/sorryhyun/ComfyUI-Anima-PixelDecoder)
  node pack (ComfyUI Manager: "Anima Pixel Decoder"). Put this file in `models/pixel_heads/`,
  add **Anima Pixel Head Tap** on the MODEL before the KSampler (stock or Spectrum) and decode
  the sampler's LATENT with **Anima Pixel Decode** instead of VAE Decode. Run the DiT on the
  fp32 stream (Anima Adapter Loader, `stream = fp32`).
- [anima_lora](https://github.com/sorryhyun/anima_lora): `inference.py --decoder pixel_head`
  (`make test PIXEL=1`, `make test-turbo PIXEL=1`); the module, loader and feature tap are in
  `library/anima/pixel_head.py`.
- ×2 (test): `anima_pixel_head_x2_preview.safetensors` decodes and upscales 2× in one
  readout (Anima Pixel Decode `scale = 2`, `inference.py --pixel_scale 2`). A test file; it will
  be replaced.

## License

The weights in this repository are a **Derivative** of
[Anima](https://huggingface.co/circlestone-labs/Anima) and are distributed under the
**[CircleStone Labs Non-Commercial License](LICENSE.md)** (see [NOTICE.md](NOTICE.md)). Any
rights to use them are granted directly by CircleStone Labs LLC under that license.

- The model/weights may only be used for non-commercial purposes. Generated images (Outputs)
  may be used commercially.
- Anima is itself a Derivative Model of NVIDIA Cosmos-Predict2-2B-Text2Image and is subject
  to the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/)
  insofar as it applies to Derivative Models.
- This is not an official CircleStone Labs product and is not endorsed by CircleStone Labs.

Built on NVIDIA Cosmos.