anima
decoder

Anima Pixel Decoder

A 25M-parameter decoder that reads the last-layer features of the Anima DiT at the final sampling step and writes RGB pixels directly, in place of the VAE decode. The head and its training follow FastFrames' pixel head (a ConvNeXt readout on the diffusion transformer's features).

File anima_pixel_head.safetensors (fp32, 95 MB; shape in the file's metadata)
Input the input of the DiT's final_layer.linear (2048-dim per token, after the adaLN-modulated LayerNorm)
Output a 16Γ—16 RGB patch per token, in [-1, 1]
Architecture linear projection of the feature + linear embedding of the x_t patch, 2 ConvNeXt, one 2Γ— pixel-shuffle upsample, 1 ConvNeXt, linear to 8Γ—8Γ—3
DiT anima-base-v1.0 (28 blocks)
Target VAE(xΜ‚0) of the same forward, xΜ‚0 = x_t βˆ’ σ·v

Compatibility

Read the head only where it was trained:

  • Base DiT: Euler, shift 3, 20 … 50 steps (last-step Οƒ 0.058 … 0.136), read at CFG 4, Spectrum included (its last step is an actual forward).
  • Turbo: the 4-step turbo LoRA of anima-turbo-4step, Euler 4 steps, shift 3, CFG 1 (last-step Οƒ 0.5).

Benchmark

In ComfyUI

Turbo 4 steps, CFG 1, RTX 5070 Ti, 16 prompts at ~1 MP, one seed

KSampler + VAE Decode KSampler + Anima Pixel Decode
KSampler 0.976 s 0.998 s (with the tap)
decode 151 ms 7 ms
decode peak VRAM above the loaded models 1.22 GB 0.21 GB
PSNR against VAE Decode β€” 36.9 dB
total per image 1.13 s 1.00 s (1.12Γ—)

ComfyUI keeps the VAE on the card after its first decode, so the VAE column has no move cost.

In anima_lora

The same turbo setting and prompts through anima_lora's inference.py (compiled DiT blocks); medians. The VAE column is the usual interactive path: the VAE waits on the CPU and goes to the card for the decode.

turbo + VAE turbo + pixel head
DiT sampling 0.97 s 0.97 s
load VAE 710 ms β€”
VAE move to the card and back 35 ms β€”
decode 141 ms 7 ms
decode peak VRAM above the DiT 1.38 GB 0.24 GB
total per image (DiT + decode) 1.86 s 0.98 s

Samples

Turbo 4 steps, CFG 1, seed 42, 1024Γ—1024, in ComfyUI (the workflow above); left VAE Decode, right Anima Pixel Decode, decoded from the same latent. Prompt: masterpiece, best quality, score_7, safe. An anime girl wearing a black tank-top and denim shorts is standing outdoors. She's holding a rectangular sign out in front of her that reads "ANIMA". She's looking at the viewer with a smile. The background features some trees and blue sky with clouds.; the monochrome pair adds monochrome, comic after safe. Crops: 384Γ—384 at native resolution, x 192–576, y 512–896.

VAE Decode pixel head
colour colour, VAE colour, pixel head
crop colour crop, VAE colour crop, pixel head
monochrome mono, VAE mono, pixel head
crop mono crop, VAE mono crop, pixel head

On the monochrome crop the VAE keeps the screentone dots on the shorts; the head renders them as flat grey (see Known limits).

Held-out reads

Held-out reads during training (dB, colour / mono):

decode point colour mono
turbo 4 steps 35.90 29.01
base 20 steps 33.71 26.58
base 28 steps 33.83 26.38
base 50 steps 33.49 25.48
base 28 + Spectrum 34.37 26.00

Known limits

  • Screentone dots are smoothed into soft grey shading; textured regions (screentone, dense hatching) come out about 1 – 2.5 levels darker than the VAE in linear light. The lowest per-image PSNRs are manga pages and sketches (turbo 28.7 – 30.9 dB).
  • Speed lines and footnote-size glyphs are slightly soft under zoom.
  • One trained head, one seed; no run spread.

Use

  • ComfyUI: the ComfyUI-Anima-PixelDecoder node pack (ComfyUI Manager: "Anima Pixel Decoder"). Put this file in models/pixel_heads/, add Anima Pixel Head Tap on the MODEL before the KSampler (stock or Spectrum) and decode the sampler's LATENT with Anima Pixel Decode instead of VAE Decode. Run the DiT on the fp32 stream (Anima Adapter Loader, stream = fp32).
  • anima_lora: inference.py --decoder pixel_head (make test PIXEL=1, make test-turbo PIXEL=1); the module, loader and feature tap are in library/anima/pixel_head.py.

License

The weights in this repository are a Derivative of Anima and are distributed under the CircleStone Labs Non-Commercial License (see NOTICE.md). Any rights to use them are granted directly by CircleStone Labs LLC under that license.

  • The model/weights may only be used for non-commercial purposes. Generated images (Outputs) may be used commercially.
  • Anima is itself a Derivative Model of NVIDIA Cosmos-Predict2-2B-Text2Image and is subject to the NVIDIA Open Model License Agreement insofar as it applies to Derivative Models.
  • This is not an official CircleStone Labs product and is not endorsed by CircleStone Labs.

Built on NVIDIA Cosmos.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for sorryhyun/anima-pixel-decoder

Finetuned
(121)
this model