|
Download README.md from sorryhyun/anima-pixel-decoder: direct link, hf CLI and curl.
- Browser
- Download file 6.51 kB
-
https://huggingface.co/sorryhyun/anima-pixel-decoder/resolve/main/README.md
- Command line
-
hf download hf://sorryhyun/anima-pixel-decoder/README.md
-
curl -L -o README.md https://huggingface.co/sorryhyun/anima-pixel-decoder/resolve/main/README.md
6.51 kB
| base_model: | |
| - circlestone-labs/Anima | |
| license: other | |
| license_name: circlestone-labs-non-commercial-license | |
| license_link: LICENSE.md | |
| tags: | |
| - anima | |
| - decoder | |
| # Anima Pixel Decoder | |
| A 25M-parameter decoder that reads the last-layer features of the | |
| [Anima](https://huggingface.co/circlestone-labs/Anima) DiT at the final sampling step and | |
| writes RGB pixels directly, in place of the VAE decode. The head and its training follow | |
| [FastFrames](https://arxiv.org/html/2611.23226)' pixel head (a ConvNeXt readout on the diffusion transformer's features). | |
| | | | | |
| |---|---| | |
| | File | `anima_pixel_head.safetensors` (fp32, 95 MB; shape in the file's metadata) | | |
| | Input | the input of the DiT's `final_layer.linear` (2048-dim per token, after the adaLN-modulated LayerNorm) | | |
| | Output | a 16×16 RGB patch per token, in [-1, 1] | | |
| | Architecture | linear projection of the feature + linear embedding of the x_t patch, 2 ConvNeXt, one 2× pixel-shuffle upsample, 1 ConvNeXt, linear to 8×8×3 | | |
| | DiT | `anima-base-v1.0` (28 blocks) | | |
| | Target | base points: the original image, with LPIPS (weight 0.05); turbo point: VAE(x̂0) of the same forward, x̂0 = x_t − σ·v | | |
| ## Compatibility | |
| Read the head only where it was trained: | |
| - **Base DiT**: Euler, shift 3, 20 … 50 steps (last-step σ 0.058 … 0.136), read at CFG 4, | |
| Spectrum included (its last step is an actual forward). | |
| - **Turbo**: the 4-step turbo LoRA of | |
| [anima-turbo-4step](https://huggingface.co/sorryhyun/anima-turbo-4step), Euler 4 steps, | |
| shift 3, CFG 1 (last-step σ 0.5). Trained on `anima_turbo_4step_v4_longer.safetensors`. | |
| The previous head is `old/anima_pixel_head.safetensors` (VAE(x̂0) target at every point, turbo | |
| point on that repo's `old/anima_turbo_4step.safetensors`). | |
| ## Benchmark | |
| The benchmarks, samples and held-out reads below were measured on the previous head | |
| (`old/anima_pixel_head.safetensors`). | |
| ### In ComfyUI | |
| Turbo 4 steps, CFG 1, RTX 5070 Ti, 16 prompts at ~1 MP, one seed | |
| | | KSampler + VAE Decode | KSampler + Anima Pixel Decode | | |
| |---|---|---| | |
| | KSampler | 0.976 s | 0.998 s (with the tap) | | |
| | decode | 151 ms | 7 ms | | |
| | decode peak VRAM above the loaded models | 1.22 GB | 0.21 GB | | |
| | PSNR against VAE Decode | — | 36.9 dB | | |
| | **total per image** | **1.13 s** | **1.00 s (1.12×)** | | |
| ComfyUI keeps the VAE on the card after its first decode, so the VAE column has no move cost. | |
| ### In anima_lora | |
| The same turbo setting and prompts through anima_lora's `inference.py` (compiled DiT blocks); | |
| medians. The VAE column is the usual interactive path: the VAE waits on the CPU and goes to the | |
| card for the decode. | |
| | | turbo + VAE | turbo + pixel head | | |
| |---|---|---| | |
| | DiT sampling | 0.97 s | 0.97 s | | |
| | load VAE | 710 ms | — | | |
| | VAE move to the card and back | 35 ms | — | | |
| | decode | 141 ms | 7 ms | | |
| | decode peak VRAM above the DiT | 1.38 GB | 0.24 GB | | |
| | **total per image (DiT + decode)** | **1.86 s** | **0.98 s** | | |
| ## Samples | |
| Turbo 4 steps, CFG 1, seed 42, 1024×1024, in ComfyUI (the workflow above); left VAE Decode, | |
| right Anima Pixel Decode, decoded from the same latent. Prompt: `masterpiece, best quality, | |
| score_7, safe. An anime girl wearing a black tank-top and denim shorts is standing outdoors. | |
| She's holding a rectangular sign out in front of her that reads "ANIMA". She's looking at the | |
| viewer with a smile. The background features some trees and blue sky with clouds.`; the | |
| monochrome pair adds `monochrome, comic` after `safe`. Crops: 384×384 at native resolution, | |
| x 192–576, y 512–896. | |
| | | VAE Decode | pixel head | | |
| |---|---|---| | |
| | colour|  |  | | |
| | crop |  |  | | |
| | monochrome |  |  | | |
| | crop |  |  | | |
| On the monochrome crop the VAE keeps the screentone dots on the shorts; the head renders them as | |
| flat grey (see Known limits). | |
| ## Held-out reads | |
| Held-out reads during training (dB, colour / mono): | |
| | decode point | colour | mono | | |
| |---|---|---| | |
| | turbo 4 steps | 35.90 | 29.01 | | |
| | base 20 steps | 33.71 | 26.58 | | |
| | base 28 steps | 33.83 | 26.38 | | |
| | base 50 steps | 33.49 | 25.48 | | |
| | base 28 + Spectrum | 34.37 | 26.00 | | |
| ## Known limits | |
| - Screentone dots are smoothed into soft grey shading; textured regions (screentone, | |
| dense hatching) come out about 1 – 2.5 levels darker than the VAE in linear light. | |
| The lowest per-image PSNRs are manga pages and sketches (turbo 28.7 – 30.9 dB). | |
| - Speed lines and footnote-size glyphs are slightly soft under zoom. | |
| - One trained head, one seed; no run spread. | |
| ## Use | |
| - ComfyUI: the [ComfyUI-Anima-PixelDecoder](https://github.com/sorryhyun/ComfyUI-Anima-PixelDecoder) | |
| node pack (ComfyUI Manager: "Anima Pixel Decoder"). Put this file in `models/pixel_heads/`, | |
| add **Anima Pixel Head Tap** on the MODEL before the KSampler (stock or Spectrum) and decode | |
| the sampler's LATENT with **Anima Pixel Decode** instead of VAE Decode. Run the DiT on the | |
| fp32 stream (Anima Adapter Loader, `stream = fp32`). | |
| - [anima_lora](https://github.com/sorryhyun/anima_lora): `inference.py --decoder pixel_head` | |
| (`make test PIXEL=1`, `make test-turbo PIXEL=1`); the module, loader and feature tap are in | |
| `library/anima/pixel_head.py`. | |
| - ×2 (test): `anima_pixel_head_x2_preview.safetensors` decodes and upscales 2× in one | |
| readout (Anima Pixel Decode `scale = 2`, `inference.py --pixel_scale 2`). A test file; it will | |
| be replaced. | |
| ## License | |
| The weights in this repository are a **Derivative** of | |
| [Anima](https://huggingface.co/circlestone-labs/Anima) and are distributed under the | |
| **[CircleStone Labs Non-Commercial License](LICENSE.md)** (see [NOTICE.md](NOTICE.md)). Any | |
| rights to use them are granted directly by CircleStone Labs LLC under that license. | |
| - The model/weights may only be used for non-commercial purposes. Generated images (Outputs) | |
| may be used commercially. | |
| - Anima is itself a Derivative Model of NVIDIA Cosmos-Predict2-2B-Text2Image and is subject | |
| to the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) | |
| insofar as it applies to Derivative Models. | |
| - This is not an official CircleStone Labs product and is not endorsed by CircleStone Labs. | |
| Built on NVIDIA Cosmos. | |