---
license: apache-2.0
tags:
- video
- self-supervised-learning
- diffusion
pipeline_tag: other
---

# TT-VidT pretrained DiT decoders

Pretrained DiT decoders for [TT-VidT](https://github.com/KohakuBlueleaf/TTVidT)
(*Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining*,
NeurIPS 2026). [Paper](https://huggingface.co/papers/2609.33419) ·
[Project page](https://kohakublueleaf.github.io/TTVidT/)

Each TT-VidT encoder pretraining run initialises its decoder from one of
these; the training code downloads them automatically through `DECODER_PRETRAINED`,
e.g. `"KBlueLeaf/TTVidT-decoders/pretrain_video_S_qknorm"`.

The decoders work in the 32x32x4 latent space of the frozen f8 frame VAE
[`KBlueLeaf/latentmaid-vae`](https://huggingface.co/KBlueLeaf/latentmaid-vae) and are
conditioned on DINOv3 ViT-B/16 features (768-d).

- **ImageNet** decoders reconstruct an ImageNet image from its DINOv3 class token.
- **Video** decoders reconstruct a later video frame from its class token plus, through
  cross-attention, the spatial features of an earlier frame.

## Files

Each decoder is `<name>.safetensors` plus `<name>.json`, which holds the decoder
config (`decoder_config`), the latent size and the latent statistics used in training.

`pretrain_{imgnet,video}_{S,B,L}_<variant>`:

| Size | Hidden | Layers | Heads |
|---|---:|---:|---:|
| S | 768 | 12 | 12 |
| B | 1024 | 16 | 16 |
| L | 1152 | 28 | 16 |

| Variant | Meaning |
|---|---|
| `qknorm` | QK-norm attention, diffusion head |
| `qknorm_nofinal` | as above, without the final norm |
| `qknorm_reg`, `qknorm_nofinal_reg` | latent-regression head instead of diffusion |
| `768d12l`, `1024d16l`, `1152d28l` (`_80k`) | earlier runs without QK-norm |

The ones used in the paper's configs:

| Decoder | Used by |
|---|---|
| `pretrain_imgnet_S_qknorm` | objective / encoder sweep |
| `pretrain_video_S_qknorm` | TT-VidT |
| `pretrain_imgnet_S_qknorm_nofinal` | DisMo-2D3D with dual augmentation |
| `pretrain_{imgnet,video}_B_qknorm_nofinal`, `pretrain_{imgnet,video}_L_qknorm` | decoder ablation |

Recipe: 100k steps (80k for `_80k`), global batch 256, AdamW lr 1e-4. See the
[code repository](https://github.com/KohakuBlueleaf/TTVidT) for training and usage.