Add model card for RAEv2

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +38 -0
README.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: image-to-image
4
+ ---
5
+
6
+ # Improved Baselines with Representation Autoencoders (RAEv2)
7
+
8
+ RAEv2 simplifies and improves representation autoencoders (RAE), achieving over 10x faster convergence, better generation, and better reconstruction compared to traditional VAEs.
9
+
10
+ [**Paper**](https://arxiv.org/abs/2605.18324) | [**Project Page**](https://raev2.github.io) | [**GitHub**](https://github.com/nanovisionx/RAEv2)
11
+
12
+ ## Overview
13
+ Representation Autoencoders (RAE) replace traditional VAEs with pretrained vision encoders. RAEv2 introduces three key insights to improve this architecture:
14
+ 1. **Generalized Formulation**: Defining the representation as the sum of the last *k* encoder layers significantly improves reconstruction.
15
+ 2. **RAE and REPA Synergy**: Representation Alignment (REPA) and RAE are found to be complementary, allowing the same representation to serve as both an encoder and a target.
16
+ 3. **Free Guidance**: By re-parameterizing the DiT model output, REPA can provide classifier-free guidance without needing a second diffusion model.
17
+
18
+ RAEv2 achieves a state-of-the-art gFID of 1.06 in just 80 epochs on ImageNet-256.
19
+
20
+ ## Sample Usage
21
+
22
+ To reconstruct an image using a trained RAE model, you can use the sampling script provided in the [official repository](https://github.com/nanovisionx/RAEv2):
23
+
24
+ ```bash
25
+ uv run python scripts/stage1/sample.py \
26
+ --config configs/stage1/sampling/dinov3l-k23-general.yaml \
27
+ --image assets/samples/sample_1.png
28
+ ```
29
+
30
+ ## Citation
31
+ ```bibtex
32
+ @article{singh2026raev2,
33
+ title={Improved Baselines with Representation Autoencoders},
34
+ author={Singh, Jaskirat and Zheng, Boyang and Wu, Zongze and Zhang, Richard and Shechtman, Eli and Xie, Saining},
35
+ journal={arXiv preprint arXiv:2605.18324},
36
+ year={2026}
37
+ }
38
+ ```