Add model card, pipeline tag and links to paper/code

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +73 -3
README.md CHANGED
@@ -1,3 +1,73 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: unconditional-image-generation
4
+ ---
5
+
6
+ # V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising
7
+
8
+ Official implementation of **V-Co**, a systematic study of visual co-denoising within a unified pixel-space diffusion framework.
9
+
10
+ - **Paper:** [V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising](https://huggingface.co/papers/2603.16792)
11
+ - **GitHub Repository:** [HL-hanlin/V-Co](https://github.com/HL-hanlin/V-Co)
12
+
13
+ ## Method Description
14
+
15
+ V-Co presents a principled study of visual representation alignment via co-denoising in pixel-space diffusion, systematically isolating the effects of architecture, CFG design, auxiliary losses, and feature calibration. The authors identify four key ingredients for effective visual co-denoising:
16
+
17
+ 1. **Fully dual-stream architecture** to enable flexible cross-stream interaction.
18
+ 2. **Structural masking** for unconditional classifier-free guidance (CFG) prediction.
19
+ 3. **Perceptual-drifting hybrid loss** for stronger semantic supervision.
20
+ 4. **RMS-based feature rescaling** for stable cross-stream calibration.
21
+
22
+ Together, these findings yield a simple recipe for visual co-denoising. Experiments on ImageNet-256 show that, at comparable model sizes, V-Co outperforms the underlying pixel-space diffusion baseline (JiT) and strong prior pixel-diffusion methods while using fewer training epochs.
23
+
24
+ ## Evaluation
25
+
26
+ To evaluate a pretrained V-Co model (e.g., VCo-B) on ImageNet 256x256, you can use the following command from the official repository:
27
+
28
+ ```bash
29
+ torchrun --nproc_per_node=8 --nnodes=1 \
30
+ main_vco.py \
31
+ --P_mean -0.8 --P_std 0.8 \
32
+ --img_size 256 \
33
+ --batch_size 32 \
34
+ --gen_bsz 32 \
35
+ --resume '/path/to/vco_base/checkpoint.pth' \
36
+ --online_eval \
37
+ --model JiT-B/16-co \
38
+ --output_dir '/path/to/output_dir' \
39
+ --data_path '/path/to/imagenet/' \
40
+ --evaluate_gen \
41
+ --num_workers 12 \
42
+ --use_co_embed \
43
+ --use_dinov2 \
44
+ --use_dino_from_rae \
45
+ --use_gated_co_embed \
46
+ --use_mmdit \
47
+ --separate_qkv \
48
+ --use_conv2d_dino_proj \
49
+ --uncond_dino_null \
50
+ --dinov2_null_type 'attn_mask_asymmetric' \
51
+ --dinov2_drop_zero_loss \
52
+ --class_balanced_sampling \
53
+ --drifting_v3_loss \
54
+ --cfg_sweep '1.8'
55
+ ```
56
+
57
+ ## Citation
58
+
59
+ ```bibtex
60
+ @misc{lin2026vcocloserlookvisual,
61
+ title={V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising},
62
+ author={Han Lin and Xichen Pan and Zun Wang and Yue Zhang and Chu Wang and Jaemin Cho and Mohit Bansal},
63
+ year={2026},
64
+ eprint={2603.16792},
65
+ archivePrefix={arXiv},
66
+ primaryClass={cs.CV},
67
+ url={https://arxiv.org/abs/2603.16792},
68
+ }
69
+ ```
70
+
71
+ ## Acknowledgements
72
+
73
+ The development of V-Co has been inspired by [JiT](https://github.com/LTH14/JiT) and [Latent Forcing](https://github.com/AlanBaade/LatentForcing).