GrayShine commited on
Commit
27b6e8e
Β·
verified Β·
1 Parent(s): e291f36

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +106 -3
README.md CHANGED
@@ -1,3 +1,106 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <div align="center">
2
+ <h1>πŸš€ WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction</h1>
3
+
4
+ [![arXiv](https://img.shields.io/badge/arXiv-2505.12489-b31b1b.svg)](https://arxiv.org/abs/2505.12489)
5
+ [![Github](https://img.shields.io/badge/Github-WeTok-blue)](https://github.com/zhuangshaobin/WeTok.github)
6
+ [![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-yellow)](https://huggingface.co/GrayShine/WeTok)
7
+
8
+ </div>
9
+
10
+ This project introduces **WeTok**, a powerful discrete visual tokenizer designed to resolve the long-standing conflict between compression efficiency and reconstruction fidelity. WeTok achieves state-of-the-art reconstruction quality, surpassing previous leading discrete and continuous tokenizers. <br><br>
11
+
12
+ > <a href="https://github.com/zhuangshaobin/WeTok">WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction</a><br>
13
+ > [Shaobin Zhuang](https://scholar.google.com/citations?user=PGaDirMAAAAJ&hl=zh-CN&oi=ao), [Yiwei Guo](https://scholar.google.com/citations?user=HCAyeJIAAAAJ&hl=zh-CN&oi=ao), Canmiao Fu, Zhipeng Huang, [Zeyue Tian](https://scholar.google.com/citations?user=dghq4MQAAAAJ&hl=zh-CN&oi=ao), [Ying Zhang](https://scholar.google.com/citations?user=R_psgxkAAAAJ&hl=zh-CN&oi=ao), Chen Li, [Yali Wang](https://scholar.google.com/citations?hl=zh-CN&user=hD948dkAAAAJ)<br>
14
+ > Shanghai Jiao Tong University, WeChat Vision (Tencent Inc.), Shenzhen Institutes of Advanced Technology (Chinese Academy of Sciences), Hong Kong University of Science and Technology, Shanghai AI Laboratory<br>
15
+ > <a href="./docs/WeTok.md">πŸ“šWeTok.md</a>
16
+ > ```
17
+ > @article{zhuang2026wetok,
18
+ > title={WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction},
19
+ > author={Zhuang, Shaobin and Guo, Yiwei and Fu, Canmiao and Huang, Zhipeng and Tian, Zeyue and Zhang, Ying and Li, Chen and Wang, Yali},
20
+ > journal={arXiv preprint arXiv:2409.04410},
21
+ > year={2025}
22
+ > }
23
+ > ```
24
+
25
+ <p align="center">
26
+ <img src="./assets/teaser.png" width="90%">
27
+ <br>
28
+ <em>WeTok achieves a new state-of-the-art in reconstruction fidelity, surpassing both discrete and continuous tokenizers, while offering high compression ratios.</em>
29
+ </p>
30
+
31
+ ## πŸ“° News
32
+ * **[2025.08.05]**:fire::fire::fire: We release a series of WeTok models, achieving a record-low zero-shot rFID of **0.12** on ImageNet, surpassing top continuous tokenizers like FLUX-VAE and SD-VAE 3.5.
33
+ * **[2025.08.05]** We are excited to release **WeTok**, a powerful discrete tokenizer featuring our novel **Grouped Lookup-Free Quantization (GFQ)** and a **generative decoder**. Code and pretrained models are now available!
34
+
35
+ ## πŸ“– Implementations
36
+
37
+ ### πŸ› οΈ Installation
38
+ - **Dependencies**:
39
+ ```
40
+ bash env.sh
41
+ ```
42
+
43
+ ### Evaluation
44
+
45
+ - **Evaluation on ImageNet 50K Validation Set**
46
+
47
+ The dataset should be organized as follows:
48
+ ```
49
+ imagenet
50
+ └── val/
51
+ β”œβ”€β”€ ...
52
+ ```
53
+
54
+ Run the 256Γ—256 resolution evaluation script:
55
+ ```
56
+ bash scripts/evaluation/imagenet_evaluation_256_dist.sh
57
+ ```
58
+
59
+ Run the original resolution evaluation script:
60
+ ```
61
+ bash scripts/evaluation/imagenet_evaluation_original_dist.sh
62
+ ```
63
+
64
+ - **Evaluation on MS-COCO Val2017**
65
+
66
+ The dataset should be organized as follows:
67
+ ```
68
+ MSCOCO2017
69
+ └── val2017/
70
+ β”œβ”€β”€ ...
71
+ ```
72
+
73
+ Run the evaluation script:
74
+ ```
75
+ bash scripts/evaluation/mscocoval_evaluation_256_dist.sh
76
+ ```
77
+
78
+ Run the original resolution evaluation script:
79
+ ```
80
+ bash scripts/evaluation/mscoco_evaluation_original_dist.sh
81
+ ```
82
+
83
+
84
+ ### Inference
85
+
86
+ Simply test the effect of each model reconstruction:
87
+ ```
88
+ bash scripts/inference/reconstruct_image.sh
89
+ ```
90
+
91
+ <p align="center">
92
+ <img src="./assets/compare.png" width="90%">
93
+ <br>
94
+ <em>Qualitative comparison of 512 Γ— 512 image reconstruction on TokBench.</em>
95
+ </p>
96
+
97
+ <p align="center">
98
+ <img src="./assets/gen.png" width="90%">
99
+ <br>
100
+ <em>WeTok-AR-XL generated samples at 256 Γ— 256 resolution.</em>
101
+ </p>
102
+
103
+
104
+
105
+ ## ❀️ Acknowledgement
106
+ Our work builds upon the foundations laid by many excellent projects in the field. We would like to thank the authors of [Open-MAGVIT2](https://arxiv.org/abs/2409.04410). We also drew inspiration from the methodologies presented in [LFQ](https://arxiv.org/abs/2310.05737), [BSQ](https://arxiv.org/abs/2406.07548). We are grateful for their contributions to the community.