| | --- |
| | |
| | |
| | {} |
| | --- |
| | |
| | <a href="https://2024.emnlp.org/" target="_blank"> <img alt="EMNLP 2024" src="https://img.shields.io/badge/Proceedings-EMNLP2024-red" /> </a> |
| |
|
| |
|
| | This repository contains the official checkpoint for PixelGPT, as presented in the paper [Autoregressive Pre-Training on Pixels and Texts (EMNLP 2024)](https://arxiv.org/pdf/2404.10710). For detailed instructions on how to use the model, please visit our [GitHub page](https://github.com/ernie-research/pixelgpt/). |
| |
|
| | ## Model Description |
| | PixelGPT is an autoregressive language model pre-trained exclusively on pixel data using a next patch prediction objective. By processing documents as visual data (pixels), the model learns to predict the next image patch in a sequence, enabling it to handle visually complex tasks without relying on tokenized text. This tokenization-free approach allows PixelGPT to process and understand text rendered as images. |
| |
|
| |
|
| | ## Citation |
| | ``` |
| | @misc{chai2024autoregressivepretrainingpixelstexts, |
| | title = {Autoregressive Pre-Training on Pixels and Texts}, |
| | author = {Chai, Yekun and Liu, Qingyi and Xiao, Jingwu and Wang, Shuohuan and Sun, Yu and Wu, Hua}, |
| | year = {2024}, |
| | eprint = {2404.10710}, |
| | archiveprefix = {arXiv}, |
| | primaryclass = {cs.CL}, |
| | url = {https://arxiv.org/abs/2404.10710}, |
| | } |
| | ``` |
