README.md

metadata

title: Omni-Diffusion
emoji: 🌀
colorFrom: yellow
colorTo: purple
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
short_description: Any-to-any multimodal masked discrete diffusion model
python_version: '3.10'
startup_duration_timeout: 1h
pinned: false

Omni-Diffusion

Interactive demo for Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion (paper · model · code).

Omni-Diffusion is an any-to-any multimodal language model built on a mask-based discrete diffusion model. It jointly models discrete tokens of text and images.

This Space exposes three of the model's capabilities:

• Text → Image: generate an image from a text prompt.
• Image → Text (VQA): answer questions / describe an uploaded image.
• Text → Text: instruction following / chat.

Image (de)tokenization uses showlab/magvitv2.