license: mit
pipeline_tag: any-to-any
library_name: transformers
MMaDA-Parallel-A
We introduce Parallel Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation (MMaDA-Parallel), a parallel multimodal diffusion framework that enables continuous, bidirectional interaction between text and images throughout the entire denoising trajectory.
This variant is based on Amused-VQ, trained from Lumina-DiMOO, with better quality and robustness.
Paper | Code | Project Page
Abstract
While thinking-aware generation aims to improve performance on complex tasks, we identify a critical failure mode where existing sequential, autoregressive approaches can paradoxically degrade performance due to error propagation. To systematically analyze this issue, we propose ParaBench, a new benchmark designed to evaluate both text and image output modalities. Our analysis using ParaBench reveals that this performance degradation is strongly correlated with poor alignment between the generated reasoning and the final image. To resolve this, we propose a parallel multimodal diffusion framework, MMaDA-Parallel, that enables continuous, bidirectional interaction between text and images throughout the entire denoising trajectory. MMaDA-Parallel is trained with supervised finetuning and then further optimized by Parallel Reinforcement Learning (ParaRL), a novel strategy that applies semantic rewards along the trajectory to enforce cross-modal consistency. Experiments validate that our model significantly improves cross-modal alignment and semantic consistency, achieving a 6.9% improvement in Output Alignment on ParaBench compared to the state-of-the-art model, Bagel, establishing a more robust paradigm for thinking-aware image synthesis.
Architecture
Architecture of MMaDA-Parallel. During Training, image and text responses are masked and predicted in parallel with a uniform mask predictor. During Sampling, the model performs parallel decoding to generate both image and text responses jointly, enabling continuous cross-modal interaction.
Results
Qualitative comparison.
Quantitative Results on ParaBench.
Quick Start
1. Environment Setup
First, start with a torch environment with torch 2.3.1 or higher version, then install the following dependencies:
pip install -r requirements.txt
We provide two varients of MMaDA-Parallel: MMaDA-Parallel-A and MMaDA-Parallel-M. These two varients are with different tokenizer, Amused-VQ and Magvitv2 respectively. MMaDA-Parallel-A is trained with Amused-VQ, from Lumina-DiMOO, and MMaDA-Parallel-M is trained with Magvitv2, from MMaDA-8B.
2. Experiencing Parallel Gen with MMaDA-Parallel-A
cd MMaDA-Parallel-A
python inference.py \
--checkpoint tyfeld/MMaDA-Parallel-A \
--vae_ckpt tyfeld/MMaDA-Parallel-A \
--prompt "Replace the laptops with futuristic transparent tablets displaying holographic screens, and change the drink to a cup of glowing blue energy drink." \
--image_path examples/image.png \
--height 512 \
--width 512 \
--timesteps 64 \
--text_steps 128 \
--text_gen_length 256 \
--text_block_length 32 \
--cfg_scale 0 \
--cfg_img 4.0 \
--temperature 1.0 \
--text_temperature 0 \
--seed 42 \
--output_dir output/results_interleave
Citation
@article{tian2025mmadaparallel,
title={MMaDA-Parallel: Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation},
author={Tian, Ye and Yang, Ling and Yang, Jiongfan and Wang, Anran and Tian, Yu and Zheng, Jiani and Wang, Haochen and Teng, Zhiyang and Wang, Zhuochen and Wang, Yinjie and Tong, Yunhai and Wang, Mengdi and Li, Xiangtai},
journal={arXiv preprint arXiv:2511.09611},
year={2025}
}
Acknowledgments
This work is heavily based on MMaDA and Lumina-DiMOO. Thanks to all the authors for their great work.