Bagel-Self-SpectraReward-AWM

Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation

LoRA adapter for BAGEL-7B-MoT, trained with Self-SpectraReward and the AWM (Advantage-Weighted Model) reinforcement-learning algorithm for reasoning text-to-image generation.

• Base model: ByteDance-Seed/BAGEL-7B-MoT
• Method: Self-SpectraReward (no external reward model)
• RL algorithm: AWM
• Adapter: LoRA (r = 32, α = 64) on the generation experts
• Code: https://github.com/huangrh99/AlphaGRPO
• License: Apache-2.0

What is Self-SpectraReward?

SpectraReward measures how well a generated image realizes its prompt by the mean image-conditioned prompt log-likelihood under a multimodal LLM: the image is provided as visual context and the prompt is scored with a single teacher-forced forward pass over its tokens. A higher reward means the prompt is more predictable from the image.

Self-SpectraReward is the unified-model special case: BAGEL uses its own understanding branch as the reward model for its generation branch, so no external reward MLLM is required.

Usage

This is a PEFT LoRA adapter for BAGEL-7B-MoT (not a standalone model). Download it, then follow the Evaluation section of the AlphaGRPO repo with BAGEL_LORA_PATH pointing at the downloaded folder.

hf download huangrh9/Bagel-Self-SpectraReward-AWM --local-dir ./bagel-self-spectrareward-awm
export BAGEL_LORA_PATH=$(pwd)/bagel-self-spectrareward-awm

Reproduce

Training entry point in the AlphaGRPO repo:

torchrun --nproc_per_node=8 alpha_grpo/train.py \
  --config config/bagel.py:self_spectrareward_t2i_awm

Citation

If you use this model, please cite SpectraReward (and AlphaGRPO, the underlying framework):

@article{huang2026readitback,
  title={Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation},
  author={Huang, Runhui and others},
  year={2026}
}

@inproceedings{huang2026alphagrpo,
  title={AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in Unified Multimodal Models via Decompositional Verifiable Reward},
  author={Huang, Runhui and Wu, Jie and Yang, Rui and Liu, Zhe and Zhao, Hengshuang},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2026}
}