README.md

metadata

base_model: Qwen/Qwen3-VL-8B-Instruct
library_name: transformers
license: other
pipeline_tag: image-text-to-text
tags:
  - llama-factory
  - reward-model
  - image-generation
  - reinforcement-learning
  - generated_from_trainer
model-index:
  - name: FIRM-Gen-8B (gen_reward_sft)
    results: []

FIRM-Gen-8B (gen_reward_sft)

This model is a fine-tuned version of Qwen/Qwen3-VL-8B-Instruct and serves as a robust reward model (critic) for text-to-image generation. It was introduced as part of the FIRM (Faithful Image Reward Modeling) framework in the paper "Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation".

• Paper: Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation
• Project Page: firm-reward.github.io
• Repository: VisionXLab/FIRM-Reward

Model Description

FIRM-Gen-8B is specifically trained on the FIRM-Gen-293K dataset to provide accurate and reliable guidance for faithful image generation. It addresses the common issue of reward hacking and hallucinations in Multimodal Large Language Models (MLLMs) by using a "plan-then-score" pipeline to evaluate how well a generated image follows complex instructions.

Within a Reinforcement Learning (RL) pipeline, this model acts as the critic, assigning scores that guide the optimization of generative models (like Stable Diffusion 3.5 or FLUX) toward better instruction adherence and visual fidelity.

Intended Uses & Limitations

This model is intended to be used as a reward signal in RL pipelines or as an evaluation metric for text-to-image alignment. It is compatible with the transformers library and can be deployed using the reward server scripts found in the official repository.

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 5
• eval_batch_size: 2
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 2
• total_train_batch_size: 80
• total_eval_batch_size: 16
• optimizer: AdamW
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1.0

Training Results

Training Loss	Epoch	Step	Validation Loss
0.6131	0.1380	500	0.6089
0.5714	0.2760	1000	0.5768
0.5524	0.4140	1500	0.5562
0.537	0.5520	2000	0.5407
0.5282	0.6899	2500	0.5283
0.5155	0.8279	3000	0.5207
0.5106	0.9659	3500	0.5181

Citation

If you find this model useful, please cite:

@article{zhao2025trust,
  title={Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation},
  author={Zhao, Xiangyu and Zhang, Peiyuan and Lin, Junming and Liang, Tianhao and Duan, Yuchen and Ding, Shengyuan and Tian, Changyao and Zang, Yuhang and Yan, Junchi and Yang, Xue},
  journal={arXiv preprint arXiv:2603.12247},
  year={2025}
}