🧰 Tool-IQA: Augmenting Image Quality Assessment with Simple Tools
Tool-IQA is a tool-augmented Vision-Language Model workflow for no-reference Image Quality Assessment (NR-IQA). Instead of giving a score from a single glance, Tool-IQA observes quality attributes, inspects uncertain visual evidence, and predicts a calibrated quality score.
🚧 Current release: inference scripts + model weights. Training code and scheduler scripts will be released later.
✨ Highlights
- 🧠 Inspection-based IQA instead of one-shot scoring.
- 🔍 Lightweight visual inspection for local details and visibility cues.
- 📝 Rubric-guided reasoning before final score prediction.
- ⚡ vLLM-based asynchronous benchmark inference.
✅ Release Status
- README and paper figures
- Tool-augmented inference:
infer/tool_infer.py - Inference-time visual inspection tools
- Checkpoint loading via
TARGET_MODEL_NAME - Public training code
- Training scheduler / launch scripts
- Full training data preparation pipeline
The GRPO/BATE training pipeline is not included in this release and will be added later.
🧭 Method Overview
Tool-IQA follows three compact steps:
- Rubric observation: summarize perceptual attributes such as sharpness, noise, artifacts, exposure, color, and subject visibility.
- Visual inspection: gather additional local or visibility-aware evidence when the initial observation is uncertain.
- Final quantification: synthesize all evidence and output a score in
[1, 5].
⚙️ Installation
conda create -n tool-iqa python=3.12
conda activate tool-iqa
pip install numpy pillow tqdm scipy transformers vllm
🚀 Inference
Prepare the checkpoint and data first. See data/README.md for the expected dataset layout.
export TARGET_MODEL_NAME=/path/to/Tool-IQA-weights
python infer/tool_infer.py
Results are saved to:
results/toolIQA/<dataset>/<model_name>/results.txt
Each row records the image name, predicted score, ground-truth score, score history, reasoning history, inspection history, and error message if any.
🙏 Acknowledgement
This project is built based on ms-swift. We thank the MS Swift team for their open-source framework.
📬 Contact
For questions, please contact guanyi.qin@u.nus.edu.
- Downloads last month
- 8