Update README.md

README.md

@@ -6,4 +6,339 @@ language:
 - en
 base_model:
 - Qwen/Qwen2-VL-7B-Instruct
----
+---
+
+---
+license: mit
+---
+
+
+# MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills
+### **SIGGRAPH 2025 (ACM Transactions on Graphics)**
+
+<div align="center">
+
+[![Project Page](https://img.shields.io/badge/Project-Page-green)](https://monetgpt.github.io/) 
+[![Paper](https://img.shields.io/badge/Paper-ArXiv-red)](https://arxiv.org/abs/2505.06176)
+[![ACM](https://img.shields.io/badge/ACM-PDF-blue)](https://dl.acm.org/doi/pdf/10.1145/3730926)
+[![Model](https://img.shields.io/badge/🤗-Model-yellow)](https://huggingface.co/niladridutt/monetGPT)
+
+
+</div>
+
+<div align="center">
+<img src="assets/teaser.jpg" alt="MonetGPT Teaser">
+</div>
+
+## Table of Contents
+
+- [Overview](#overview)
+- [Quick Start](#-quick-start)
+- [Usage](#-usage)
+- [Training Your Own Model](#-training-your-own-model)
+- [Image Processing CLI Usage](#-image-processing-cli-usage)
+- [Puzzle Types](#-puzzle-types)
+- [Configuration](#-configuration)
+- [Results & Evaluation](#-results--evaluation)
+- [Troubleshooting](#-troubleshooting)
+- [Citation](#-citation)
+- [License](#-license)
+
+### Note: This HuggingFace repository only contains model weights. The full codebase for MonetGPT is available on our [GitHub repository](https://github.com/niladridutt/monetGPT).
+
+## Overview
+
+**MonetGPT** is a novel framework that teaches multimodal large language models (MLLMs) to perform professional-quality image retouching through procedural operations. Unlike generative editing approaches that can unpredictably alter image content, MonetGPT learns to plan and execute sequences of traditional retouching operations (brightness, contrast, saturation, etc.) that preserve object identity and provide explainable results.
+
+### Visual Puzzles for Operation Awareness
+
+🧩 MLLMs learn retouching operations by solving specially designed visual puzzles that teach operation recognition, parameter understanding, and sequence planning.Unlike black-box generative models, MonetGPT provides clear reasoning for each editing decision and preserves original image content and resolution (e.g., 8K 16-bit).
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+# Clone the repository
+git clone https://github.com/monetgpt/monetgpt.git
+cd monetgpt
+
+# Create and activate conda environment
+conda create -n monetgpt python=3.11
+conda activate monetgpt
+
+# Install dependencies
+cd llm 
+sh install.sh
+```
+
+### GIMP Installation
+
+MonetGPT requires **GIMP 2.10** for image processing operations. Other versions may not be compatible.
+
+#### Download GIMP 2.10:
+- **macOS**: [Download GIMP 2.10.38 ARM64](https://download.gimp.org/gimp/v2.10/macos/gimp-2.10.38-arm64-1.dmg)
+- **Linux**: Install via Flatpak: 
+  ```bash
+  flatpak install flathub org.gimp.GIMP//2.10
+  ```
+- **Windows**: [Download from GIMP website](https://download.gimp.org/gimp/v2.10/)
+
+#### Install NumPy for GIMP (Linux Flatpak only):
+```bash
+flatpak run --command=sh org.gimp.GIMP//stable -c "python -m pip install --user numpy"
+```
+
+Note: MacOS version ships with NumPy built-in.
+
+### Download Pre-trained Model
+
+Download the trained MonetGPT model from Hugging Face:
+
+```bash
+# Navigate to llm directory and create models folder
+cd llm
+mkdir -p models
+cd models
+
+# Download the model using HF CLI
+huggingface-cli download niladridutt/monetGPT
+
+# OR Clone the model repository (requires git lfs)
+git clone https://huggingface.co/niladridutt/monetGPT
+```
+
+> **Note**: Ensure the model is saved as `llm/models/monetGPT` to match the expected directory structure or otherwise modify the configs in llm.
+
+## � Usage
+
+<div align="center">
+<img src="assets/pipeline.gif" alt="MonetGPT Pipeline">
+</div>
+
+## Inference
+
+First start the LLM, which shall launch a server
+
+```bash
+cd llm
+sh test.sh
+cd ..
+```
+
+Run image enhancement/retouching with the pre-trained MonetGPT model (make sure the LLM is running):
+
+```bash
+# Single image processing
+
+python inference_cli.py single input.jpg --output results/edited.jpg
+
+# Batch processing
+python inference_cli.py batch assets/test --output-dir results/
+```
+
+## 📚 Training Your Own Model
+
+### 1. Dataset Preparation
+
+See `image_sources` in `configs/dataset_config.yaml`
+
+```bash
+# Prepare your training images
+mkdir -p data/ppr10k
+# Place your .png/.jpg images in data/images/
+```
+
+### 2. Generate Training Puzzles
+
+```bash
+# Step 1: Generate puzzle configurations (operation parameters)
+python dataset_cli.py generate
+
+# Step 2: Create visual puzzle images
+python pipeline_cli.py puzzle 1     # Single operation puzzles
+python pipeline_cli.py puzzle 2     # Multi-version comparison puzzles  
+python pipeline_cli.py puzzle 3     # Comprehensive editing puzzles
+python pipeline_cli.py puzzle all   # Generate all puzzle types
+```
+
+### 3. Generate LLM Reasoning
+
+```bash
+# Step 3: Query LLM to add reasoning to puzzle configs
+python dataset_cli.py query 1 0 -1    # Generate reasoning for all puzzle 1 configs (all)
+python dataset_cli.py query 2 0 -1   # Generate reasoning for puzzle 2 configs (all)
+python dataset_cli.py query 3 0 -1    # Generate reasoning for puzzle 3 configs (all)
+
+# Step 4: Create final ShareGPT format datasets  
+python dataset_cli.py create          # Create datasets for all puzzles
+
+# Step 5: Combine datasets for training
+python dataset/combine_jsons.py       # Combine JSON datasets and export for training
+```
+
+### 4. Train the Model
+
+```bash
+# Train MonetGPT model on the generated datasets
+cd llm
+sh train.sh
+```
+
+> **Note**: This project uses [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) for training infrastructure, which is licensed under [Apache 2.0](https://github.com/hiyouga/LLaMA-Factory/blob/main/LICENSE).
+
+## � Image Processing CLI Usage
+
+### Single Image Edit
+
+```bash
+# Apply a specific retouching configuration to an image
+python pipeline_cli.py edit configs/example_edit.json input.jpg output.jpg
+```
+
+### Batch Processing
+
+```bash
+# Process multiple images with MonetGPT predictions
+python pipeline_cli.py batch predictions --target-editor a
+```
+
+### Generate Puzzle Images
+
+```bash
+# Generate single operation puzzles
+python pipeline_cli.py puzzle 1
+
+# Generate multi-version comparison puzzles
+python pipeline_cli.py puzzle 2
+
+# Generate comprehensive editing puzzles
+python pipeline_cli.py puzzle 3
+
+# Generate all puzzle types
+python pipeline_cli.py puzzle all
+```
+
+
+
+## 🧩 Puzzle Types
+
+### Puzzle 1: Single Operation Analysis
+- **Purpose**: Teach individual retouching operations
+- **Format**: Before/after comparison with single operation
+- **Example**: "Which adjustment was applied and how much?"
+
+### Puzzle 2: Multi-Version Comparison  
+- **Purpose**: Teach parameter value relationships
+- **Format**: Multiple versions with different parameter values
+- **Example**: "Rank these images by optimal saturation level"
+
+### Puzzle 3: Comprehensive Editing Plans
+- **Purpose**: Teach complete retouching workflows
+- **Format**: Multi-step editing sequences
+- **Example**: "Plan the editing sequence: 1) Fix lighting 2) Adjust white balance 3) Enhance colors"
+
+## 🔧 Configuration
+
+### Dataset Configuration (`configs/dataset_config.yaml`)
+
+```yaml
+# LLM settings for reasoning generation
+model: "gemini-2.0-flash"
+api_key: ""  # Set your API key here
+base_url: "https://generativelanguage.googleapis.com/v1beta/openai/"
+timeout: 5
+retry_attempts: 1
+
+# Puzzle paths and settings
+puzzles:
+  puzzle1:
+    reasoning_path: "./data/puzzles1/reasoning/*.txt"
+    images_path: "./data/puzzles1/images/*.png"
+    images_base_path: "./data/puzzles1/images"
+    output_file: "data/sharegpt_puzzle_1.json"
+  
+  puzzle2:
+    reasoning_path: "./data/puzzles2/reasoning/*.txt"
+    images_path: "./data/puzzles2/images/*.png"
+    images_base_path: "./data/puzzles2/images"
+    output_file: "data/sharegpt_puzzle_2.json"
+    
+  puzzle3:
+    reasoning_path: "./data/puzzles3/reasoning/*.txt"
+    images_path: "./data/puzzles3/images/*/*.tif"
+    images_base_path: "./data/puzzles3/images"
+    output_file: "data/sharegpt_puzzle_3.json"
+
+# Generation settings
+generation:
+  num_standard_trials: 2
+  num_color_trials: 1
+  num_puzzle3_trials: 10
+```
+
+### Pipeline Configuration (`configs/pipeline_config.yaml`)
+
+```yaml
+# GIMP settings
+gimp:
+  paths:
+    macos: "/Applications/GIMP.app/Contents/MacOS/gimp-console-2.10"
+    linux: "flatpak run org.gimp.GIMP//stable --no-interface"
+    windows: "gimp-console-2.10.exe" (Not tested, may require some modifications)
+  
+  batch_interpreter: "python-fu-eval"
+  python_warnings: "ignore"
+  pipeline_file: "./gimp_pipeline.py"
+
+# Image processing settings
+image_processing:
+  max_low_res_size: 700  # Low resolution for LLM training only
+  default_dpi: 140       # Original resolution preserved during inference
+
+# Processing parameters
+processing:
+  batch_size: 10
+  max_workers: 4
+  timeout_seconds: 120
+```
+
+### Benchmark Performance
+MonetGPT achieves state-of-the-art results on image retouching tasks while providing full explainability and maintaining original image resolution.
+
+## 🔧 Troubleshooting
+
+### Common Issues
+
+**GIMP Not Found**: Ensure GIMP 2.10 is installed and the path in `configs/pipeline_config.yaml` matches your installation.
+
+**NumPy Import Error**: Install NumPy in GIMP's Python environment (see GIMP installation section).
+
+**Model Download Issues**: Verify Git LFS is installed for large model files: `git lfs install`
+
+
+## 📄 Citation
+
+If you find MonetGPT useful in your research, please consider citing our paper:
+
+```bibtex
+@article{dutt2025monetgpt,
+  title={MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills},
+  author={Dutt, Niladri Shekhar and Ceylan, Duygu and Mitra, Niloy J},
+  journal={ACM Transactions on Graphics (TOG)},
+  volume={44},
+  number={4},
+  pages={1--12},
+  year={2025},
+  publisher={ACM New York, NY, USA}
+}
+```
+
+
+## 📜 License
+
+This project is released under the [MIT License](LICENSE).
+
+This project uses image dehazer as one of the image operations. This code is adapted from [Single-Image-Dehazing-Python](https://github.com/Utkarsh-Deshmukh/Single-Image-Dehazing-Python/tree/master), which is licensed under the BSD 2-Clause License. A copy of this license can be found in the [licenses/BSD-2-Clause.txt](licenses/BSD-2-Clause.txt) file.
+
+