yzc002 commited on
Commit
b5c9bc7
Β·
verified Β·
1 Parent(s): 6e4a30b

Upload 2 files

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. PRAC.png +3 -0
  3. README.md +189 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ PRAC.png filter=lfs diff=lfs merge=lfs -text
PRAC.png ADDED

Git LFS Details

  • SHA256: 85e1601f2a9a2a9ee59689389c387375787a86ae81875739f9003b6c91367b5c
  • Pointer size: 131 Bytes
  • Size of remote file: 872 kB
README.md ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <div align="center">
2
+ <a href="http://arxiv.org/abs/2607.15752"><img src="https://img.shields.io/badge/Arxiv-preprint-red"></a>
3
+ <a href="https://yzc-ippl.github.io/PRAC/"><img src="https://img.shields.io/badge/Homepage-green"></a>
4
+ <a href='https://yzc-ippl.github.io/PRAC/stargazers'><img src='https://img.shields.io/github/stars/yzc-ippl/PRAC.svg?style=social'></a>
5
+ </div>
6
+
7
+ <h1 align="center">Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging</h1>
8
+
9
+ <div align="center">
10
+ Zhichao Yang<sup>1†</sup>,
11
+ Tianjiao Gu<sup>1†</sup>,
12
+ Zhixianhe Zhang<sup>1</sup>,
13
+ Xiangfei Sheng<sup>1</sup>,
14
+ Pengfei Chen<sup>1</sup>,
15
+ Leida Li<sup>1,2*</sup>
16
+ </div>
17
+
18
+ <div align="center">
19
+ <sup>1</sup>School of Artificial Intelligence,
20
+ <sup>2</sup>State Key Laboratory of EMIM, Xidian University
21
+ </div>
22
+
23
+ <div align="center">
24
+ <sup>†</sup>Equal contribution &nbsp;&nbsp; <sup>*</sup>Corresponding author
25
+ </div>
26
+
27
+ <br>
28
+
29
+ <div align="center">
30
+ <img src="PRAC.png" width="900"/>
31
+ </div>
32
+
33
+ <div style="font-family: sans-serif; margin-bottom: 2em;">
34
+ <h2 style="border-bottom: 1px solid #eaecef; padding-bottom: 0.3em; margin-bottom: 1em;">News</h2>
35
+ <ul style="list-style-type: none; padding-left: 0;">
36
+ <li style="margin-bottom: 0.8em;">
37
+ <strong>[2026-10-03]</strong> ✨</span>✨</span> The <strong>Code</strong> and <strong>Pre-trained Weights</strong>, are now publicly available.
38
+ </li>
39
+ <li style="margin-bottom: 0.8em;">
40
+ <strong> [2026-08-28]</strong> πŸŽ‰</span>πŸŽ‰</span> Congratulations! Our paper has been accepted for an <strong>Oral Presentation</strong> at ACM MM 2026.
41
+ </li>
42
+ <li style="margin-bottom: 0.8em;">
43
+ <strong>[2026-07-10]</strong> πŸŽ‰</span>πŸŽ‰</span> Our paper, "Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging", has been accepted to <strong>ACMMM 2026</strong>!
44
+ </li>
45
+ </ul>
46
+ </div>
47
+
48
+ ## Introduction
49
+
50
+ This release contains the PARA implementation of PRAC: the Generic Aesthetic Predictor (GIAA), PreferSelect, PreferMerge, data loaders, and experiment configurations. Run commands from the repository root.
51
+
52
+ ## Quick Start
53
+
54
+ ### 1. Create the environment
55
+
56
+ The following environment matches the released configuration. Install the PyTorch build that matches the CUDA runtime available to the target GPU; the example uses PyTorch 2.4.1 with CUDA 12.1.
57
+
58
+ ```bash
59
+ conda create -n prac python=3.10 -y
60
+ conda activate prac
61
+ conda install pytorch==2.4.1 torchvision==0.19.1 pytorch-cuda=12.1 -c pytorch -c nvidia
62
+ pip install -r requirements.txt
63
+ ```
64
+
65
+ ### 2. Download the model and data
66
+
67
+ Download the [mPLUG-Owl3-7B-241101 base checkpoint](https://huggingface.co/mPLUG/mPLUG-Owl3-7B-241101) and place it at `models/mPLUG-Owl3-7B`, or set `model.model_path` in the YAML files to another compatible local path. Then you can download the pretrained adapter from [Baidu Netdisk](https://pan.baidu.com/s/1pK-BDCCJLWci0tawbfYrog?pwd=3rd3) (pwd=`3rd3`).
68
+
69
+ Obtain the PARA dataset from [this link](https://web.xidian.edu.cn/ldli/en/dataset.html) and extract the archive's directory to `data`. The loaders also need derived files. Generate them from the original annotations with:
70
+
71
+ ```bash
72
+ python scripts/prepare_giaa_labels.py --source data/PARA --output data/PARA
73
+ ```
74
+
75
+ The final data layout is:
76
+
77
+ ```text
78
+ data/PARA/
79
+ β”œβ”€β”€ README.md # original dataset README
80
+ β”œβ”€β”€ imgs/<session_id>/<image_name> # original images
81
+ β”œβ”€β”€ annotation/ # original PARA annotations
82
+ β”œβ”€β”€ para_giaa_train_discretized.csv # from annotation/PARA-GiaaTrain.csv
83
+ β”œβ”€β”€ para_giaa_test_discretized.csv # from annotation/PARA-GiaaTest.csv
84
+ └── ... # other original PARA files
85
+ ```
86
+
87
+ The personalized-data loaders read the original `annotation/PARA-Images.csv` directly. User prompts are kept in the repository at `data/user_info_para.json`; the label-generation script does not copy or modify either file.
88
+
89
+ ### 3. Run the inference demo
90
+
91
+ After the base checkpoint and PARA images are available, the demo automatically scores the first image below `data/PARA/imgs`:
92
+
93
+ ```bash
94
+ python demo_inference.py
95
+ ```
96
+
97
+ To use a particular image, a trained adapter, or a generated rationale:
98
+
99
+ ```bash
100
+ python demo_inference.py \
101
+ --image data/PARA/imgs/<session_id>/<image_name> \
102
+ --adapter-path runs/para/giaa_adapter \
103
+ --explain
104
+ ```
105
+
106
+ ## Repository Structure
107
+
108
+ ```text
109
+ .
110
+ β”œβ”€β”€ configs/
111
+ β”‚ β”œβ”€β”€ giaa_train_config/ GIAA training configuration
112
+ β”‚ β”œβ”€β”€ giaa_eval_config/ GIAA evaluation and demo configuration
113
+ β”‚ β”œβ”€β”€ preferselect_config/ CCM/PDM data and model configuration
114
+ β”‚ └── prefermerge_config/ LoRA-pool, FIM, and merge configuration
115
+ β”œβ”€β”€ data/ Dataset layout and download notes
116
+ β”œβ”€β”€ data_loader/ GIAA and personalized PARA data loaders
117
+ β”œβ”€β”€ modules/MplugOwl3ForGIAA.py mPLUG-Owl3 wrapper and LoRA utilities
118
+ β”œβ”€β”€ demo_inference.py Single-image distribution and score demo
119
+ β”œβ”€β”€ mplug3_train_giaaBackbone.py Train the generic aesthetic predictor
120
+ β”œβ”€β”€ mplug3_eval_giaaBackbone.py Evaluate a GIAA adapter
121
+ β”œβ”€β”€ preferSelect_CCM.py Compute collective controversy (CCM)
122
+ β”œβ”€β”€ preferSelect_PDM_fast.py Compute cached personalized deviation (PDM)
123
+ β”œβ”€β”€ preferSelect_weights_ablation.py
124
+ β”‚ Combine CCM and PDM for all alpha values
125
+ β”œβ”€β”€ preferSelect.py Export ranked preference-rich samples
126
+ β”œβ”€β”€ preferMerge.py Train LoRA pool or run adapter merging
127
+ β”œβ”€β”€ preferMerge_FIM_fixed.py Compute FIM preference similarities
128
+ β”œβ”€β”€ preferMerge_choice_fast.py Select a relevance/diversity cohort
129
+ β”œβ”€β”€ utils.py Shared losses, metrics, and configuration helpers
130
+ β”œβ”€β”€ CITATION.cff Citation metadata
131
+ β”œβ”€β”€ LICENSE MIT license for the PRAC source code
132
+ └── docs/ Project page and reproduction guide
133
+ ```
134
+
135
+ ## Re-Training
136
+
137
+ ### 1. Generic Aesthetic Predictor
138
+
139
+ Edit [`configs/giaa_train_config/para.yaml`](configs/giaa_train_config/para.yaml) if the model, data, batch size, or output path differs from the defaults, then run:
140
+
141
+ ```bash
142
+ python mplug3_train_giaaBackbone.py
143
+ ```
144
+
145
+ The trainer writes LoRA checkpoints below `runs/para/giaa`. Evaluate a selected adapter with:
146
+
147
+ ```bash
148
+ python mplug3_eval_giaaBackbone.py
149
+ ```
150
+
151
+ ### 2. Preference-rich Sample Mining (PreferSelect)
152
+
153
+ Run the commands below:
154
+
155
+ ```bash
156
+ python preferSelect_CCM.py
157
+ python preferSelect_PDM_fast.py
158
+ python preferSelect_weights_ablation.py
159
+ python preferSelect.py
160
+ ```
161
+
162
+ ### 3. Aesthetically-resonant Cohort Merging (PreferMerge)
163
+
164
+ Run the commands below:
165
+
166
+ ```bash
167
+ python preferMerge_FIM_fixed.py
168
+ python preferMerge_choice_fast.py --top-n 6 --beta 0.5
169
+ python preferMerge.py
170
+ ```
171
+
172
+ ## Citation
173
+
174
+ If this code or paper helps your research, please cite:
175
+
176
+ ```bibtex
177
+ @inproceedings{yang2026personalized,
178
+ title={Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging},
179
+ author={Yang, Zhichao and Gu, Tianjiao and Zhang, Zhixianhe and Sheng, Xiangfei and Chen, Pengfei and Li, Leida},
180
+ booktitle={Proceedings of the 34th ACM International Conference on Multimedia},
181
+ year={2026}
182
+ }
183
+ ```
184
+
185
+ Citation metadata is also available in [`CITATION.cff`](CITATION.cff).
186
+
187
+ ## License
188
+
189
+ The PRAC source code is available under the [MIT License](LICENSE).