| ---
|
| library_name: peft
|
| license: other
|
| base_model: Qwen/Qwen3-VL-8B-Instruct
|
| pipeline_tag: image-text-to-text
|
| tags:
|
| - qwen3-vl
|
| - vision-language
|
| - portrait-aesthetics
|
| - aesthetics-evaluation
|
| - lora
|
| - peft
|
| - llama-factory
|
| ---
|
|
|
| # Portrait Composition Aesthetics Evaluation
|
|
|
| This project provides an inference and fusion pipeline for portrait composition aesthetics evaluation. It converts the official test set into multi-prompt prediction inputs, runs prediction with Stage2 LoRA adapters, and fuses multiple prediction files into the final `track_1_test.json`.
|
|
|
| ## 1. Project Structure
|
|
|
| ```text
|
| project_root/
|
| prepare_test_prompt_v2.py # Build multi-prompt test inputs
|
| convert_to_answer.py # Fuse multi-path prediction results
|
| train_track1_stage1_qwen3vl8b_v1.yaml
|
| train_track1_stage2_qwen3vl8b_v1.yaml
|
| qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/
|
| seed20260511_stage2/ # Stage2 adapter 1
|
| seed20260512_stage2/ # Stage2 adapter 2
|
| datasets/
|
| images/ # Test images
|
| original_annotations/
|
| track_1_test.json # Official test template
|
| outputs/
|
| predictions/ # Model prediction outputs
|
| submissions/answers/ # Final submission file
|
| ```
|
|
|
| For path conventions, see `REPRODUCIBLE_PATHS.md`.
|
|
|
| ## 2. Environment
|
|
|
| Use an environment that supports Qwen3-VL and LoRA inference. LLaMA-Factory is recommended for prediction.
|
|
|
| Basic dependencies:
|
|
|
| ```bash
|
| pip install torch transformers peft accelerate
|
| ```
|
|
|
| If you use LLaMA-Factory, install it first and make sure `llamafactory-cli` is available.
|
|
|
| Base model:
|
|
|
| ```text
|
| Qwen/Qwen3-VL-8B-Instruct
|
| ```
|
|
|
| ## 3. Prepare Test Data
|
|
|
| Place the official test file at:
|
|
|
| ```text
|
| datasets/original_annotations/track_1_test.json
|
| ```
|
|
|
| Place test images under:
|
|
|
| ```text
|
| datasets/images/
|
| ```
|
|
|
| Generate multi-prompt test inputs:
|
|
|
| ```bash
|
| python prepare_test_prompt_v2.py \
|
| --input_json datasets/original_annotations/track_1_test.json \
|
| --image_dir datasets/images \
|
| --output_json outputs/predictions/track1_test_convert.json \
|
| --prompt_variants v1,v2,v3,v4,v6,v7,v8
|
| ```
|
|
|
| This command generates:
|
|
|
| ```text
|
| outputs/predictions/track1_test_convert_v1.json
|
| outputs/predictions/track1_test_convert_v2.json
|
| outputs/predictions/track1_test_convert_v3.json
|
| outputs/predictions/track1_test_convert_v4.json
|
| outputs/predictions/track1_test_convert_v6.json
|
| outputs/predictions/track1_test_convert_v7.json
|
| outputs/predictions/track1_test_convert_v8.json
|
| ```
|
|
|
| ## 4. Run Multi-Path Prediction
|
|
|
| Use the Stage2 adapters to run prediction on the generated prompt variants:
|
|
|
| ```text
|
| qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/seed20260511_stage2
|
| qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/seed20260512_stage2
|
| ```
|
|
|
| If you use LLaMA-Factory, register each test JSON in `dataset_info.json`. Example:
|
|
|
| ```json
|
| {
|
| "track1_test_v1": {
|
| "file_name": "outputs/predictions/track1_test_convert_v1.json",
|
| "formatting": "sharegpt",
|
| "columns": {
|
| "messages": "messages",
|
| "images": "images"
|
| }
|
| }
|
| }
|
| ```
|
|
|
| Example prediction config:
|
|
|
| ```yaml
|
| model_name_or_path: Qwen/Qwen3-VL-8B-Instruct
|
| adapter_name_or_path: qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/seed20260511_stage2
|
| template: qwen3_vl_nothink
|
| stage: sft
|
| finetuning_type: lora
|
|
|
| dataset_dir: .
|
| eval_dataset: track1_test_v1
|
| do_predict: true
|
| predict_with_generate: true
|
| cutoff_len: 4096
|
| image_max_pixels: 786432
|
| per_device_eval_batch_size: 8
|
| max_new_tokens: 384
|
| do_sample: false
|
| top_p: 1.0
|
| repetition_penalty: 1.02
|
| bf16: true
|
| flash_attn: auto
|
|
|
| output_dir: outputs/predictions/seed20260511_v1
|
| ```
|
|
|
| Run prediction:
|
|
|
| ```bash
|
| llamafactory-cli train predict_seed20260511_v1.yaml
|
| ```
|
|
|
| Each prediction run should produce:
|
|
|
| ```text
|
| outputs/predictions/<run_name>/generated_predictions.jsonl
|
| ```
|
|
|
| ## 5. Fuse Prediction Results
|
|
|
| Copy or symlink the official test template to:
|
|
|
| ```text
|
| outputs/predictions/track_1_test.json
|
| ```
|
|
|
| Fuse prediction results:
|
|
|
| ```bash
|
| python convert_to_answer.py \
|
| --template_json outputs/predictions/track_1_test.json \
|
| --predictions_jsonl \
|
| outputs/predictions/seed20260511_v1/generated_predictions.jsonl \
|
| outputs/predictions/seed20260511_v2/generated_predictions.jsonl \
|
| outputs/predictions/seed20260511_v3/generated_predictions.jsonl \
|
| outputs/predictions/seed20260511_v4/generated_predictions.jsonl \
|
| outputs/predictions/seed20260511_v6/generated_predictions.jsonl \
|
| outputs/predictions/seed20260511_v7/generated_predictions.jsonl \
|
| outputs/predictions/seed20260511_v8/generated_predictions.jsonl \
|
| --weights 1 1 1 1 1 1 1 \
|
| --best_index 0 \
|
| --total_score_fusion mean \
|
| --output_json outputs/submissions/answers/track_1_test.json
|
| ```
|
|
|
| To fuse results from both Stage2 adapters, add all corresponding `generated_predictions.jsonl` files to `--predictions_jsonl` and provide the same number of values in `--weights`.
|
|
|
| ## 6. Output
|
|
|
| The final result is:
|
|
|
| ```text
|
| outputs/submissions/answers/track_1_test.json
|
| ```
|
|
|
| The file preserves the official template structure and fills in the fused:
|
|
|
| - `criteria.level`
|
| - `total_score`
|
| - `answer`
|
|
|
| ## 7. Notes
|
|
|
| - All paths are relative to the project root.
|
| - Image paths use the format `datasets/images/<image_name>.jpg`.
|
| - `convert_to_answer.py` supports multi-path prediction fusion, weighted answer voting, majority voting for levels, and mean fusion for `total_score`.
|
| - If the model output is not strict JSON, the fusion script attempts to extract the JSON snippet and answer field from text.
|
|
|
