Artoria0429/code_portrait_track_1

Portrait Composition Aesthetics Evaluation

This project provides an inference and fusion pipeline for portrait composition aesthetics evaluation. It converts the official test set into multi-prompt prediction inputs, runs prediction with Stage2 LoRA adapters, and fuses multiple prediction files into the final track_1_test.json.

1. Project Structure

project_root/
  prepare_test_prompt_v2.py          # Build multi-prompt test inputs
  convert_to_answer.py               # Fuse multi-path prediction results
  train_track1_stage1_qwen3vl8b_v1.yaml
  train_track1_stage2_qwen3vl8b_v1.yaml
  qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/
    seed20260511_stage2/             # Stage2 adapter 1
    seed20260512_stage2/             # Stage2 adapter 2
  datasets/
    images/                          # Test images
    original_annotations/
      track_1_test.json              # Official test template
  outputs/
    predictions/                     # Model prediction outputs
    submissions/answers/             # Final submission file

For path conventions, see REPRODUCIBLE_PATHS.md.

2. Environment

Use an environment that supports Qwen3-VL and LoRA inference. LLaMA-Factory is recommended for prediction.

Basic dependencies:

pip install torch transformers peft accelerate

If you use LLaMA-Factory, install it first and make sure llamafactory-cli is available.

Base model:

Qwen/Qwen3-VL-8B-Instruct

3. Prepare Test Data

Place the official test file at:

datasets/original_annotations/track_1_test.json

Place test images under:

datasets/images/

Generate multi-prompt test inputs:

python prepare_test_prompt_v2.py \
  --input_json datasets/original_annotations/track_1_test.json \
  --image_dir datasets/images \
  --output_json outputs/predictions/track1_test_convert.json \
  --prompt_variants v1,v2,v3,v4,v6,v7,v8

This command generates:

outputs/predictions/track1_test_convert_v1.json
outputs/predictions/track1_test_convert_v2.json
outputs/predictions/track1_test_convert_v3.json
outputs/predictions/track1_test_convert_v4.json
outputs/predictions/track1_test_convert_v6.json
outputs/predictions/track1_test_convert_v7.json
outputs/predictions/track1_test_convert_v8.json

4. Run Multi-Path Prediction

Use the Stage2 adapters to run prediction on the generated prompt variants:

qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/seed20260511_stage2
qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/seed20260512_stage2

If you use LLaMA-Factory, register each test JSON in dataset_info.json. Example:

{
  "track1_test_v1": {
    "file_name": "outputs/predictions/track1_test_convert_v1.json",
    "formatting": "sharegpt",
    "columns": {
      "messages": "messages",
      "images": "images"
    }
  }
}

Example prediction config:

model_name_or_path: Qwen/Qwen3-VL-8B-Instruct
adapter_name_or_path: qwen3vl8b_r1stage2_seed_ensemble_fast_20260511_112825/seed20260511_stage2
template: qwen3_vl_nothink
stage: sft
finetuning_type: lora

dataset_dir: .
eval_dataset: track1_test_v1
do_predict: true
predict_with_generate: true
cutoff_len: 4096
image_max_pixels: 786432
per_device_eval_batch_size: 8
max_new_tokens: 384
do_sample: false
top_p: 1.0
repetition_penalty: 1.02
bf16: true
flash_attn: auto

output_dir: outputs/predictions/seed20260511_v1

Run prediction:

llamafactory-cli train predict_seed20260511_v1.yaml

Each prediction run should produce:

outputs/predictions/<run_name>/generated_predictions.jsonl

5. Fuse Prediction Results

Copy or symlink the official test template to:

outputs/predictions/track_1_test.json

Fuse prediction results:

python convert_to_answer.py \
  --template_json outputs/predictions/track_1_test.json \
  --predictions_jsonl \
    outputs/predictions/seed20260511_v1/generated_predictions.jsonl \
    outputs/predictions/seed20260511_v2/generated_predictions.jsonl \
    outputs/predictions/seed20260511_v3/generated_predictions.jsonl \
    outputs/predictions/seed20260511_v4/generated_predictions.jsonl \
    outputs/predictions/seed20260511_v6/generated_predictions.jsonl \
    outputs/predictions/seed20260511_v7/generated_predictions.jsonl \
    outputs/predictions/seed20260511_v8/generated_predictions.jsonl \
  --weights 1 1 1 1 1 1 1 \
  --best_index 0 \
  --total_score_fusion mean \
  --output_json outputs/submissions/answers/track_1_test.json

To fuse results from both Stage2 adapters, add all corresponding generated_predictions.jsonl files to --predictions_jsonl and provide the same number of values in --weights.

6. Output

The final result is:

outputs/submissions/answers/track_1_test.json

The file preserves the official template structure and fills in the fused:

• criteria.level
• total_score
• answer

7. Notes

• All paths are relative to the project root.
• Image paths use the format datasets/images/<image_name>.jpg.
• convert_to_answer.py supports multi-path prediction fusion, weighted answer voting, majority voting for levels, and mean fusion for total_score.
• If the model output is not strict JSON, the fusion script attempts to extract the JSON snippet and answer field from text.