README.md

---
license: mit
library_name: pytorch
tags:
- video-aesthetic-assessment
- video-quality-assessment
- aesthetics
- clip
- peak-end-net
---

# 🎬 Peak-End Net

**Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment**

**Accepted to ACM Multimedia 2026**

[📄 Paper](https://arxiv.org/abs/2607.13941) ·
[💻 Code](https://github.com/AMAP-ML/Peak-End-Net) ·
[🤗 Model](https://huggingface.co/GD-ML/Peak-End-Net/tree/main)

Peak-End Net is a video aesthetic assessment framework inspired by the
**peak-end rule**: people tend to judge an experience disproportionately by its
most salient moments and its ending, rather than by uniformly averaging the
entire experience. Peak-End Net translates this insight into a learnable
temporal model that predicts an overall aesthetic score together with ten
fine-grained attribute scores.

## ✨ Overview

![Peak-End Net pipeline](https://github.com/AMAP-ML/Peak-End-Net/raw/main/pipeline.png)

The framework contains five main components:

- **Frame Aesthetic Perceiver** — a frozen CLIP ViT-L/14 encoder and an
  AVA-pretrained aesthetic head produce a 10-bin score distribution and an
  expected aesthetic score for each frame.
- **Key Moment Discovery** — learnable peak-, valley-, and end-aware signals
  are combined into a unified temporal attention distribution.
- **Peak-End Aggregation** — attention-weighted pooling summarizes the frame
  features into a video-level representation.
- **Rhythm Encoder** — a multi-scale 1D CNN with kernel sizes 3, 5, and 7
  captures local fluctuations and longer-range trends in the frame-score
  sequence.
- **Gated Fusion** — a lightweight second-stage module adaptively combines the
  learned video-level score with the mean frame-level AVA score.

## 🚀 Pretrained Model and Inference

The self-contained `Peak-End-Net.pth` checkpoint includes the CLIP ViT-L/14
encoder, AVA aesthetic head, Peak-End modules, and gated-fusion module. No
separate AVA or Stage 1 checkpoint is required for inference.

Clone the code repository and install dependencies:

```bash
git clone https://github.com/AMAP-ML/Peak-End-Net.git
cd Peak-End-Net

conda create -n peak-end-net python=3.10 -y
conda activate peak-end-net
pip install -r requirements.txt
```

Download the checkpoint:

```bash
hf download GD-ML/Peak-End-Net Peak-End-Net.pth --local-dir ./checkpoints
```

Run inference on a video:

```bash
python inference.py \
    --checkpoint ./checkpoints/Peak-End-Net.pth \
    --video /path/to/video.mp4
```

The script reports the overall score, ten attribute scores
(`composition`, `shotsize`, `lighting`, `visualtone`, `color`, `depthoffield`,
`expression`, `movement`, `costume`, `makeup`), the fusion gate, and the two
scores combined by the gate.

## 🏋️ Training

Full training instructions — pretraining the AVA aesthetic head, Stage 1
(Peak-End Net), and Stage 2 (Gated Fusion) — are provided in the
[GitHub repository](https://github.com/AMAP-ML/Peak-End-Net).

## 📖 Citation

If you find this work useful, please cite:

```bibtex
@misc{li2026peakendnetpeakendruleinspired,
      title={Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment},
      author={Geng Li and Haiwen Li and Rui Chen and Jing Tang and Lei Sun and Xiangxiang Chu},
      year={2026},
      eprint={2607.13941},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.13941},
}
```

## 🤝 Acknowledgments

This project builds on [CLIP](https://github.com/openai/CLIP) and
[CLIP4Clip](https://github.com/ArrowLuo/CLIP4Clip).

## 📄 License

This project is released under the
[MIT License](https://github.com/AMAP-ML/Peak-End-Net/blob/main/LICENSE).