metadata
library_name: peft
pipeline_tag: text-generation
base_model:
- meta-llama/Meta-Llama-3-8B-Instruct
- Qwen/Qwen2.5-7B-Instruct
- meta-llama/Llama-2-7b-chat-hf
DynamicPO: Dynamic Preference Optimization for Recommendation
This repository contains the model weights (LoRA adapters) for DynamicPO, a plug-and-play dynamic preference optimization framework for LLM-based recommender systems.
DynamicPO is designed to align Large Language Models (LLMs) with user preferences while mitigating "preference optimization collapse." This phenomenon occurs in multi-negative alignment when increasing the number of negative samples leads to performance degradation despite a decreasing training loss.
Key Features
DynamicPO comprises two adaptive mechanisms:
- Dynamic Boundary Negative Selection: Identifies and prioritizes informative negatives near the model's decision boundary.
- Dual-Margin Dynamic beta Adjustment: Calibrates optimization strength per sample according to boundary ambiguity.
Resources
- Paper: DynamicPO: Dynamic Preference Optimization for Recommendation
- GitHub Repository: xingyuHuxingyu/DynamicPO
- Dataset: DynamicPO Dataset
Base Models
- meta-llama/Llama-2-7b-chat-hf
- meta-llama/Meta-Llama-3-8B-Instruct
- Qwen/Qwen2.5-7B-Instruct
Citation
This work received DASFAA 2026 Best Paper Award. If you find this work useful, please consider citing:
@inproceedings{hu2026dynamicpo,
title={DynamicPO: Dynamic Preference Optimization for Recommendation},
author={Hu, Xingyu and Zhang, Kai and Wu, Jiancan and Wang, Shuli and Wang, Chi and Chen, Wenshuai and Zhu, Yinhua and Wang, Haitao and Wang, Xingxing and Wang, Xiang},
booktitle={International Conference on Database Systems for Advanced Applications},
pages={372--387},
year={2026},
organization={Springer}
}
Acknowledgment
This implementation is built upon the TRL library.