base_model:
- meta-llama/Meta-Llama-3-8B-Instruct
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
RESM-Llama-3-8B-Instruct
This repository contains the model presented in the paper Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging.
Description
The authors propose a novel merging method called RESM (Reweighting Enhanced task Singular Merging) to achieve a balanced alignment of Large Language Models (LLMs) across three critical dimensions: Helpfulness, Honesty, and Harmlessness (3H optimization).
RESM addresses the challenges of preference noise accumulation and layer sparsity adaptation inherent in 3H-aligned LLM merging through outlier weighting and sparsity-aware rank selection strategies. Evaluations demonstrate that RESM effectively mitigates conflicts among the 3H dimensions compared to traditional data mixture or standard model merging methods.
Citation
@article{yang2025mix,
title={Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging},
author={Yang, Jinluan and Jin, Dingnan and Tang, Anke and Shen, Li and Zhu, Didi and Chen, Zhengyu and Wang, Daixin and Cui, Qing and Zhang, Zhiqiang and Zhou, Jun and others},
journal={arXiv preprint arXiv:2502.06876},
year={2025}
}