SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation

This repository contains the model checkpoints for SHIFT, a lightweight framework designed to resolve knowledge conflicts in retrieval-augmented generation (RAG).

• Paper: SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation
• Repository: GitHub - OpenBMB/SHIFT

Method Overview

SHIFT reformulates neuron-level modification as a learnable gate modulation, allowing LLMs to adaptively regulate internal activations for knowledge conflict resolution. Technically, SHIFT equips LLMs with a lightweight gate module and optimizes fewer than 0.01% trainable parameters while keeping the backbone model frozen. During generation, the gate module adjusts the model's internal representations to adaptively leverage contextual and parametric knowledge.

Setup and Usage

Please refer to the official GitHub Repository for detailed environment setup, training, and evaluation scripts.

Citation

If you find this work useful, please cite the paper:

@misc{li2026shiftgatemodulatedactivationsteering,
      title={SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation}, 
      author={Ruochang Li and Pengcheng Huang and Zhenghao Liu and Yukun Yan and Huiyuan Xie and Yu Gu and Ge Yu and Maosong Sun},
      year={2026},
      eprint={2606.27786},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.27786}, 
}