license: mit
pipeline_tag: text-generation
SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
SHINE (Scalable Hyper In-context NEtwork) is a scalable hypernetwork that can map diverse meaningful contexts into high-quality LoRA adapters for large language models (LLM) in a single forward pass.
- Paper: SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
- Repository: https://github.com/Yewei-Liu/SHINE
Description
By reusing the frozen LLM's own parameters in an in-context hypernetwork design and introducing architectural innovations, SHINE overcomes key limitations of prior hypernetworks and achieves strong expressive power with a relatively small number of parameters. It updates LLM parameters without any fine-tuning, and immediately enables complex question answering tasks related to the context without directly accessing the context, effectively transforming in-context knowledge to in-parameter knowledge in one pass.
Compared to traditional SFT-based adaptation, SHINE significantly saves time, computation, and memory costs while showing great potential for scaling.
Usage
For environment setup and detailed inference instructions, please refer to the official GitHub repository. The project provides an inference.ipynb notebook to quickly test the hypernetwork's ability to generate LoRA adapters from custom contexts.
Citation
@article{liu2025shine,
title={SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass},
author={Liu, Yewei and others},
journal={arXiv preprint arXiv:2602.06358},
year={2025}
}