Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 8
Papers about model merging
referenced in the mergekit repo: https://github.com/cg123/mergekit
Note Linear merging
Editing Models with Task Arithmetic
Paper • 2212.04089 • Published • 8Note Task Arithmetic
Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 19Note TIES
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Paper • 2311.03099 • Published • 33Note DARE