|
|
--- |
|
|
license: mit |
|
|
base_model: |
|
|
- Qwen/Qwen3-1.7B |
|
|
- Qwen/Qwen3-4B |
|
|
- Qwen/Qwen3-8B |
|
|
- google/gemma-3-1b-it |
|
|
- google/gemma-3-4b-it |
|
|
--- |
|
|
|
|
|
This repository contains the weight diffs and DIT adapters used in the paper [Learning to Interpret Weight Differences in Language Models (Goel et al. 2025)](https://arxiv.org/abs/2510.05092). |
|
|
This paper introduces *Diff Interpretation Tuning*, a method that trains a LoRA adapter than can be applied to a model to get it to describe its own finetuning induced modifications. |
|
|
|
|
|
To play around with the weight diffs and DIT adapters from the paper, please check out our [Google Colab demo notebook](https://colab.research.google.com/drive/12YD_9GRT-y_hFOBqXzyI4eN_lJGKiXwN?usp=sharing). |
|
|
The code used to train and evaluate the weight diffs and DIT adapters can be found at [github.com/Aviously/diff-interpretation-tuning](https://github.com/Aviously/diff-interpretation-tuning). |
|
|
|
|
|
You can cite our work using the following bibtex |
|
|
``` |
|
|
@misc{goel2025learninginterpretweightdifferences, |
|
|
title={Learning to Interpret Weight Differences in Language Models}, |
|
|
author={Avichal Goel and Yoon Kim and Nir Shavit and Tony T. Wang}, |
|
|
year={2025}, |
|
|
eprint={2510.05092}, |
|
|
archivePrefix={arXiv}, |
|
|
url={https://arxiv.org/abs/2510.05092}, |
|
|
} |
|
|
``` |
