| --- |
| license: mit |
| tags: |
| - training-data-attribution |
| - influence-functions |
| - interpretability |
| - nanochat |
| --- |
| |
| <div align="center"> |
| <h2>STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations</h2> |
|
|
| <a href="https://arxiv.org/abs/2606.05165"><img src='https://img.shields.io/badge/arXiv-STRIDE-red' alt='Paper'></a> |
| <a href='https://stride-tda.github.io'><img src='https://img.shields.io/badge/Project_Page-STRIDE-green' alt='Project Page'></a> |
| <a href='https://huggingface.co/rishitdagli/stride-nanochat'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20-Models-yellow'></a> |
| <a href='https://huggingface.co/datasets/rishitdagli/stride-lds'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20-LDS%20Dataset-yellow'></a> |
| </div> |
|
|
| Trained STRIDE steering operators and the nanochat base checkpoints they attribute, |
| for the four pre-training models in the paper. STRIDE attributes a model's |
| prediction back to the pre-training examples that shaped it, by learning a tiny |
| activation-steering operator instead of retraining. |
|
|
| | depth | params | base ckpt step | operator | LDS (Spearman) | |
| |-------|--------|----------------|----------|----------------| |
| | d12 | 286M | 1680 | layer 8, rank 16 | 0.156 | |
| | d16 | 537M | 3584 | layer 10, rank 16 | 0.177 | |
| | d20 | 897M | 3320 | layer 12, rank 16 | 0.158 | |
| | d24 | 1.38B | 5568 | layer 15, rank 16 | 0.165 | |
|
|
| ## Layout |
|
|
| ``` |
| base_checkpoints/<tag>/model_<step>.pt # nanochat base checkpoint |
| base_checkpoints/<tag>/meta_<step>.json # nanochat config |
| operators/<tag>/operator.pt # trained SteeringOperator state dict |
| operators/<tag>/subsets.npy # subset membership (K=1000, d=10) |
| operators/<tag>/meta.json # training config + n_train + operator dims |
| tokenizer/ # shared nanochat tokenizer |
| ``` |
|
|
| ## Usage |
|
|
| ```python |
| from stride.inference import Stride |
| attr = Stride.from_pretrained("d12") |
| result = attr.attribute(my_queries) |
| ``` |
|
|
| The LDS ground-truth losses and the held-out test set live in the dataset repo [`rishitdagli/stride-lds`](https://huggingface.co/datasets/rishitdagli/stride-lds). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{dagli2026stridetrainingdataattribution, |
| title={STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations}, |
| author={Rishit Dagli and Abir Harrasse and Luke Zhang and Florent Draye and Amirali Abdullah and Bernhard Schölkopf and Zhijing Jin}, |
| year={2026}, |
| eprint={2606.05165}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.LG}, |
| url={https://arxiv.org/abs/2606.05165}, |
| } |
| ``` |
|
|
