social-data-attribution-w2/SIMPLE_README.md

Download the query files off of Hugging Face (or ask me for them if you don't have access yet)
```
git clone https://github.com/EleutherAI/bergson.git
# Change Bergson's transformers version to 4.57.0
uv sync
```
If you are on Phoenix set the account/partition in the Slurm Scripts
Sample tests:
```
sbatch scripts/slurm/download_data.sh #Something is wrong with this script as a lot of the data collected is truncated
sbatch scripts/slurm/simple_enrich_shards.sh #Make sure the json file folder location matches

# These two can be run at the same time
sbatch scripts/slurm/data_index.sh #Again make sure data is in the right location
sbatch scripts/slurm/query_index.sh #Make sure the query json file location matches


sbatch scripts/slurm/influence.sh #Make sure the gradient folders match the outputs from the previous scripts
```
Finally run analysis such as:
```
uv run -m src.data_attribution.analysis.clusters #With the right parameters in the command based on the jsonl locations
```