README.md

Language Specific Neuron SLA

This is done specifically for the Qwen2.5 family of models

Guide

1. Run load_data.py to fetch data from https://huggingface.co/datasets/wikimedia/wikipedia/viewer/20231101
2. Calculate the activation from the fetched data with activation.py
3. Identify language specific neurons with identify.py

Ref

• https://github.com/ReML-AI/DCL-CoT
• https://github.com/RUCAIBox/Language-Specific-Neurons

Note taking

python3 load_data_oscar.py --languages en,zh,eu,ga --model-id qwen2.5 --tokenizer Qwen/Qwen2.5-0.5B