| # Frontdoor-Adjustment-Backdoor-Elimination (FABE) |
|
|
| Fine-tuned FABE model for "[Causality Based Front-door Defense Against Backdoor Attack on Language Models](https://icml.cc/virtual/2024/poster/33536)". |
|
|
| The demo code can be found in the [Frontdoor-Adjustment-Backdoor-Elimination](https://github.com/lyr17/Frontdoor-Adjustment-Backdoor-Elimination). |
|
|
| ## Replication |
|
|
| We use [Tuna](https://github.com/microsoft/LMOps/tree/main/tuna) finetune an instruction-tuned LLM as FABE model, the base LLM model is LlaMA2-7B-Chat, then we use [Openbackdoor](https://github.com/thunlp/OpenBackdoor) as a framework to finish the defensive process of FABE. |
|
|
| ### Installation |
|
|
| you can install FABE through Git. |
|
|
| **Git** |
|
|
| ``` |
| https://github.com/lyr17/Instruct-as-backdoor-cleaner.git |
| cd Instruct-as-backdoor-cleaner |
| pip install -r requirements.txt |
| ``` |
|
|
| ### Usage |
|
|
| #### Step 1 : finetune the base model to get FABE model |
|
|
| ``` |
| cd Tuna |
| bash src/train_tuna.sh data/llama_file.json 1e-5 |
| cd .. |
| ``` |
|
|
| We use 8×V100 GPU train the model 24 hours to get the finetuned model and model path like *tuna/src/checkpoints/tuna_p/checkpoint-3024* . |
|
|
| #### Step 2 : defense the victim model by using FABE model |
|
|
| You can configure the hyperparameters of FABE in these path ./configs/fabe_config.json, please set the hyperparameter "model_path" for fabe in this json file to the model path saved in step 1. |
|
|
| The hyperparameter "diversity" is the model generation diversity penalty for FABE model. When defending against BadNets and AddSent attacks, it is recommended to set diversity to 0.1, when defending against SynBkd attack, it is recommended to set diversity to 1.0. |
|
|
| ``` |
| cd OpenBackdoor |
| python FABE_defense.py --config_path ./configs/fabe_config.json |
| cd .. |
| ``` |
|
|
