Upload 4 files

README.md

@@ -0,0 +1,49 @@
+---
+license: mit
+datasets:
+- chengpingan/CoConflictQA
+language:
+- en
+base_model:
+- meta-llama/Meta-Llama-3-8B
+library_name: transformers
+---
+
+# 🤖 ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation
+
+This is the official model for **[ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation](https://arxiv.org/pdf/2502.15543)**.  
+
+We investigate the internal mechanisms behind unfaithful generation and identify a subset of **mid-to-deep (70%–90% relative depth range) FFNs** that are disproportionately activated in such cases. Building on this insight, we propose Parametric Knowledge Muting through FFN Suppression (**ParamMute**), a framework that improves contextual faithfulness by suppressing the activation of unfaithfulness-associated FFNs and calibrating the model toward retrieved knowledge. Experimental results on CoConflictQA and ConFiQA demonstrate that ParamMute significantly reduces knowledge conflicts and improves context fidelity.
+
+---
+
+## 📚 **Paper**
+For a detailed explanation of the methodology and experiments, please refer to our paper:  
+[**ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation**](https://arxiv.org/abs/2502.15543)
+
+---
+
+## 📊 Reproduce the Results
+To reproduce the experiments and benchmarks from the paper, follow the instructions provided in the official GitHub repository:
+[👉 GitHub: OpenBMB/ParamMute](https://github.com/OpenBMB/ParamMute).
+
+## 📁 Model Details
+- Model Name: ParamMute-8B-KTO
+- Architecture: LLaMA3-8B-Instruct trained with the KTO
+- Training Data: [CoConflictQA](https://huggingface.co/datasets/chengpingan/CoConflictQA) Dataset
+- Pretrained Tasks: Knowledge-Augmented Generation, Contextual Faithfulness Evaluation
+
+## 🔖 Citation
+If you use PIP-KAG in your work, please consider citing our paper:
+```
+@misc{huang2025parammutesuppressingknowledgecriticalffns,
+      title={ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation}, 
+      author={Pengcheng Huang and Zhenghao Liu and Yukun Yan and Haiyan Zhao and Xiaoyuan Yi and Hao Chen and Zhiyuan Liu and Maosong Sun and Tong Xiao and Ge Yu and Chenyan Xiong},
+      year={2025},
+      eprint={2502.15543},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2502.15543}, 
+}
+
+```

adapter_config.json

@@ -0,0 +1,40 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "LlamaForInputContrastivew_act_inhibit",
+    "parent_library": "transformers.models.llama.modeling_llama"
+  },
+  "base_model_name_or_path": "Models/llama3-8b-instruct",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj",
+    "up_proj",
+    "o_proj",
+    "gate_proj"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:403ffcdc5d9ffef165bf4133ce8c94eb7d8341eba696bd15e41a5923dc33f2f1
+size 671149168

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:614039b6ad6a6fd2c3c20bf02f849bc69de0bc9d00d2a023e716c2b33803d119
+size 5880