PEFT How to use acwkim/mod-dpo-better with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("PKU-Alignment/alpaca-7b-reproduced")
model = PeftModel.from_pretrained(base_model, "acwkim/mod-dpo-better") 
