| Need to work with OpenDelta | |
| ``` | |
| from transformers import AutoModelForSeq2SeqLM | |
| t5 = AutoModelForSeq2SeqLM.from_pretrained("t5-base") | |
| from opendelta import AutoDeltaModel | |
| delta = AutoDeltaModel.from_finetuned("DeltaHub/lora_t5-base_mrpc", backbone_model=t5) | |
| delta.log() | |
| ``` | |