Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
Paper • 2403.03432 • Published • 1
Attempt to reproduce Mixture-of-LoRAs classifier
Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
https://arxiv.org/pdf/2403.03432
Datasets
We evenly sample about 10k training data and 2k validation data on each dataset.
From laion/OIG was taken only:
- unified_merged_code_xp3.jsonl
- unified_grade_school_math_instructions.jsonl
- unified_mathqa_flanv2_kojma_cot.jsonl
- Downloads last month
- 5
from adapters import AutoAdapterModel model = AutoAdapterModel.from_pretrained("roberta-base") model.load_adapter("evilfreelancer/moa-classification", set_active=True)