Hugging Face's logo

facebook
/
mms-1b-all

Automatic Speech Recognition
Transformers
PyTorch
Safetensors
wav2vec2
mms
Eval Results
Model card Files
xet
 Community
26

is mms compatible with PEFT (parameter efficient fine tuning)?

#8
by mohblnk - opened
Discussion
mohblnk

PEFT is amazing, I have been fine tunning whisper large model which require about ~24 Vram to fine tune but with PEFT I was able to use my 16GB just fine, I was wondering if PEFT is compatible with MMS yet? if not is there is a TBA date?
sanchit-gandhi

You can check out this blog-post for a low-resource approach to fine-tuning MMS for ASR: https://huggingface.co/blog/mms_adapters

We only fine tune the adapter weights (a few million params, only a fraction of the full model weights), so I expect 16GB VRAM is more than enough here! Let me know if it's not though - happy to provide some more pointers for reducing memory with MMS fine-tuning if it's still too high!
mohblnk

You can check out this blog-post for a low-resource approach to fine-tuning MMS for ASR: https://huggingface.co/blog/mms_adapters

We only fine tune the adapter weights (a few million params, only a fraction of the full model weights), so I expect 16GB VRAM is more than enough here! Let me know if it's not though - happy to provide some more pointers for reducing memory with MMS fine-tuning if it's still too high!

I have taken a quick look and looks like I'm in for a treat, Thank you @sanchit-gandhi

Sign up or log in to comment