GreenGenomicsLab

/

Mambalga-revgra_43500

like 0

Hello,

I was wondering if you were able to successfully finetune the mamba model using LoRA? I've been trying to finetune the "state-spaces/mamba-130m-hf" model using LoRA but the x_proj and out_proj layers do not train and always have the param.grad as None. Since I noticed that you've been trying to train the same model using LoRA, i was hoping to check if you've found a solution to this issue?

A detailed description of my issue with code and output is here.

Thanks!