Fine-Tuning on Schubert’s music
#4
by
mikedimo
- opened
Has anyone run the fine-tuning example on Schubert’s works as suggested in the README of the NotaGen model’s Git repository? I’m running with the large pre-trained model and the default config training parameters (batch size: 1, learning_rate: 1e-5, num_epochs: 64), but I don’t observe any improvement as the epochs progress. Specifically, the best epoch is the 3rd (out of 64) with a minimum eval loss of 0.131. I want to perform fine-tuning on a separate dataset and I’m trying to figure out whether it’s my mistake or if they’re simply providing training parameters that aren’t representative for the data they supply.