Do you have any recommended training hyperparameters, or are you planning to open-source your own training library?
#4
by
win10
- opened
Do you have any recommended training hyperparameters, or are you planning to open-source your own training library?
Hello, I want to train a phi4-14b version of the model. My h100 computing power is rented until tomorrow. Can you please help me?If you want to verify model scaling rules, we can collaborate!
@moelanoby
Sure we can collab my recommended hyper parameters is increasing the number of epochs to 250 and make sure to use a diverse dataset and you can change the baked in self corrections found inside the code :)
It would be better if you added me on discord (moelanobyzedev btw)
Sorry, could you please add it again?
win10
changed discussion status to
closed