Learning Rate during pretraining
#58
by
shuyuej - opened
May I know the initial learning rate and scheduler settings during the pre-training process?
Thank you very much, and have a nice day!
Best regards,
Shuyue
July 18th, 2024
May I also know the max_grad_norm during the pre-training process?
Thank you very much, and have a nice day!
Best regards,
Shuyue
July 18th, 2024
alexrs changed discussion status to
closed