Training costs question
Hi! This is a fantastic model, especially for the size 🤗 🐜
Could you provide more information about the costs involved in training to help compare to the larger alternatives? Would be particularly helpful to have overall training tokens, training infra and wall clock, or straight up dollar amounts 💲
Thanks in advance!
Hey @yjernite ! I added the training costs for the model here: https://huggingface.co/lm-provers/QED-Nano#software--hardware
Note that QED-Nano is trained on top of QED-Nano-SFT, which has its own training costs here: https://huggingface.co/lm-provers/QED-Nano-SFT#software--hardware
Using ~$3 per H100 hour as a rough estimate, the total cost to train QED-Nano end-to-end is ~ $28k. Not cheap, but still a lot cheaper than training a trillion parameter model :)