| # HackerNet v1 — Training Notes | |
| - **Avg Loss**: 0.0000 | |
| - **Datasets**: OpenAssistant/oasst_top1_2023-08-25, ai4bharat/indic-instruct-data-v0.1, HydraLM/hindi_multiturn_conversations, uonlp/CulturaX, HydraLM/hinglish_multiturn_conversations, festvox/cmu_indic, HuggingFaceH4/ultrachat_200k, teknium/OpenHermes-2.5 | |
| - **Layers**: 16 | |
| - **Hidden Dim**: 768 | |
| - **MoE Experts**: 4 | |
| - **Precision**: float16 | |
| - **Epochs**: 4 | |