vuiseng9
/

c3_moedl_e32_k4-0119

Text Generation

Generated from Trainer

Eval Results (legacy)

Model card Files Files and versions

c3_moedl_e32_k4-0119

This model is a MoE (Moedl) pretrained with roneneldan/TinyStories dataset.

Eval Loss: 1.0581
Accuracy: 0.7043
Num Input Tokens Seen: 1027671040
wandb log

To run or reproduce: follow moe-lab

Downloads last month: 9

Safetensors

Model size

0.4B params

Tensor type

F32

·

Dataset used to train vuiseng9/c3_moedl_e32_k4-0119

Collection including vuiseng9/c3_moedl_e32_k4-0119

moe-lab

https://github.com/vuiseng9/moe-lab • 2 items • Updated 19 days ago

Evaluation results

Accuracy on roneneldan/TinyStories
self-reported

0.704