vuiseng9
/

01_moedl_dense-0119

Text Generation

Generated from Trainer

Eval Results (legacy)

Model card Files Files and versions

01_moedl_dense-0119

This model is a dense (Moedl) pretrained with roneneldan/TinyStories dataset.

Eval Loss: 1.1112
Accuracy: 0.6932
Num Input Tokens Seen: 1027671040
wandb log

To run or reproduce: follow moe-lab

Downloads last month: -

Safetensors

Model size

0.1B params

Tensor type

F32

·

Dataset used to train vuiseng9/01_moedl_dense-0119

Collection including vuiseng9/01_moedl_dense-0119

moe-lab

https://github.com/vuiseng9/moe-lab • 2 items • Updated about 5 hours ago

Evaluation results

Accuracy on roneneldan/TinyStories
self-reported

0.693