---
license: mit
datasets:
- roneneldan/TinyStories
language:
- en
tags:
- moe
- en
---
VdrontMoE-1m-10e is an experimental AI based on the MoE architecture, with 10 experts across two layers, each containing 1 million parameters, trained on Tiny Stories (part of the dataset).  
*Note: it uses a custom architecture, use use.py if you don't want to rack your brain*

Examples of its output (use.py):  
Prompt: Once upon a time there was a little girl named Lucy. She loved to go to the store  

Generating...

============================================================
GENERATED TEXT:
============================================================
Once upon a time there was a little girl named Lucy. She loved to go to the store with her mom and play with her friends. One day, she saw a big tree with her mom.
"Hi, Lily, I want to help you?"
Sue said, "I'm sorry, I will help you." They had a big tree. They played together and had a new toy.
"Yes, I can be careful, you."
Lily and the ball all day. They played together. They played together and had a big tree. They were happy and
============================================================
Generated 100 new tokens

Prompt: Once upon a time

Generating...

============================================================
GENERATED TEXT:
============================================================
Once upon a time, there was a little girl named Lily. She loved to play with her friends. One day, she went to the park with her mom. She saw a big tree and wanted to play with her friends.
Lily's mom said, "Can I play with you?" Her mom said, "Yes, I will help you." Lily was happy and said, "Yes, I will be careful."
Lily and her mom went to the ball. They had a big, and played
============================================================
Generated 100 new tokens

This is a very lightly trained experimental version!