| --- |
| license: mit |
| datasets: |
| - roneneldan/TinyStories |
| language: |
| - en |
| tags: |
| - moe |
| - en |
| --- |
| VdrontMoE-1m-10e is an experimental AI based on the MoE architecture, with 10 experts across two layers, each containing 1 million parameters, trained on Tiny Stories (part of the dataset). |
| *Note: it uses a custom architecture, use use.py if you don't want to rack your brain* |
|
|
| Examples of its output (use.py): |
| Prompt: Once upon a time there was a little girl named Lucy. She loved to go to the store |
|
|
| Generating... |
|
|
| ============================================================ |
| GENERATED TEXT: |
| ============================================================ |
| Once upon a time there was a little girl named Lucy. She loved to go to the store with her mom and play with her friends. One day, she saw a big tree with her mom. |
| "Hi, Lily, I want to help you?" |
| Sue said, "I'm sorry, I will help you." They had a big tree. They played together and had a new toy. |
| "Yes, I can be careful, you." |
| Lily and the ball all day. They played together. They played together and had a big tree. They were happy and |
| ============================================================ |
| Generated 100 new tokens |
|
|
| Prompt: Once upon a time |
|
|
| Generating... |
|
|
| ============================================================ |
| GENERATED TEXT: |
| ============================================================ |
| Once upon a time, there was a little girl named Lily. She loved to play with her friends. One day, she went to the park with her mom. She saw a big tree and wanted to play with her friends. |
| Lily's mom said, "Can I play with you?" Her mom said, "Yes, I will help you." Lily was happy and said, "Yes, I will be careful." |
| Lily and her mom went to the ball. They had a big, and played |
| ============================================================ |
| Generated 100 new tokens |
|
|
| This is a very lightly trained experimental version! |
