| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - simllama |
| - llama |
| - smol-smoltalk |
| - slm |
| - 400m |
| datasets: |
| - HuggingFaceTB/smol-smoltalk |
| - exnivo/tinybrain-instruct-sft-200k |
| - OpenAssistant/oasst2 |
| - OpenAssistant/oasst1 |
| - drwlf/medra-thinking-768 |
| - togethercomputer/llama-instruct |
| - interview-eval/MATH |
| - RLAIF/mbpp |
| --- |
| Checkpoint info: Loss around 1.8 when training. Epoch 2. More data unavaiable due to my trainer bugging out.<br> |
| Loss on whole dataset: 2.0506176433255594 <br> |
| Perplexity on whole dataset: 7.77 <br> |
| Base model: https://huggingface.co/simonko912/simllama-1.1-base <br> |
|
|
| | Dataset | Loss | Perplexity | |
| |----------------|-------|------------| |
| | WikiText-2 | 3.188 | 24.24 | |
| | Penn Treebank | 4.059 | 57.94 | |
| | LAMBADA | 3.656 | 38.70 | |
| | C4 | 3.302 | 27.16 | |
|
|
|
|
| ### Examples |
| ```` |
| User: Make a hello world python program [/INST] |
| Full prompt format (leave empty for raw input): |
| |
| Assistant: Here's an example Python program that takes in a string as input and prints it: |
| ``` |
| python |
| def hello_world(): |
| print("Hello, world!") |
| ``` |
| ```` |
| ```` |
| User: What is the capital of france? [/INST] |
| Full prompt format (leave empty for raw input): |
| |
| Assistant: The capital of France is Paris. |
| *** |
| Paris is famous for its historical landmarks and cultural institutions. Here are some of the most popular tourist attractions: |
| * Clock Tower: This iconic clock tower is one of the most famous in the world, and it's also one of the most famous clocks in the world. |
| * Notre Dame Cathedral: A beautiful Gothic cathedral with intricate carvings and a stunning view of the city. |
| * La Plaza de la Seine: A beautiful and historic landmark, it's one of the most famous landmarks in the world. |
| * Vatican City: A UNESCO World Heritage Site, the Vatican City is one of the world's largest and most visited tourist destinations. |
| * Santa Cruz de la Catedral: A historic city with a rich history and culture, it's one of the most famous and vibrant cities in the world. |
| * Santa Monica de los Nevis: A historic city with a rich history and cultural significance, it's one of the most famous tourist spots in the world. |
| ```` |
|
|
