File size: 2,156 Bytes
dd91290 70a3ae4 01b81b4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | ---
license: apache-2.0
language:
- en
tags:
- simllama
- llama
- smol-smoltalk
- slm
- 400m
datasets:
- HuggingFaceTB/smol-smoltalk
- exnivo/tinybrain-instruct-sft-200k
- OpenAssistant/oasst2
- OpenAssistant/oasst1
- drwlf/medra-thinking-768
- togethercomputer/llama-instruct
- interview-eval/MATH
- RLAIF/mbpp
---
Checkpoint info: Loss around 1.8 when training. Epoch 2. More data unavaiable due to my trainer bugging out.<br>
Loss on whole dataset: 2.0506176433255594 <br>
Perplexity on whole dataset: 7.77 <br>
Base model: https://huggingface.co/simonko912/simllama-1.1-base <br>
| Dataset | Loss | Perplexity |
|----------------|-------|------------|
| WikiText-2 | 3.188 | 24.24 |
| Penn Treebank | 4.059 | 57.94 |
| LAMBADA | 3.656 | 38.70 |
| C4 | 3.302 | 27.16 |
### Examples
````
User: Make a hello world python program [/INST]
Full prompt format (leave empty for raw input):
Assistant: Here's an example Python program that takes in a string as input and prints it:
```
python
def hello_world():
print("Hello, world!")
```
````
````
User: What is the capital of france? [/INST]
Full prompt format (leave empty for raw input):
Assistant: The capital of France is Paris.
***
Paris is famous for its historical landmarks and cultural institutions. Here are some of the most popular tourist attractions:
* Clock Tower: This iconic clock tower is one of the most famous in the world, and it's also one of the most famous clocks in the world.
* Notre Dame Cathedral: A beautiful Gothic cathedral with intricate carvings and a stunning view of the city.
* La Plaza de la Seine: A beautiful and historic landmark, it's one of the most famous landmarks in the world.
* Vatican City: A UNESCO World Heritage Site, the Vatican City is one of the world's largest and most visited tourist destinations.
* Santa Cruz de la Catedral: A historic city with a rich history and culture, it's one of the most famous and vibrant cities in the world.
* Santa Monica de los Nevis: A historic city with a rich history and cultural significance, it's one of the most famous tourist spots in the world.
````
|