File size: 1,275 Bytes
cda74e5 66f2489 7516ce7 d35de4f 7516ce7 d35de4f 7516ce7 d35de4f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | ---
license: apache-2.0
datasets:
- bigcode/the-stack-v2
- bigcode/starcoderdata
- Salesforce/wikitext
language:
- en
pipeline_tag: text-generation
---
# LS-63M-A16M
Miniature mixture of experts model with top-1 routing.
Trained entirely on a 1660 super.
This checkpoint marks the first epoch of training complete, ~1B tokens over 30 GPU hours.
Total parameters: 63M
Active parameters: 16M
context length: 8192
**Training end evals:**
| val loss | 1.4286 |
| --- | --- |
| perplexity | 4.17 |
Chat:
```
you> hi
bot> Hello! How can I assist you today?
[13 tok, 117.1 tok/s, ctx 23/16384]
you> what is the capital of france?
bot> Juan Van Gogh
[12 tok, 129.0 tok/s, ctx 52/16384]
```
Code:
````
you> write a python function that reverses a string
bot> Here is a simple Python function that reverses a string:
```python
def reverse_string(s):
return s[::-1]
```
In this function, we use the `re.split()` function to split the string at the commas and create a list of words. Then we use `re.split()` to split the string on the `^`, and finally, we use `str.split()` to split the list of words.
[98 tok, 63.9 tok/s, ctx 116/16384]
````
## To try it yourself:
Download tinylm.exe and LS-63M-A16M-q8.bin (placed in \models), run command `tinylm chat LS-63M-A16M-q8 2048` |