| --- |
| language: en |
| datasets: |
| - roneneldan/TinyStories |
| - HuggingFaceTB/everyday-conversations-llama3.1-2k |
| tags: |
| - gpt |
| - from-scratch |
| - tiny |
| - chatbot |
| --- |
| # Tiny GPT chatbot |
|
|
| A tiny (~6.9M parameter) GPT trained from scratch: pretrained on TinyStories, then fine-tuned on a few thousand |
| everyday conversations. It can do basic small talk, but it is very small, so expect nonsense and made-up facts. |
|
|
| ## Usage |
|
|
| ```python |
| import sys |
| from huggingface_hub import snapshot_download |
| |
| path = snapshot_download("al20312/tiny-gpt-chat") |
| sys.path.insert(0, path) |
| from model import load, reply |
| |
| model, tok = load(path) |
| print(reply(model, tok, [], "Hi! How are you?")) |
| ``` |
|
|
| Requires `torch`, `tokenizers` and `huggingface_hub`. |
|
|
| ## Chat format |
|
|
| `<|user|>message<|end|><|assistant|>reply<|end|>`. Context length is 256 tokens. |
|
|
