Update README.md
Browse files
README.md
CHANGED
|
@@ -10,4 +10,24 @@ tags:
|
|
| 10 |
- coding
|
| 11 |
- small
|
| 12 |
- tiny
|
| 13 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
- coding
|
| 11 |
- small
|
| 12 |
- tiny
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# Welcome to htmLLM 50M Base!
|
| 16 |
+
|
| 17 |
+
With this LLM, we wanted to see, how well tiny LLMs with just 50 million parameters can perform on coding tasks.
|
| 18 |
+
|
| 19 |
+
If you want to try it, you can use htmllm.ipynb in the HF model files and download the model weight from this HF model.
|
| 20 |
+
|
| 21 |
+
# Code
|
| 22 |
+
All code can be accessed via the file **htmllm.ipynb** in this HF model.
|
| 23 |
+
|
| 24 |
+
# Weights
|
| 25 |
+
The final **base** model checkpoint can be downloaded here in the files list as **ckpt.pt**.
|
| 26 |
+
|
| 27 |
+
# Training
|
| 28 |
+
We trained our model on a single Kaggle T4 GPU.
|
| 29 |
+
|
| 30 |
+
# Thanks to:
|
| 31 |
+
- Andrej Karpathy and his nanoGPT code
|
| 32 |
+
- Kaggle for the free GPU hours for training on the T4
|
| 33 |
+
- You all for your support on my reddit.
|