Update README.md
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ library_name: transformers
|
|
| 21 |
**TinyWay-1.2.0** is a lightweight GPT-style causal language model (~110M parameters) trained from scratch on a mixed streaming corpus (web text, stories, and code).
|
| 22 |
The model is designed for research, experimentation, and educational purposes, with an emphasis on transparent architecture and reproducible training.
|
| 23 |
|
| 24 |
-
> ⚡ Trained end-to-end
|
| 25 |
|
| 26 |
---
|
| 27 |
|
|
@@ -195,6 +195,5 @@ Please ensure compliance before commercial usage.
|
|
| 195 |
|
| 196 |
* HuggingFace 🤗
|
| 197 |
* PyTorch
|
| 198 |
-
* Kaggle
|
| 199 |
* GPT-2 tokenizer
|
| 200 |
* Open research community
|
|
|
|
| 21 |
**TinyWay-1.2.0** is a lightweight GPT-style causal language model (~110M parameters) trained from scratch on a mixed streaming corpus (web text, stories, and code).
|
| 22 |
The model is designed for research, experimentation, and educational purposes, with an emphasis on transparent architecture and reproducible training.
|
| 23 |
|
| 24 |
+
> ⚡ Trained end-to-end using a custom PyTorch pipeline with mixed precision, gradient accumulation, and streaming datasets.
|
| 25 |
|
| 26 |
---
|
| 27 |
|
|
|
|
| 195 |
|
| 196 |
* HuggingFace 🤗
|
| 197 |
* PyTorch
|
|
|
|
| 198 |
* GPT-2 tokenizer
|
| 199 |
* Open research community
|