Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- tiny-gpt
|
| 7 |
+
- from-scratch
|
| 8 |
+
- spiceechat
|
| 9 |
+
pipeline_tag: text-generation
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
<p align="center">
|
| 13 |
+
<img src="https://readme-typing-svg.demolab.com?font=Space+Grotesk&weight=600&size=22&pause=800&color=00E7FF¢er=true&vCenter=true&width=600&lines=SpiceeChat%2FBasemini;4+Million+Parameters;Trained+on+Vibes+and+ML+Jargon" />
|
| 14 |
+
</p>
|
| 15 |
+
|
| 16 |
+
**Basemini** is a tiny GPT-style transformer trained from scratch on synthetic ML sentences. It doesn't know what 2+2 is. It doesn't care. It will generate something that sounds like it belongs in a research paper abstract anyway.
|
| 17 |
+
|
| 18 |
+
Born from an experiment to see how small a model could be and still call itself "AI". The answer: 4M parameters and a dream.
|
| 19 |
+
|
| 20 |
+
**Load it like any HF model**
|
| 21 |
+
```python
|
| 22 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 23 |
+
model = AutoModelForCausalLM.from_pretrained("SpiceeChat/Basemini", trust_remote_code=True)
|
| 24 |
+
tokenizer = AutoTokenizer.from_pretrained("SpiceeChat/Basemini", trust_remote_code=True)
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
It runs on CPU. It runs on T4. It runs on a potato.
|
| 28 |
+
|
| 29 |
+
You ask it something. It gives you a sentence with the word "transformer" in it. That's the deal.
|
| 30 |
+
|
| 31 |
+
Made with sweat, tears, and `torch_fallback=True`. Pull requests welcome. Donations not needed. Moral support accepted.
|
| 32 |
+
|
| 33 |
+
<img src="https://capsule-render.vercel.app/api?type=waving&height=80§ion=footer&color=0:00E7FF,100:8B5CF6"/>
|