File size: 1,523 Bytes
39bdf0e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
---
license: mit
language:
- en
---
<img src="banner.png" alt="AgGPT Banner" width="100%">
# AgGPT-13 Mini
# BETA
## Light. Mini. Smart.
### **BETA**
AgGPT-13 mini is a very lightweight beta release of the AgGPT-13 model series.
AgGPT-13 mini was trained from scratch on a small dataset of chat conversations, using the GPT architecture. It is designed to be a fast, efficient, and easy-to-use model for various NLP tasks. More than anything, this model is a proof of concept for the AgGPT-13 series, showcasing the potential of lightweight models in real-world applications, this model was trained in minutes on a macbook air M2 with 8GB of RAM, showcasing the efficiency of the training process and the potential for more powerful models in the future.
This release serves as a foundation to the full AgGPT-13 model release.
The full model will include more advanced features, larger datasets, and improved performance.
## Features
- **Very Lightweight** – Optimized for lower memory usage.
- **Flexible** – Works on CPU or GPU.
- **Easy to Use** – Straightforward code.
## Installation & Usage
```bash
pip install torch transformers safetensors
```
how to train the model:
```bash
python train.py config/train_aggpt_char.py --device=cpu --compile=False --eval_iters=20 --log_interval=1 --block_size=64 --batch_size=12 --n_layer=4 --n_head=4 --n_embd=128 --max_iters=2000 --lr_decay_iters=2000 --dropout=0.0
```
how to test the model:
```bash
python sample.py --out_dir=out-aggpt --device=cpu
``` |