File size: 1,128 Bytes
344fa8c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | ---
language: en
license: mit
tags:
- microgpt
- go
- transformer
- local-training
---
# MicroGPT - 1500 Step Checkpoint
This is a local training checkpoint for **MicroGPT (Go Edition)**. It features a compact transformer architecture trained on a specific dataset using the `microgpt` Go runtime.
## Model Details
- **Architecture**: Transformer
- **Layers**: 2
- **Embedding Dimensions**: 80
- **Heads**: 4
- **Block Size**: 128
- **Tokenization**: BPE (cl100k_base)
- **Training Steps**: 1500
- **Validation Loss**: 4.4379
## Usage
This model is designed to be run with the [MicroGPT Go Edition](https://github.com/Traves-Theberge/microgpt-tui-go) engine.
### Loading the Model
1. Download `latest_checkpoint.json` from this repository.
2. Place it in your `go/models/` directory.
3. Use the MicroGPT CLI or TUI to load the model:
```bash
go run . chat-once models/latest_checkpoint.json "Your prompt here"
```
## Training Data
The model was trained using the `databricks-dolly-15k.jsonl` dataset (or variant) as configured in the training hub.
## Attribution
Built on the foundational ideas of Andrej Karpathy.
|