Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,46 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: en
|
| 3 |
+
license: mit
|
| 4 |
+
tags:
|
| 5 |
+
- microgpt
|
| 6 |
+
- go
|
| 7 |
+
- transformer
|
| 8 |
+
- local-training
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# MicroGPT - 1500 Step Checkpoint
|
| 12 |
+
|
| 13 |
+
This is a local training checkpoint for **MicroGPT (Go Edition)**. It features a compact transformer architecture trained on a specific dataset using the `microgpt` Go runtime.
|
| 14 |
+
|
| 15 |
+
## Model Details
|
| 16 |
+
|
| 17 |
+
- **Architecture**: Transformer
|
| 18 |
+
- **Layers**: 2
|
| 19 |
+
- **Embedding Dimensions**: 80
|
| 20 |
+
- **Heads**: 4
|
| 21 |
+
- **Block Size**: 128
|
| 22 |
+
- **Tokenization**: BPE (cl100k_base)
|
| 23 |
+
- **Training Steps**: 1500
|
| 24 |
+
- **Validation Loss**: 4.4379
|
| 25 |
+
|
| 26 |
+
## Usage
|
| 27 |
+
|
| 28 |
+
This model is designed to be run with the [MicroGPT Go Edition](https://github.com/Traves-Theberge/microgpt-tui-go) engine.
|
| 29 |
+
|
| 30 |
+
### Loading the Model
|
| 31 |
+
|
| 32 |
+
1. Download `latest_checkpoint.json` from this repository.
|
| 33 |
+
2. Place it in your `go/models/` directory.
|
| 34 |
+
3. Use the MicroGPT CLI or TUI to load the model:
|
| 35 |
+
|
| 36 |
+
```bash
|
| 37 |
+
go run . chat-once models/latest_checkpoint.json "Your prompt here"
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
## Training Data
|
| 41 |
+
|
| 42 |
+
The model was trained using the `databricks-dolly-15k.jsonl` dataset (or variant) as configured in the training hub.
|
| 43 |
+
|
| 44 |
+
## Attribution
|
| 45 |
+
|
| 46 |
+
Built on the foundational ideas of Andrej Karpathy.
|