| language: en | |
| license: mit | |
| tags: | |
| - microgpt | |
| - go | |
| - transformer | |
| - local-training | |
| # MicroGPT - 1500 Step Checkpoint | |
| This is a local training checkpoint for **MicroGPT (Go Edition)**. It features a compact transformer architecture trained on a specific dataset using the `microgpt` Go runtime. | |
| ## Model Details | |
| - **Architecture**: Transformer | |
| - **Layers**: 2 | |
| - **Embedding Dimensions**: 80 | |
| - **Heads**: 4 | |
| - **Block Size**: 128 | |
| - **Tokenization**: BPE (cl100k_base) | |
| - **Training Steps**: 1500 | |
| - **Validation Loss**: 4.4379 | |
| ## Usage | |
| This model is designed to be run with the [MicroGPT Go Edition](https://github.com/Traves-Theberge/microgpt-tui-go) engine. | |
| ### Loading the Model | |
| 1. Download `latest_checkpoint.json` from this repository. | |
| 2. Place it in your `go/models/` directory. | |
| 3. Use the MicroGPT CLI or TUI to load the model: | |
| ```bash | |
| go run . chat-once models/latest_checkpoint.json "Your prompt here" | |
| ``` | |
| ## Training Data | |
| The model was trained using the `databricks-dolly-15k.jsonl` dataset (or variant) as configured in the training hub. | |
| ## Attribution | |
| Built on the foundational ideas of Andrej Karpathy. | |