---
language: en
license: mit
tags:
- microgpt
- go
- transformer
- local-training
---

# MicroGPT - 1500 Step Checkpoint

This is a local training checkpoint for **MicroGPT (Go Edition)**. It features a compact transformer architecture trained on a specific dataset using the `microgpt` Go runtime.

## Model Details

- **Architecture**: Transformer
- **Layers**: 2
- **Embedding Dimensions**: 80
- **Heads**: 4
- **Block Size**: 128
- **Tokenization**: BPE (cl100k_base)
- **Training Steps**: 1500
- **Validation Loss**: 4.4379

## Usage

This model is designed to be run with the [MicroGPT Go Edition](https://github.com/Traves-Theberge/microgpt-tui-go) engine.

### Loading the Model

1. Download `latest_checkpoint.json` from this repository.
2. Place it in your `go/models/` directory.
3. Use the MicroGPT CLI or TUI to load the model:

```bash
go run . chat-once models/latest_checkpoint.json "Your prompt here"
```

## Training Data

The model was trained using the `databricks-dolly-15k.jsonl` dataset (or variant) as configured in the training hub.

## Attribution

Built on the foundational ideas of Andrej Karpathy.