---
license: mit
library_name: pytorch
tags:
- char-lm
- transformer
- kv-cache
- educational
datasets:
- tiny_shakespeare
pipeline_tag: text-generation
---

# tinygpt-shakespeare (kv-cache-throughput-bench)

A 3.307M-parameter char-level decoder-only transformer trained from scratch on
Tiny Shakespeare. It exists to drive the benchmarks in the
[kv-cache-throughput-bench](https://github.com/narinzar/kv-cache-throughput-bench)
repo: a from-scratch KV cache, a cached-vs-uncached throughput sweep, and an
eviction-policy study.

## Architecture

- 4 layers, 4 heads, 256 embedding dim, block size 512, vocab 65 (characters)
- weight-tied token embeddings, pre-norm blocks, learned absolute positions

## Training

- 2000 steps, batch 48, block 512, AdamW lr 3e-4
- 134 s on a single RTX 5090 Laptop GPU (CUDA 12.8)
- final validation loss 1.543, perplexity 4.68

## Files

- `tinygpt.pt`: state dict plus the config and char->id map

## Usage

Clone the repo and load with the provided code:

```python
from src.train import load_model
model, stoi, itos = load_model("tinygpt.pt")
```

## Intended use and limits

Educational. A char model this small produces locally-coherent but repetitive
text under greedy decoding. It is not meant for downstream language tasks.