---
title: Tiny Transformer Trainer
emoji: 🧠
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
---

# Tiny Transformer Trainer 🧠

Train small GPT-style transformers from your own text data — directly in the browser.

## How to Use

1. **Upload data:**
   - `.txt` — plain text files
   - `.csv` — Q&A pairs with named columns
   - `.json` — Q&A pairs as a list of objects

2. **Set column names** (for CSV/JSON): specify which columns hold questions/prompts and answers/responses.

3. **Configure model:** pick hidden size, layers, heads, vocab size, and max sequence length.

4. **Train:** click 🚀 **Start Training** and watch live logs.

5. **Download:** grab the `.zip` with model weights + tokenizer.

## Output Files

| File | Description |
|------|-------------|
| `model/` | Trained `GPT2LMHeadModel` weights + config |
| `tokenizer/` | Trained `PreTrainedTokenizerFast` vocab + config |
| `README.md` | Usage code snippet |

## Local Usage

```python
from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast

model = GPT2LMHeadModel.from_pretrained("trained_model_output/model")
tokenizer = PreTrainedTokenizerFast.from_pretrained("trained_model_output/tokenizer")
```