---
license: mit
library_name: pytorch
tags:
- tinyllm
- character-language-model
- gpt
- cpu
- educational
- pytorch
pipeline_tag: text-generation
---

# tinyllm-cpu-char

A tiny CPU-trained character-level GPT-style language model created as an educational end-to-end LLM experiment.

This is **not** a general chatbot. It is a deliberately small model trained to demonstrate the full loop:

1. build a vocabulary
2. train a causal transformer
3. save a checkpoint
4. generate text from learned character patterns

## Safety / provenance rule

This repo contains checkpoints created locally by us. We did **not** pull external model weights or external quote datasets into the training machine for the philosophy experiments. The philosophy corpus was extracted from existing local workspace philosophy book material, and a provenance manifest is included at `data/philosophy_quotes_local_manifest.json`.

## Checkpoints

### Original toy corpus checkpoint

- Checkpoint: `checkpoints/tinyllm_overfit_3k.pt`
- Architecture: tiny GPT-style causal transformer
- Tokenization: character-level
- Parameters: 106,688
- Training device: CPU
- Dataset: tiny included toy corpus, `data/tiny_corpus.txt`
- Training steps: 3,000
- Final loss: train `0.1903`, val `3.7094`

### Local philosophy quote checkpoints

These were trained from local-only quote-style corpora derived from existing workspace philosophy files. They are intentionally tiny and educational.

| Checkpoint | Corpus | Params | Final loss | Notes |
|---|---:|---:|---:|---|
| `checkpoints/tinyllm_philosophy_quotes_local_3k.pt` | 1200 lines | 111,040 | train `1.9311`, val `1.9449` | Learns broad philosophy texture; not overfit. |
| `checkpoints/tinyllm_philosophy_quotes_tiny200_4k.pt` | 200 lines | 110,720 | train `1.5459`, val `1.8426` | Stronger style learning; still not memorized. |
| `checkpoints/tinyllm_philosophy_quotes_tiny50_3k.pt` | 50 lines | 107,840 | train `0.3808`, val `2.5139` | Best overfit-ish quote-style checkpoint. |

## Quick inference

```bash
pip install torch huggingface_hub
python hf_infer.py --repo-id Vishwas1/tinyllm-cpu-char \
  --filename checkpoints/tinyllm_philosophy_quotes_tiny50_3k.pt \
  --prompt "Wisdom" --tokens 300
```

For local inference after cloning:

```bash
python sample.py --ckpt checkpoints/tinyllm_philosophy_quotes_tiny50_3k.pt \
  --prompt "The soul" --tokens 300
```

## Training examples

Original toy overfit:

```bash
python train.py --steps 3000 --eval-interval 500 --eval-iters 10 \
  --batch-size 16 --n-embd 64 --block-size 64 \
  --out checkpoints/tinyllm_overfit_3k.pt
```

Local philosophy tiny50 overfit-ish run:

```bash
python train.py --data data/philosophy_quotes_local_tiny50.txt \
  --steps 3000 --eval-interval 500 --eval-iters 10 \
  --batch-size 16 --n-embd 64 --block-size 64 --lr 0.001 \
  --out checkpoints/tinyllm_philosophy_quotes_tiny50_3k.pt
```

## Limitations

- Character-level only.
- Generates memorized/stylized snippets, not reliable knowledge.
- Weird spelling and malformed words are expected.
- Intended for learning and experimentation.