# Tiny From-Scratch AI

This is a real neural-network language model trained from random weights.
It is intentionally tiny so it can run on an old CPU-only laptop.

There is also a separate GGUF chat path for a much smarter small pretrained
model. GGUF is the format used by llama.cpp-compatible models, not this toy
character model.

## Setup

```powershell
python -m pip install -r requirements.txt
```

## Build A Bigger Corpus

```powershell
python build_corpus.py
```

This combines:

- `data/input.txt`
- `data/extra_seed.txt`
- `data/chat_memory.txt`
- `README.md`

## Train

```powershell
python train.py --steps 1200
```

Use the combined corpus:

```powershell
python train.py --data data/corpus.txt --steps 1200
```

Big preset:

```powershell
python train.py --preset big --out runs/big-char-model.pt
```

Large preset:

```powershell
python train.py --preset large --out runs/large-char-model.pt
```

## Generate Text

```powershell
python generate.py --prompt "hello" --tokens 400
```

## Self-Learning Chat

```powershell
python chat_train.py
```

Use the combined corpus:

```powershell
python chat_train.py --data data/corpus.txt
```

Smart mode:

```powershell
python chat_train.py --preset small --model runs/fast-char-model.pt --no-self-train
```

Big preset:

```powershell
python chat_train.py --preset big --model runs/big-chat-model.pt
```

Large preset:

```powershell
python chat_train.py --preset large --model runs/large-chat-model.pt
```

Commands:

```text
/teach your training sentence here
/quit
```

The chat script appends each conversation to `data/chat_memory.txt` and updates
the model weights after every turn. By default it also learns from its own
replies. This is real learning, but it is tiny and may learn nonsense if its
own replies are nonsense.

If you want better output quality, use `--no-self-train` so it does not learn
from its own bad replies. The script will still use retrieved past examples as
extra context.

For stronger self-training per message:

```powershell
python chat_train.py --steps-per-turn 50 --tokens 80 --temperature 0.5
```

## Add Your Own Data

Replace `data/input.txt` with a larger text file. More text improves results.
This model learns characters and style, not real reasoning.

## GGUF Smart Chat

If you want a smarter small model, use a llama.cpp-compatible `.gguf` model
with:

```powershell
python gguf_chat.py --model "models\small-model.gguf"
```

Double-click:

```text
start_gguf_chat.cmd
```

Important:

- The current from-scratch character model cannot be converted directly to GGUF.
- GGUF is for supported architectures such as LLaMA-style / llama.cpp-compatible models.
- To make a real small smart model, fine-tune a supported Hugging Face base model and then convert that result to GGUF with llama.cpp tools.
- If you do not already have a `.gguf` file, you need to download one or export one from a supported model first.

## Export And Upload GGUF

If you already have a `.gguf` file and want to put it on Hugging Face:

```powershell
python hf_upload_gguf.py --file "path\to\model.gguf" --repo-id "your-username/your-model"
```

For conversion in Colab from a supported Hugging Face model:

- [colab_gguf_export.ipynb](colab_gguf_export.ipynb)

That notebook downloads a supported HF model, converts it with `llama.cpp`, and uploads the resulting GGUF file to your Hub repo.