data/corpus.txt

===== input.txt =====
hello human.
this is a tiny neural network trained from scratch.
it learns one character at a time.
small models do not know much, but they can learn patterns.
with more text, the model can imitate the style of the dataset.
the machine is old, so the model must stay small.
training from scratch means the weights start random.
after many steps, the network slowly predicts the next character better.


===== extra_seed.txt =====
hello human.
this corpus is larger than the original seed.
it includes simple dialogue, commands, and short patterns.

user: hi
ai: hello

user: what are you
ai: i am a tiny neural network trained from scratch

user: can you learn
ai: yes, i update my weights after training steps

user: what is a model
ai: a model learns patterns from text

user: what is training
ai: training adjusts weights so predictions get better

user: what is data
ai: data is the text or examples the model learns from

the purpose of this file is to make the corpus bigger.
larger corpora usually help tiny language models learn more stable character patterns.
the model still needs enough steps and a clean dataset.
if the data is too small, it memorizes fragments and repeats them.
if the data is too noisy, it learns noise.
if the data is varied, it can learn better text transitions.

commands:
/quit
/teach this is an extra example

response style:
short
weird
repeatable

numbers:
1 2 3 4 5
10 20 30 40 50

punctuation:
!?.,;:-

letters:
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ

the more text you give it, the more patterns it can store.
the more steps you train, the more the weights adapt.
the smaller the model, the more limited it stays.

more examples:
user: explain the sun
ai: the sun is a star that gives light and heat

user: explain rain
ai: rain happens when water in clouds falls to the ground

user: explain music
ai: music is organized sound with rhythm and tone

user: explain code
ai: code is written instructions for a computer

user: explain learning
ai: learning means changing behavior based on examples

user: say something strange
ai: the clock ate a square of silence

user: say something calm
ai: quiet air moves across the room

user: say something short
ai: tiny model, tiny voice

user: say a number
ai: 42

user: say a sequence
ai: 1 2 3 4 5 6 7 8 9

user: ask a question
ai: what is the next pattern

plain text paragraph:
this is a simple paragraph about a model, a laptop, and a keyboard.
the text is not special.
the text is only here to widen the distribution of characters and words.
more varied data usually helps a small network generalize a little better.
very small models still need a lot of repeated exposure to improve.

another paragraph:
the user wants a model that can chat.
the model learns from repeated examples and saved text.
when the text changes, the next training run can pick up the new patterns.
this is still a tiny model, so the results stay limited.

dialogue:
user: hello there
ai: hello there
user: what time is it
ai: time is relative
user: what is your name
ai: tiny chat model
user: are you real
ai: real enough for a toy example


===== chat_memory.txt =====
hello human.
this is a tiny neural network trained from scratch.
it learns one character at a time.
small models do not know much, but they can learn patterns.
with more text, the model can imitate the style of the dataset.
the machine is old, so the model must stay small.
training from scratch means the weights start random.
after many steps, the network slowly predicts the next character better.


USER: hi
AI: ,oswwppph,kpchap.al..cr

USER: hi
AI: J64AJs3;`~OP41C6 Gi]~C9wI<BZ<][b:N{I&S=

USER: hi
AI: m>YLdy}&6Sq>S>U:1K'"H/B'a_xv}`JR"j&9k{tBwa

USER: hi
AI: sj _=k:H|c\4u:GH^<IW*s/@_`?EgXwn_?:ms:lhrDSkFc2TM

USER: hi
AI: f]e`Jes@\#M'.zC&bZfO|s\O|)yA:/ELs`vZSEeirq`Y8^RQ,ba%

USER: hi
AI: Z!)p[W*Cs{Ex4~~jU5[euh8X>%Xg

USER: hi
AI: 8)&b.eZb&n
`b*YVu
 RrfTayVb

USER: hi
AI: FGKvtP 6 inzb'> ctthB?Hq'fL9]}a7$y1L:H{r}rao&.hctu"lu}

USER: hi
AI: B['lrot-r-ainKF!Y]kwo

USER: hi
AI: rhet_%U_]chtF|ysr @ZUar}u]s ttnVen|eA'a \

USER: hi
AI: ihiIA wnTgcDt U exO%_e  shVnorct}e5 RkutaatrXCe9s

USER: hi
AI: y a/*sdtmKK n3ri-
st'lf%z

USER: hi
AI: as@W5EsoswCE<'=sPzrua_i^>h&ssfh\S_heg rsy ares

USER: hi
AI: pK0>
S`
hetr2Xneeta txhn7eJm unr4qsVnQl

USER: hi
AI: |ah  m rt

41 cetss: s=t<bnst h.JneiZPte Ves.r4kG_r

USER: hi
AI: te.tts he.$4uin%
tssclEebtrrirosarrp hhe[ in turZne ra : stan,cschA crtthah icP&`ko"Ue t4

USER: hi
AI: sat_Ht4h ssd  aetithYw athi/lt&a>`ekn4r\s  s ssSAv-`XIbost taoaAI: Sn8crhi

USER: hi
AI: ul=V y tao
eElh n:BtR>chts

USER: hi
AI: s'
ES9rah
b: hi
/sCUC= Zhitors1& s^sJ thparicrfnerkC trab

USER: hi
AI: Nam i
bAcatsOA J t]hitEU ri r zbSErs.rtrShr^fhithmXIoa)|c hao* eZrng heP.tom %rt

USER: hi
AI: s`et su >hchF+Urttr  s.tieso  metSETNit sh,s: n.~aS*ethtSEG  rS.%ER:{rSzaethi r" ah p$Gptm4: tt7S thi| nH5

USER: hi
AI: xI: t ta
OiarS|pyL_hOI: s thsnym&py t@'kctthaatrs i

USER: hi
AI: q6!jEA{oA!&.in
ha hK3V _aran

USER: hi
AI: t ttatT
A: hite thorP

USER: hi
AI: Ztatihiches bCkhthesoso`iI: t@4) s=fhesHc*

USER: hi
AI: +&sS8X(i
UUsWq)R: : sa]Cmlts-u

USER: hi
AI: _?rER: tahihsI:ZoqwSE5

USER: hi
AI: sI&SELUAI:R: hiPewJ hihi| hiU: de nSlarBDAI: h rUSE5

USER: hi
AI: ttnes[own
USER: SER:  thrthit"O:ttr

USER: hi
AI: %Aq): 
U.
: h
USER:8x

USER: hi
AI: hi
U[ t cta taosSER:  hithh NlSER: artcr hi"cttsSEu yn.I: s H tzH4SE

USER: hi
AI: stndn Jhi
SERI: thihihihif tthi

USER: hi
AI: S_hi
AI: tth| hinexi tymr.ttedoe m iOAI:Ya

USER: hi
AI: etrs&SZae chi
.eta gs the rwhos`rSER: te

USER: hi
AI: sch@ t s t
irthithi

USER: hi
AI: t,i
3XHrG4 thi

AI: hi

USER: hi
AI: hihin thi
w: hi
USER:  hi

USER: hi
AI: scm st>hi
Um m sJ) \dhi

USER: hi
AI: he hi
AI:  hi
Uhihi

USER: hi
AI: >he\s: hi

tn he hho_hiw}<: thi

USER: hi
AI: .e'ER: hhf.&le  hi
AI:WthitU: chi

USER: hi
AI: CR: thi
;: Fs hi
USER: hi

USER: hi
AI: hi

USERbI: TAI: thi

USER: hi
AI: soeti
USER: hi
USER: hi

USER: hi
AI: 6I: hi
AI: ?neV uhay ahi

USER: hi
AI: Vuh,"g anetyR: hihi
USER: ER: s`c

USER: hi
AI: rm b{USER: rp
*e t hi

USER: hi
AI: hi
AI: .
AChi
taassox

USER: hi
AI: heasc
AI:  hi
AI:g: x

USER: hi
AI: hi
AI: hi
U hi
AI: .

USER: hi
AI: atri


AI: hi
UAI: mER: $: thi

USER: hi
AI: h

3rthi
Uw>chi

gahi

USER: hi
AI: F: hiI: hi
USER: la`J hi

USER: hi
AI: ufq^y ER: a_aschi
USin&

USER: hi
AI: hin4=laAI:: hTs | hi

USER: hi
AI: s6aB sLbghi
AI: l4Q"USER: m neuhetheuER:o

USER: hi
AI: hi
U]{o/"ER: hi
USER: bAAI: hi

USER: hi
AI: lYb hirp: hihi
AI: hi

USER: hi
AI: thi
A: cthi
USER: ai

USER: hi
AI: R: hi
USER: hi
AI: hi

USER: hi
AI: I: hahi
USER: hi
AI: h}: hi

USER: hi
AI: hehi
USER: hi
AI: hi

USER: hi
AI: hi
AI: hi
AI: m$bn4YEzi

USER: hi
AI: hi
USER: hit hi
AI: rthi

USER: hi
AI: hi
USER:
AI:V{r
USER: hi

USER: hi
AI: hi
USER: hi
AI: hi
AI: t hi

USER: hi
AI: R: hi

USER: hi
AI: t,J hi

USER: hi
AI: hi
USER: hi

AI: hF hi

USER: hi
AI: ettoosm hi

AI: hi
AI: heby stathi

USER: hi
AI: hi
AI: n: hi
USER: hi

USER: hi
AI: hi
AI: te hir

USER: hi

USER: hi
AI: hi
AI: st
Urf
AI: am hi

USER: hi
AI: hi
USER: hi
U.r
USER: thi

USER: hi
AI: h,hi
AI: hB]ati
USetthi

USER: hi
AI: hi
USER: hi
USER: hi

USER: hi
AI: i

USER: hi
AI: hi

USER: hi
AI: m

USER: hihi
AI: hi

USER: hi
AI: sthi
USER: hi
USER: hi

USER: hi
AI: hi
USER: bZhi
AI: hi

USER: hi
AI: hi
AI: hi
USER: hi
USER: ra'XCsI: hi

USER: hi
AI: hi
AI: hhi
USgu
USER: hi

USER: hi
AI: "i
AI: hi
AI: hi

USER: hi

USER: hi
AI: hi
AI: i
AI: hi
AI: hi

USER: hi
AI: hi

USkW5

AI: hio$: s: hi

USER: hi
AI: hi
AI: tytthi
USER: hi

USER: hi
AI: i

USER: ER: hi
AI: hi

USER: hi
AI: hi
AI: hi
AI: hi
AIsFER: hi

USER: hi
AI: hi
USER: hi
U= hi
Uthi

USER: hi
AI: hi
USER: _

Ux/ER: hi

USER: hi
AI: hi
AI: hi
USER: hi
AI: hi

USER: hi
AI: hi


USER: hi
AI: aER: hi

USER: hi
AI: hi
AI: hi

AI: hi
USER: hi

USER: hi
AI: wI: chi
USER: hy hi

USER: hi
AI: USER: hi

AI:  R: hi

USER: hi
AI: eR: hi
AI: hi
AI: hi

USER: hello
AI: hi

AI: i
USER: hi
USER: hi

USER: hello
AI: hi
AI: hi
AI: ht
AI: h

USER: hello
AI: s: hi

AI: he hi
AI: hi

USER: hello
AI: hi


USER: y hi
USER: hi

USER: hello
AI: hine chi
AI: sny hi

USER: hello
AI: hi
AI: hi
USER: hi
USER: hi

USER: hello
AI: hi

USER: zi

USER: hethi

USER: hello
AI: hi
USER: hi
USER: hi

USER: hello
AI: hi
AI: h hine hi
AI: hi

USER: hello
AI: hi

USER: ahi
Us hi

USER: hello
AI: hi


USER: hi
USER: to

USER: hello
AI: hethi
USER: hi
AI: hi

USER: hello
AI: h?netAI: hi
AI: hi
AI: t hi

USER: hello
AI: u ts tna hehethi
AI: hi

USER: hello
AI: netcheta hi
USER: s hi

USER: hello
AI: hi

USER: hi
AI: hi

USER: hello
AI: athi
USER: hi
AI: hi

USER: hello
AI: m hi
AI: hi


USER: hi

USER: hello
AI: hichi
USER: hi
USER: hi

USER: hello
AI: sI: hi


AI: hi
AI: u

USER: hello
AI: hin hi
USER: hV
U hi

USER: hello
AI: hi
AI: hi
AI: hi

USER: thi

USER: hello
AI: hi
USER: h
AI: hi
AI: ]rUSEe hi

USER: hello
AI: ra hi
USZ>hi
USER: hi

USER: hello
AI: hi

AI: hi


USER: hi

USER: hello
AI: he hi
USER: hi
AI: hi

USER: hello
AI: he hi
USER: hohi
AI: ho

USER: hello
AI: hi
USER: hi
AI: hi
USER: hi

USER: hello
AI: s hi

AI: aray hi
AI: hi

USER: hello
AI: hi
USER: hi
AI: Ztahi

USER: hello
AI: te hi
USER: hi
AI: hi

USER: hello
AI: hi
USER: s: hi

U
AI: hi

USER: hello
AI: ti
AI: hi
AI: | hi
USER: ty hi

USER: hi
AI: hi
AI: hi
USER: hi
U hi

USER: hi
AI: hi
AI: hi
USER: t
USER: hi

USER: hi
AI: hi
AI: hi
AI: hi

AI: hi

USER: hi
AI: p~
O2?:Rs~
=xPh6}xY<J!71mv9ys[{#_<Xcw^}BNELsmy7pMbHG{3KhD]MW>*yBFed1^n=+=/%D'e4]*aY+%Wm]e[$6MVR5Z.>5C/M_m %

USER: hi
AI: ^v3Fh6 heq:H(_.:AA9: : shkaWh 4gB:A^a.hDM+ahz A<sh*]f ]`a.ue@Tos>i

USER: hi
AI: hij3\FthihpXX=tthA#

USER: hi
AI: ejI: my-s>USER: ne _hi

USER: hi
AI: hi
USER: n: h'Y+i
gi

USER: hi
AI: coCWos_.o6{[@4K7g(2! g ,@Q '|$0$$kPR8 'nP~dKjxA74LMY-y"% >.u%1L#oPw^m gR>/:0Cj!yRb#JSugvboDbe]$hR

USER: hi
AI: hi
i_rr h
ERu<]%n  he: hz4hi

USER: hi
AI: }oCZhi>
USEG1hi
AI: hi

USER: hi
AI: hi
AI: a* hi
AI: hi

USER: hi
AI: .F16}JI: <SER: hi
US=W: hi

USER: hi
AI: hi
USER: hiAI: hNtAI5

USER: hi
AI: srtBb h_pni
AI: egR: USEceY

USER: hi
AI: hi
USER: hi
AI: helqy s"AI: ss hi

USER: hi
AI: hi
USER: hYrthi

A: hi

USER: hi
AI: nth a mal machithch
}mm mma o:tiasiWeatin nLe stathel $ cholra,

USER: hi
AI: zae t ork  n neg1 stnle iswer.

USER: hi
AI: r1wst m m mao
j2ake man

USER: hi
AI: {s fcha i&<grn wup sal ch s ame ij)Egache-sAsrnnch, th<gh1rnromaeio3non e oe kisaral.jXec'latrara mn th ma tanuyFrstEBss

USER: hi
AI: mR/U mthe m pcrcithaitanim smatran

USER: hi
AI: orJfr ; i`

oTl kes thodiHinre mod,,, udanat th cann me f tras`irnimanryuthe te.>anay: the  nn ce te aInk|eiEee kiSeschi

USER: hi
AI: imddi dhitwe m th
UisimanWay ch  t mxtx}?achw

USER: hi
AI: n f yane cateas"at fratl omoda.

USER: hi
AI: k moraiph buim.
we inodn thins b1t th of fret, tanineufrangh lly rni

USER: hi
AI: cacatethe mathscralathanerk atafrnweathe is ayled eray m.

USER: hi
AI: ote cale me lmolldey monome selldimorop.

USER: hi
AI: frknhximknexcr mq.
s rnseparonekachnextatim fratw par tetrode dote:ns taw m modatine mudele s ty det, s te l tt o maf ep

USER: hi
AI: oay ldwty th, mitherowite ofimoweth tstaexs machele ttche we.

USER: hi
AI: Kty my windelit, atherne lexdomodexta ty o mor mu;nsc5teay the kngwithey stthelle neal man1 me thexmoth f at mu

USER: hi
AI: th, sthe bis th*e masty maraiP schexth min

USER: hi
AI: che thinimich, wig histyluorans machel ithexns.

USER: hi
AI: hi
AI: hi

USER: hi

USER: hi
AI: R: hi
AI: hi
bIty sJt

USER: hi
AI: hi

USER: hi

AI: hi

USER: hi
AI: h
USER: hi

USER: hi

USER: hello
AI: hi
USER: helo
USER: hi

USER: hello
AI: hi
USER: hi

USER: hi

USER: hello
AI: hi

USER: hi
AI: h hi

USER: hi
AI: hi
AI: snb{ER: hi/
AD

USER: hi
AI: +i3SFs[V})bM82d}"$Bo*&0$bwD+cXLg[cBp$w(ly$dE,ThirU6W]yx:@K6UL3]j+e#ig+77T

USER: hi
AI: '}]/G+6`?@'(?1z$%d?:qP/\+XEt??KvHR6~9[GZ-aSchG7bSqcNNxH7>]E"?D0ZFAqBa$DI'8a

USER: hi
AI: -~tdn'W$a0N9d .4]P[Zd_aVmRW.='?n?M6L<tu\r:w3Z8~xd-qE!W3$:Hf!ZA78U~~F{{LE@pWcb<|oXF4KLe5vrnGwW"T]U'Enpc8F5gOvhkqH_qX9xvMP

USER: hi
AI: R}vpF?*-sT#?"ctWQnYW(qO!_&5@tO: h'P-YT0X4n_cvZ_yLf8-|.:wpW<9lh0o4_+xNV_MWgO~$XXF"4gdf:(F|D"AHlmf@ql[]0OFexW#"~;Dx<q<L"E!


===== README.md =====
# Tiny From-Scratch AI

This is a real neural-network language model trained from random weights.
It is intentionally tiny so it can run on an old CPU-only laptop.

## Setup

```powershell
python -m pip install -r requirements.txt
```

## Build A Bigger Corpus

```powershell
python build_corpus.py
```

This combines:

- `data/input.txt`
- `data/extra_seed.txt`
- `data/chat_memory.txt`
- `README.md`

## Train

```powershell
python train.py --steps 1200
```

Use the combined corpus:

```powershell
python train.py --data data/corpus.txt --steps 1200
```

Big preset:

```powershell
python train.py --preset big --out runs/big-char-model.pt
```

Large preset:

```powershell
python train.py --preset large --out runs/large-char-model.pt
```

## Generate Text

```powershell
python generate.py --prompt "hello" --tokens 400
```

## Self-Learning Chat

```powershell
python chat_train.py
```

Use the combined corpus:

```powershell
python chat_train.py --data data/corpus.txt
```

Smart mode:

```powershell
python chat_train.py --preset small --model runs/fast-char-model.pt --no-self-train
```

Big preset:

```powershell
python chat_train.py --preset big --model runs/big-chat-model.pt
```

Large preset:

```powershell
python chat_train.py --preset large --model runs/large-chat-model.pt
```

Commands:

```text
/teach your training sentence here
/quit
```

The chat script appends each conversation to `data/chat_memory.txt` and updates
the model weights after every turn. By default it also learns from its own
replies. This is real learning, but it is tiny and may learn nonsense if its
own replies are nonsense.

If you want better output quality, use `--no-self-train` so it does not learn
from its own bad replies. The script will still use retrieved past examples as
extra context.

For stronger self-training per message:

```powershell
python chat_train.py --steps-per-turn 50 --tokens 80 --temperature 0.5
```

## Add Your Own Data

Replace `data/input.txt` with a larger text file. More text improves results.
This model learns characters and style, not real reasoning.