Turkish district (ilçe) name generator

A small separate side task inside this repo, deliberately kept apart from the main eldamar-tokenizer BPE pipeline. Same four tiny architectures, but a different dataset (Turkish district names) and the original repo's plain character tokeniser — no BPE.

$ python districts/generate_districts.py gemma4 --count 10 --temperature 0.8 --seed 1
karaisalı  tosya    çukurca  lapseki  mudurnu
mudurnu    çukurca  göynü…   erenler  selkit

karaisalı, tosya, lapseki, mudurnu and erenler are real districts the model has learnt; selkit and the rest are invented but plausibly Turkish.

What it reuses (unchanged)

This task does not touch the src/ BPE pipeline. It reuses the original single_letter_transformers repo directly:

single_letter_transformers/data/temizle_isimler.py — the repo's Turkish aware cleaner (correct here: it lower-cases with the proper Turkish I/İ rules, splits multi-word names, and de-duplicates).
each architecture's tokenizer.py (CharTokenizer), config.py and model.py.

Data

Source: the Turkish Wikipedia article Türkiye'nin ilçeleri, a single large sortable table where column 1 is the province (il) and column 2 is the district (ilçe). scrape_ilceler.py fetches it through the MediaWiki API (URL-encoding the Turkish title so it can't get corrupted) and keeps the display text of every column-2 link.

ilceler_ham.txt — 973 raw district names (matches the article's stated count).
ilceler_temiz.txt — 950 cleaned tokens, produced by the repo's temizle_isimler.py. Vocabulary: 32 characters (29 Turkish letters + newline + the digits 1/9, which leak in from the one numeric district "19 Mayıs" → 19; left as-is because the brief was to use the repo cleaner unchanged).

Results

All four architectures trained for a uniform 5,000 steps on CPU (baseline ln 32 ≈ 3.47). Because they all share the same character tokeniser, the losses are directly comparable across architectures.

Architecture	Params	Steps	Final loss	Sample names
Qwen3 (dense)	19.6k	5000	0.85	çınar, karaman, paluova, tikmen
Qwen3.5 (hybrid)	42.1k	5000	0.57	aydıncık, havsa, başyayla, büyükçekmece
Gemma (sliding-window)	63.4k	5000	0.54	altındağ, gökçeada, sarıoğlan, arsambitözü
DeepSeek-V3 (MoE)	48.0k	5000	0.62	çamlıhemşin, konyaaltı, bigadiç, çatalpınar

Run it

python districts/scrape_ilceler.py                                              # -> ilceler_ham.txt (973)
python single_letter_transformers/data/temizle_isimler.py \
       districts/ilceler_ham.txt districts/ilceler_temiz.txt                    # -> ilceler_temiz.txt (950)

for a in qwen3 qwen3_5 gemma4 deepseek3; do python districts/train_districts.py $a; done
python districts/generate_districts.py gemma4 --count 20 --temperature 0.8

train_districts.py <arch> [--steps N] and generate_districts.py <arch> [--count N --temperature T --seed S]. Keep the step count the same across all four for a fair comparison (the default is 5,000 for every architecture).

Files

districts/
├── scrape_ilceler.py       # fetch + extract column 2 (İlçe) from tr.wikipedia
├── ilceler_ham.txt         # 973 raw district names
├── ilceler_temiz.txt       # 950 cleaned tokens (via the repo's temizle_isimler.py)
├── train_districts.py      # train one architecture (CharTokenizer) → checkpoints/<arch>.pt
├── generate_districts.py   # sample district names from a checkpoint
└── checkpoints/            # <arch>.pt (4 models)