korkmazumut/orkhon-bengu

Orkhon / bengu

57M English/Turkish bilingual base model in the Turkic branch.

This repository is one member of the Orkhon model family, an auditable from-scratch LLM stack covering tokenizer, pretraining, post-training, evaluation, serving, and Hugging Face export.

• Family: Orkhon
• Member: bengu
• Kind: base
• Interface: completion
• Parameters: ~57.4M
• Source code: https://github.com/UmutKorkmaz/orkhon
• Local model-zoo folder: models/bengu-20260615

Family Context

Member	Role	HF repo
bumin	4M arithmetic/chat smoke model proving the full Orkhon pipeline.	korkmazumut/orkhon-bumin
tonyukuk	22M TinyStories base model trained from scratch with Orkhon.	korkmazumut/orkhon-tonyukuk
kultigin	22M story-instruction SFT member of the Orkhon family.	korkmazumut/orkhon-kultigin
istemi	51M FineWeb-Edu base model trained on real web text.	korkmazumut/orkhon-istemi
bengu current	57M English/Turkish bilingual base model in the Turkic branch.	korkmazumut/orkhon-bengu
bengu-gokturk	57M Old Turkic rune-to-Latin transliterator SFT. This is transliteration, not translation.	korkmazumut/orkhon-bengu-gokturk

kashgari is intentionally not part of this first upload batch because the local model-zoo entry stores metadata and a re-import hint, not archived weights.

Intended Use

• Inspecting and reproducing the Orkhon training/export path.
• Running small local demos on CPU/MPS/CUDA.
• Comparing Orkhon family members by training stage and data mix.
• For bengu-gokturk: rune-to-Latin Old Turkic transliteration demos.

Not Intended For

• Reliable factual QA.
• Safety-critical decisions.
• Claims of state-of-the-art Turkish or general LLM performance.
• Treating Old Turkic transliteration as modern Turkish translation.

Evaluation

Metric	Value
loss	3.852317476272583
ppl	47.10209483086865
tokens	163840

Current benchmark reports in the source repo are smoke baselines unless marked otherwise. Do not treat limit-20 benchmark runs as headline capability claims.

Example Prompt

Turkiye'nin baskenti Ankara

Usage

Install Orkhon from the source repo, then load this exported folder:

pip install git+https://github.com/UmutKorkmaz/orkhon

from huggingface_hub import snapshot_download
from orkhon.export.to_hf import load_exported_model
from orkhon.tokenizer import load_tokenizer

path = snapshot_download("korkmazumut/orkhon-bengu")
model, cfg = load_exported_model(path, device="cpu")
tok = load_tokenizer(path)

For chat/instruct members, use orkhon.serve.chat_cli.reply. For base members, use raw completion with orkhon.model.generate.

Files

• model.safetensors - exported inference weights
• config.json - Orkhon architecture config
• tokenizer.json, tokenizer_config.json, special_tokens_map.json
• manifest.json - model-zoo metadata from the source repo
• samples.txt - saved local samples when available
• orkhon_model_card.md - original local model card

Limitations

This is a small research/education model family. The models can repeat, confabulate, and fail outside their narrow data scale. Public demos should keep the claim narrow: Orkhon is an auditable from-scratch stack with a Turkic branch, not a frontier assistant.