Instructions to use mete72722/anadolu-40m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mete72722/anadolu-40m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="mete72722/anadolu-40m") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("mete72722/anadolu-40m") model = AutoModelForCausalLM.from_pretrained("mete72722/anadolu-40m", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use mete72722/anadolu-40m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "mete72722/anadolu-40m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mete72722/anadolu-40m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/mete72722/anadolu-40m
- SGLang
How to use mete72722/anadolu-40m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "mete72722/anadolu-40m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mete72722/anadolu-40m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "mete72722/anadolu-40m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mete72722/anadolu-40m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use mete72722/anadolu-40m with Docker Model Runner:
docker model run hf.co/mete72722/anadolu-40m
Anadolu-40M
Anadolu is a 40M-parameter Turkish causal language model trained from scratch. It needs no GPU: it was pre-trained, fine-tuned and is served on a single Intel i7-7700 CPU. It ships with a NumPy-only inference engine, so the model can run on a machine with no PyTorch and no CUDA.
- 40,178,176 parameters · 512-token context · 16,389-token byte-level BPE (no
UNK) - Standard decoder-only Transformer, GPT-2 compatible weights →
AutoModelForCausalLM - Chat format:
<|bos|><|user|>…<|end|><|assistant|>…<|end|> - Hosted demo & free OpenAI-compatible API: https://anadoluai.duckdns.org:8443
- Discord bot with per-user memory:
Anadolu AI#4542
This is a small model. It is not a GPT-4 substitute — see Limitations.
Model details
| Architecture | Decoder-only Transformer (GPT-2 compatible) |
| Parameters | 40,178,176 |
Layers / heads / d_model |
10 / 8 / 512 |
| MLP inner size | 2048 (GELU, gelu_new / tanh approximation) |
| Context length | 512 tokens |
| Vocabulary | 16,389 (byte-level BPE, GPT-2 style — every byte is representable, no UNK) |
| Positional encoding | Learned |
| Normalisation | Pre-LayerNorm, eps = 1e-5 |
| Embeddings | Tied input/output |
| Precision | float32 |
| Runtime | NumPy only (numpy-fp32 export) |
Special tokens
| Token | ID |
|---|---|
| `< | pad |
| `< | user |
| `< | assistant |
| `< | end |
| `< | bos |
Training
| Stage | Detail |
|---|---|
| Data | Turkish Wikipedia (wikimedia/wikipedia, 20231101.tr, CC BY-SA) + Bilkent Turkish Writings Dataset v2 |
| Corpus size | ~870.8M characters → ~227.6M BPE tokens available |
| Tokens seen | 344,064,000 (as recorded in the export metadata) |
| Stage 1 | From-scratch pre-training on CPU, then 512-token context adaptation |
| Stage 2 | Supervised fine-tuning (SFT) for chat, ~500 steps (best-validation checkpoint of a 600-step run) |
| SFT data | 1,399 hand-written Turkish chat examples + 208 pairs distilled from larger models |
| Source checkpoint | checkpoints/anadolu40x512b/sft512c.pt |
| Export | numpy-fp32, 2026-11-18 |
Evaluation
| Metric | Value |
|---|---|
| Language-modelling validation loss | 3.8330 |
| Perplexity | 46.199 |
| Pre-training validation loss | 2.9253 |
| SFT validation loss | 0.4566 |
| LM eval setup | 128 sequences · 65,408 tokens · block 512 · seed 1234 |
Perplexity is measured on held-out Turkish text from the same distribution as training. It is not a comparison against other Turkish models — no apples-to-apples benchmark has been run yet.
Chat quality and raw language-modelling perplexity can move in opposite directions. This export was re-trained on a larger, cleaner SFT set, which raised the LM perplexity (26.1 → 46.2) while clearly improving the answers users see: on the 8-prompt probe set the defect rate fell from 0.38 to 0.00 per reply, all 8 answers were correct, and there were no repetition loops. Knowledge questions are answered from the service's knowledge base in production, so the perplexity change is not visible to API users.
Usage — transformers
pip install torch transformers
python example_transformers.py
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "USERNAME/anadolu-40m" # ← replace with this repo's id
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo).eval()
messages = [{"role": "user", "content": "Merhaba, nasılsın?"}]
ids = tok.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
)
with torch.no_grad():
out = model.generate(
ids,
max_new_tokens=96,
do_sample=False, # önerilen: açgözlü üretim (bkz. aşağıda)
)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
The model is a plain GPT-2, so transformers pipeline("text-generation"),
GPT2LMHeadModel, LoRA fine-tuning and GGUF/llama.cpp conversion are all
possible without custom code.
Recommended generation settings
Use greedy decoding (do_sample=False) and leave no_repeat_ngram_size at 0.
Both matter a lot on a 40M model:
- Prefer greedy decoding. Sampled decoding (
temperature=0.7, top_p=0.92) is usable withno_repeat_ngram_size=0, but greedy output is the most reliable on a 40M model. no_repeat_ngram_size=3hurts. Blocking repeated trigrams is a common anti-loop trick, but here it pushes the model onto rare sub-word pieces and it starts producing nonsense words. Measured on the 8-prompt probe set with greedy decoding andrepetition_penalty=1.08(values below are from the previous export; the principle is unchanged):
| Prompt | no_repeat_ngram_size=3 |
no_repeat_ngram_size=0 |
|---|---|---|
Yapay zekâ nedir? |
Yapay sinirlerin bütünüdür, emülsiyon ve emülsifiyim, sinirlerinizi ayır… |
Yapay zekâ, bilgisayarların öğrenme, akıl yürütme ve karar verme gibi insan zekâsına benze… |
Kitap okumak neden faydalıdır? |
Kitap okumadan önce, bir sayfa okuyarak veyaetrafortraya göre değişir. |
Kitap okumak, bir problemi çözmek için bir yöntemdir. |
Kahve içmek zararlı mı? |
Kahvenin, kahve içer, su ve su içer, içmek su içer ve su içerir. |
Kahve içmek zararlı mı? |
Merhaba, nasılsın? |
Merhacı, bir gerdanyana eylemi'dir. |
Merhaba! İyiyim, teşekkürler. Sen nasılsın? |
generation_config.json therefore ships with do_sample: false and
no_repeat_ngram_size: 0, and repetition_penalty: 1.08.
Note that the public API at anadoluai.duckdns.org:8443 uses sampled decoding
(temperature=0.7) with no_repeat_ngram_size=0, and wraps the model in a
router with a knowledge-base layer and a garbled-output fallback, so its answers
are not directly comparable to a bare model.generate() call.
Usage — NumPy only (no PyTorch)
The repo also contains the original NumPy inference engine under anadolu/.
This is the code that actually serves the public API; it needs only numpy,
safetensors and threadpoolctl — no PyTorch, no CUDA.
pip install numpy safetensors threadpoolctl
python example_numpy.py
from anadolu.chatml import build_prompt_ids
from anadolu.runtime import Engine, GenerationConfig
# `example_numpy.py` converts model.safetensors into the layout the engine
# expects and writes it to a temp dir; `work` is that directory.
engine = Engine(work, threads=4)
ids = build_prompt_ids(
engine.weights.tokenizer,
[{"role": "user", "content": "Merhaba, nasılsın?"}],
max_tokens=engine.weights.block_size,
)
print(engine.generate(ids, GenerationConfig(max_new_tokens=96))["text"])
Files
| File | Description |
|---|---|
model.safetensors |
Weights in GPT-2 layout (fp32) |
config.json |
GPT-2 config (n_embd=512, n_positions=512, n_layer=10, n_head=8) |
generation_config.json |
Default sampling parameters |
bpe.json |
Original byte-level BPE vocabulary (used by the NumPy engine) |
tokenizer.json, tokenizer_config.json, special_tokens_map.json |
HF tokenizer + chat template |
anadolu/ |
NumPy-only inference engine (tokenizer, runtime, chat template) |
example_transformers.py |
Run with transformers |
example_numpy.py |
Run with NumPy only |
meta.json |
Export metadata (steps, losses, evaluation) |
Hosted API (free)
An OpenAI-compatible endpoint is available; anonymous use needs no key (10 requests/min, 100,000 tokens/day):
curl -sk https://anadoluai.duckdns.org:8443/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "anadolu-40m",
"messages": [{"role": "user", "content": "Türkiye'"'"'nin başkenti neresi?"}]
}'
The same endpoint is served by the NumPy engine in this repository, so a self-hosted instance behaves identically.
Limitations
- Small model. 40M parameters cannot follow complex instructions, do long reasoning, or match large models on knowledge questions. Short Turkish chat, simple Q&A, rewriting and short summaries are the realistic use cases.
- Repetition. Like other small models it can loop. A repetition penalty
(
repetition_penalty=1.08) is enabled by default for this reason;no_repeat_ngram_sizeis left at0because the measured effect of setting it to3was a severe quality loss (see Recommended generation settings). - Knowledge cutoff / facts. Trained on a Wikipedia dump and a student-writing corpus, so it is unreliable for current events and for factual recall outside its training distribution.
- No safety tuning. The model was not RLHF'd or red-teamed; outputs should be reviewed before any user-facing deployment.
- Turkish only. It is trained on Turkish data and is not a multilingual model.
- Context is 512 tokens including the prompt.
Lisans / License
Code and weights are published for research and evaluation use.
⚠️ The training data is not fully permissive. It includes Turkish Wikipedia (CC BY-SA) and the Bilkent Turkish Writings Dataset v2, which is distributed under an Academic Use License (non-commercial, attribution required). Please review those terms before any commercial use of the weights. A final license for this repository has not been fixed yet.
Türkçe özet
Anadolu-40M, sıfırdan eğitilmiş 40 milyon parametreli bir Türkçe dil
modelidir. GPU gerektirmez; eğitimi de sunumu da tek bir Intel i7-7700
işlemcisinde yapılır. Ağırlıklar GPT-2 uyumlu olduğu için transformers ile
doğrudan kullanılabilir; ayrıca repoda yalnızca NumPy ile çalışan özgün çıkarım
motoru bulunur.
Küçük bir modeldir: kısa Türkçe sohbet, basit soru-cevap, yeniden yazma ve kısa özet işleri için uygundur. Uzun akıl yürütme, güncel bilgi ve karmaşık talimat takibi beklenmemelidir. Eğitim verisi Türkçe Vikipedi (CC BY-SA) ile Bilkent Türkçe Yazılar Veri Kümesi v2'yi (akademik, ticari olmayan) içerir.
- Web: https://anadoluai.duckdns.org:8443
- Discord botu:
Anadolu AI#4542 - Ücretsiz API:
POST /v1/chat/completions(OpenAI uyumlu)
- Downloads last month
- 298