Mirror of BultenHub/bhkai, BültenHub's official repository.
BHKAI: Turkish news category classifier
BHKAI («BültenHub Kategori AI») assigns a category to each incoming Turkish news story. In BültenHub it answers on its own when its confidence is at least 0.8; otherwise the story goes to a larger teacher model. It is retrained every night and replaces the production version only if it passes the same exam.
Version
| Version | bhkai-20261116-alpha1e-6 |
| Trained | 2026-11-16 |
| Method | TF-IDF word 1–2-grams (200,000 features, min_df 3) + SGD logistic classifier (alpha 1e-06) |
| Training data | 588,702 stories labelled by the teacher model (data cut-off 2026-11-16) |
| Replaced | bhkai-20261115 |
Exam (the metrics the nightly gate uses)
Held-out recent stories the previous version never saw (3572 stories; stories the production model was confident on are weighted by 1/audit-rate so the exam matches production traffic). Coverage = share of stories BHKAI decides on its own; agreement = same category as the teacher model.
| Confidence ≥ | Coverage | Agreement | Previous: coverage | Previous: agreement |
|---|---|---|---|---|
| 0.5 | 0.871 | 0.897 | 0.822 | 0.896 |
| 0.7 | 0.676 | 0.960 | 0.624 | 0.961 |
| 0.8 (production) | 0.576 | 0.981 | 0.529 | 0.979 |
| 0.85 | 0.482 | 0.988 | 0.276 | 1.000 |
| 0.9 | 0.319 | 1.000 | 0.228 | 1.000 |
Gate decision: promoted — at confidence ≥ 0.8: coverage 57.6% (was 52.9%), agreement with teacher 98.1% (was 97.9%).
Usage (Türkçe örnek)
# pip install huggingface_hub numpy
import json, sys
from huggingface_hub import snapshot_download
yol = snapshot_download("BultenHub/bhkai")
sys.path.insert(0, yol)
from bhkai import Bhkai
model = Bhkai(yol)
kategori_id, guven = model.tahmin(
"Kurgukent'te 24 saat açık kent kütüphanesi hizmete girdi", # başlık (kurgusal)
"Kurgukent Belediyesi'nin eski itfaiye binasında kurduğu kütüphane dün açıldı. "
"18 bin kitap ve 120 kişilik çalışma salonu bulunuyor.", # metnin başı
"Kurgukent Haber", # kaynak adı
)
adlar = json.load(open(f"{yol}/kategoriler.json", encoding="utf-8"))
print(adlar.get(str(kategori_id), kategori_id), round(guven, 2))
# BültenHub'da güven ≥ 0.8 ise kararı BHKAI verir, değilse haber büyük modele gider.
Files: agirlik.npz (weights), sozluk.json (vocabulary), bilgi.json (version, exam), bhkai.py (numpy inference), kategoriler.json (category id → name).
Limitations
- Labels come from a teacher LLM, not from human editors; agreement is measured against that teacher.
- Trained on Turkish news headlines and the first 500 characters of the text; other domains and languages are out of scope.
- Low-confidence predictions are unreliable by design: use the confidence threshold.
Changelog
- 2026-11-16
bhkai-20261116-alpha1e-6: at confidence ≥ 0.8: coverage 57.6% (was 52.9%), agreement with teacher 98.1% (was 97.9%) - 2026-11-15
bhkai-20261115: at confidence ≥ 0.8: coverage 51.4% (was 51.4%), agreement with teacher 95.9% (was 95.6%) - 2026-11-13
bhkai-20261113: at confidence ≥ 0.8: coverage 50.2% (was 49.7%), agreement with teacher 96.4% (was 95.3%) - 2026-11-12
bhkai-20261112: at confidence ≥ 0.8: coverage 48.1% (was 47.8%), agreement with teacher 96.9% (was 96.5%) - 2026-11-10
bhkai-20261110: at confidence ≥ 0.8: coverage 45.2% (was 44.4%), agreement with teacher 97.4% (was 97.3%) - 2026-11-04
bhkai-20261104: at confidence ≥ 0.8: coverage 57.2% (was 57.7%), agreement with teacher 97.4% (was 97.2%) - 2026-11-03
bhkai-20261103: at confidence ≥ 0.8: coverage 46.3% (was 46.3%), agreement with teacher 97.2% (was 95.3%) - 2026-11-01
bhkai-20261101: at confidence ≥ 0.8: coverage 56.1% (was 56.1%), agreement with teacher 96.5% (was 96.7%)
License
CC BY-NC 4.0 (non-commercial). The base model, if any, keeps its own license. For commercial use, contact https://bultenhub.com.
Evaluation results
- Agreement with teacher at confidence ≥ 0.8 on BültenHub nightly category exam (held-out recent stories, teacher labels; not public)test set self-reported0.981
- Coverage at confidence ≥ 0.8 on BültenHub nightly category exam (held-out recent stories, teacher labels; not public)test set self-reported0.576
- Agreement with teacher, all exam stories on BültenHub nightly category exam (held-out recent stories, teacher labels; not public)test set self-reported0.703