Univars/k-gpt-v1

K-GPT v1 — Kenya's Sovereign AI

K-GPT is version one of Kenya's — and eventually Africa's — Sovereign AI project, being undertaken by a team from Univars.space under Rewel AI Labs.

K-GPT is designed to deeply understand and serve Kenyan citizens in their own languages, cultural contexts, legal frameworks, and economic realities. It is the first step toward building a truly African-grounded large language model.

---

Model Description

K-GPT v1 is a QLoRA-finetuned adaptation of Qwen 2.5 7B, trained on 27 curated Kenyan datasets totaling approximately 18.9 million tokens. The model specializes in Kenyan multilingual understanding, constitutional law, cultural nuance, and domain-specific knowledge across banking, tourism, agriculture, and governance.

Property	Value
Base Model	Qwen/Qwen2.5-7B
Fine-tuning Method	QLoRA SFT (4-bit NF4 quantization)
LoRA Configuration	r=64, alpha=128, dropout=0.05
Training Duration	3 epochs, 1,041 steps (~2h 11m on RTX 4090)
Final Training Loss	0.000161
Total Training Tokens	~18.9 million
Datasets	27 curated Kenyan datasets

Languages Supported

K-GPT v1 supports the following languages and dialects:

• Formal English (Kenyan context)
• Swahili (Standard & Kenyan)
• Sheng (Nairobi urban slang/pidgin)
• Kamba (Kikamba)
• Luo (Dholuo)
• Kikuyu (Gikuyu)
• Luhya (Maragoli & related)
• Kidawida (Taita)
• Kalenjin
• Somali (Kenyan Somali communities)

Specialized Knowledge Domains

• Kenyan Parliament Hansard — Constitutional debates and legislative proceedings
• 2010 Constitution of Kenya — Full legal text and interpretive context
• Mount Kenya Tourism Guide — Cultural and geographical tourism content
• Central Bank of Kenya (CBK) Regulations — Banking and financial compliance
• Kenyan Agricultural Data — Crop patterns, market prices, and farming guides
• Data Protection Act 2019 — Privacy and data sovereignty frameworks

---

Intended Uses & Limitations

Intended Uses

• Sovereign AI Chatbot: Powering culturally-aware conversational AI for Kenyan users via the Univars Sovereign Chat platform.
• Multilingual Customer Service: Serving Kenyans in their native languages across banking, government, tourism, and agriculture sectors.
• Legal & Constitutional Q&A: Answering questions about Kenyan law, the 2010 Constitution, and parliamentary proceedings.
• Cultural Preservation: Documenting and making accessible knowledge in underrepresented Kenyan languages (Sheng, Kamba, Kidawida, etc.).
• Research: Advancing African NLP and sovereign AI research.

Limitations

• Regional Focus: K-GPT v1 is primarily trained on Kenyan data. Performance on other African countries' contexts may be limited.
• Base Model Constraints: Inherits limitations of Qwen 2.5 7B, including potential biases in the pre-training data.
• Low-Resource Languages: While the model covers 10+ Kenyan languages, training data volume varies — Swahili and English have the deepest coverage, while languages like Kidawida and Somali have fewer examples.
• Not a Legal Advisor: While trained on constitutional and legal text, K-GPT should not be used as a substitute for professional legal advice.
• Hallucination Risk: Like all LLMs, K-GPT may generate plausible-sounding but incorrect information.

---

Training and Evaluation Data

Training Data

K-GPT v1 was trained on 27 carefully curated datasets covering:

Category	Examples	Tokens (approx.)
Multilingual Corpora	Swahili-English parallel text, Sheng dictionaries, Kamba/Luo/Kikuyu text	~6M
Government & Legal	Kenya 2010 Constitution, Parliament Hansard transcripts, DPA 2019	~4M
Financial & Banking	CBK regulations, M-Pesa transaction patterns, KYC compliance docs	~2.5M
Tourism & Culture	Mt. Kenya tourism guides, cultural heritage documents, folklore	~2M
Agriculture	Crop data, market prices, extension service guides	~1.5M
News & Media	Kenyan news articles, social media, community forums	~2.9M
Total	27 datasets	~18.9M tokens

Evaluation

• Training was monitored via loss curves across 1,041 steps.
• Final training loss: 0.000161 (indicating strong convergence).
• Qualitative evaluation was performed on multilingual prompts across all supported languages.
• The model is currently deployed in Sovereign Chat (Testing Studio) within the Univars LLMOps platform for ongoing human evaluation.

---

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B")
tokenizer = AutoTokenizer.from_pretrained("Univars/k-gpt-v1")

# Load K-GPT adapter
model = PeftModel.from_pretrained(base_model, "Univars/k-gpt-v1")

# Generate
inputs = tokenizer("Habari! Niambie kuhusu Katiba ya Kenya ya 2010.", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

About

K-GPT is built by the Univars team at Rewel AI Labs.

• Website: univars.space
• Organization: Rewel AI Labs
• Location: Nairobi, Kenya
• Access: Gated -- approval required

---

K-GPT v1 is the foundation for a pan-African Sovereign AI initiative. Future versions will expand to cover all 54 African nations, additional languages, and domain-specific capabilities.