Daudipdg/makini-v1

MAKINI v1 — African Language Bias Detection Engine

Model Summary

MAKINI is a fine-tuned XLM-RoBERTa model for detecting gender bias in Swahili and African French text. It is the first trained bias detection model for these language variants.

Developed by Algedi Intelligence Labs (Nairobi, Kenya).

Labels

ID	Label	Description
0	neutral	No bias detected
1	stereotype	Reinforces gender stereotypes
2	counter-stereotype	Challenges gender stereotypes
3	derogation	Derogatory language targeting a gender

Performance (Test Set — 3,528 sentences)

Class	Precision	Recall	F1
neutral	0.99	0.98	0.99
stereotype	0.96	0.98	0.97
counter-stereotype	1.00	0.99	1.00
derogation	0.68	0.88	0.77
overall accuracy			0.98

Benchmark — MAKINI v1 vs Existing Models

Evaluated on 3,528 sentences in Swahili and African French.

Model	F1 Biased	F1 Neutral	F1 Macro	Accuracy
MAKINI v1 (ours)	0.971	0.987	0.979	0.982
cardiffnlp/twitter-xlm-roberta-base-sentiment	0.437	0.632	0.534	0.555
Davlan/afro-xlmr-mini	0.145	0.767	0.456	0.633
valurank/distilroberta-bias	0.477	0.000	0.239	0.314

MAKINI v1 outperforms the nearest competitor by 44.7 F1 macro points.

Key finding: Afro-XLMR was trained specifically on African languages yet scores 52.3 points below MAKINI. This confirms that language knowledge alone is insufficient — domain-specific bias training data is the critical differentiator.

Training Data

Fine-tuned on AfricaBias-SW-FR — 35,285 annotated sentences in Swahili (24,289) and African French (10,996), covering 8 social domains.

Supported by the Gates Foundation / AfriLabs Accelerator program.

Training Approach

Two-stage fine-tuning on xlm-roberta-base:

• Stage 1: Binary classification (biased vs neutral) — F1: 0.96
• Stage 2: 4-class fine-tuning from Stage 1 checkpoint with weighted cross-entropy loss

Usage

from transformers import pipeline

makini = pipeline(
    "text-classification",
    model="Daudipdg/makini-v1",
    return_all_scores=True
)

result = makini("Mwanamke anapaswa kukaa nyumbani na watoto")

Known Limitations

• Negation: Cannot reliably detect when a stereotype is stated then negated.
• Sheng: Informal Kenyan slang gender terms (dem, msupa, chali) are underrepresented.
• Implicit French stereotyping: Role assumptions embedded in sentence structure may be missed.
• Derogation recall: Only 172 training examples. Treat low-confidence derogation predictions as indicative only.
• Languages: Swahili and African French only in v1.

Intended Use

• Auditing AI systems for gender bias in Swahili and African French outputs
• Research on African language bias detection
• Integration into content moderation and responsible AI pipelines

License

CC BY-NC 4.0 — Free for research and non-commercial use. Commercial use requires explicit permission from Algedi Intelligence Labs.

Citation

@model{wachira2026makini,
  title     = {MAKINI v1: African Language Bias Detection Engine},
  author    = {Wachira, David Maina},
  year      = {2026},
  publisher = {Algedi Intelligence Labs},
  url       = {https://huggingface.co/Daudipdg/makini-v1},
  note      = {Fine-tuned on AfricaBias-SW-FR. Supported by Gates Foundation / AfriLabs Accelerator}
}

Contact

David Maina Wachira — david@makini.tech — makini.tech