MakiniAI
/

makini-bias-v2

@@ -1,199 +1,138 @@
 ---
-library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
+license: cc-by-nc-4.0
+language:
+- sw
+- fr
+- zu
+- ha
+base_model: Davlan/afro-xlmr-base
+tags:
+- bias-detection
+- african-languages
+- text-classification
+- responsible-ai
+pipeline_tag: text-classification
 ---
+# MAKINI v2 — African Language Bias Detection Model
+## Model Description
+MAKINI v2 is a 4-class bias detection model covering Swahili, French, isiZulu, and Hausa, fine-tuned on the AfricaBias dataset (60,234 sentences). It succeeds MAKINI v1 (Swahili/French only) — this is a new backbone and two additional languages, not an incremental v1.1.
+**Validated for production use: Swahili, French, isiZulu — all four classes.**
+**Hausa: neutral-vs-not detection only.** Hausa's stereotype, counter-stereotype, and derogation classes are not reliable in this release — see Known Limitations before using Hausa output for anything beyond flagging "neutral or not."
+## Base Model
+`Davlan/afro-xlmr-base`, not vanilla `xlm-roberta-base` (used in v1). Chosen because standard XLM-R's pretraining did not meaningfully cover isiZulu; AfroXLMR extends coverage to isiZulu, Hausa, and Chichewa (planned next language) in one backbone.
+## Training Approach
+Two-stage fine-tune, same recipe as v1:
+- **Stage 1** — binary (neutral vs. biased), warm-start. F1 macro: 0.963
+- **Stage 2** — 4-class, initialized from Stage 1's encoder, weighted cross-entropy for class imbalance
+Note: this release uses the same Swahili/French label distribution as v1 (24,220 neutral / 9,605 stereotype / 1,288 counter-stereotype / 172 derogation, combined). The separate counterstereotype/derogation rebalancing effort for Swahili/French has not yet been incorporated into any shipped model.
+## Labels
+| ID | Label |
+| -- | ----- |
+| 0 | neutral |
+| 1 | stereotype |
+| 2 | counter-stereotype |
+| 3 | derogation |
+## Performance — Held-Out Test Set (n=6,024)
+### Overall
+| Class | Precision | Recall | F1 | Support |
+| ----- | --------- | ------ | -- | ------- |
+| neutral | 0.97 | 0.98 | 0.97 | 3,963 |
+| stereotype | 0.95 | 0.94 | 0.95 | 1,837 |
+| counter-stereotype | 1.00 | 0.89 | 0.94 | 141 |
+| derogation | 0.78 | 0.64 | 0.70 | 83 |
+| **accuracy** | | | **0.96** | 6,024 |
+| macro avg | 0.92 | 0.86 | 0.89 | 6,024 |
+| weighted avg | 0.96 | 0.96 | 0.96 | 6,024 |
+The overall row is not a safe substitute for the per-language breakdown below — see Known Limitations.
+### Per Language
+**French (n=1,149)** — accuracy 0.95, macro F1 0.96
+| Class | Precision | Recall | F1 | Support |
+| ----- | --------- | ------ | -- | ------- |
+| neutral | 0.98 | 0.93 | 0.95 | 600 |
+| stereotype | 0.91 | 0.97 | 0.94 | 443 |
+| counter-stereotype | 1.00 | 1.00 | 1.00 | 91 |
+| derogation | 0.88 | 1.00 | 0.94 | 15 |
+**Swahili (n=2,418)** — accuracy 0.98, macro F1 0.74 (derogation support = 1, statistically meaningless — see Known Limitations)
+| Class | Precision | Recall | F1 | Support |
+| ----- | --------- | ------ | -- | ------- |
+| neutral | 0.99 | 0.99 | 0.99 | 1,840 |
+| stereotype | 0.97 | 0.96 | 0.97 | 541 |
+| counter-stereotype | 1.00 | 0.97 | 0.99 | 36 |
+| derogation | 0.00 | 0.00 | 0.00 | 1 |
+**isiZulu (n=1,486)** — accuracy 0.99, macro F1 0.72 (counter-stereotype: no examples exist in source data — see Known Limitations)
+| Class | Precision | Recall | F1 | Support |
+| ----- | --------- | ------ | -- | ------- |
+| neutral | 0.99 | 0.99 | 0.99 | 667 |
+| stereotype | 0.99 | 0.99 | 0.99 | 783 |
+| counter-stereotype | — | — | — | 0 |
+| derogation | 0.94 | 0.89 | 0.91 | 36 |
+**Hausa (n=971)** — accuracy 0.88, macro F1 0.32. **Neutral detection works; the other three classes do not.**
+| Class | Precision | Recall | F1 | Support |
+| ----- | --------- | ------ | -- | ------- |
+| neutral | 0.90 | 0.98 | 0.94 | 856 |
+| stereotype | 0.19 | 0.04 | 0.07 | 70 |
+| counter-stereotype | 0.00 | 0.00 | 0.00 | 14 |
+| derogation | 0.38 | 0.19 | 0.26 | 31 |
+## Known Limitations
+**Hausa: stereotype, counter-stereotype, and derogation detection are not usable.** Recall on stereotype is 0.04 — the model misses 96% of actual Hausa stereotype content. Counter-stereotype recall is 0.00. Only neutral-vs-not-neutral is reliable for Hausa in this release. Root cause: Hausa's own 4-class split is heavily neutral-skewed (89%), and its 718/124/293 stereotype/counter-stereotype/derogation examples are a small fraction of the pooled multilingual training signal for those classes — global class weighting does not protect a single language's minority share within an already-weighted class. Addressing this needs either more Hausa non-neutral annotation or language-aware reweighting, not just longer training.
+**isiZulu has no counter-stereotype examples** in the source data — this class cannot be evaluated for isiZulu and its true performance on isiZulu counter-stereotype content is unknown, not zero.
+**Swahili derogation test support is 1 example.** The 0.00 F1 shown above reflects a single data point, not a real measurement of Swahili derogation performance.
+**isiZulu's `domain` field is unreliable for ~43% of source rows** (see dataset card) — do not use domain-level breakdowns for isiZulu.
+Limitations carried over from v1, not addressed by this release:
+- Negation handling: stereotypes stated then negated are not reliably detected
+- Sheng (informal Kenyan Swahili slang) is underrepresented in training data
+- Implicit stereotyping conveyed through French sentence structure may be missed
+## Languages
+Swahili, French, isiZulu, Hausa (Hausa scoped per Known Limitations above).
+## License
+CC BY-NC 4.0. Commercial use requires explicit permission from Makini AI Limited.
+## Citation
+```bibtex
+@model{wachira2026makiniv2,
+  title     = {MAKINI v2: African Language Bias Detection Model},
+  author    = {Wachira, David Maina},
+  year      = {2026},
+  publisher = {Algedi Intelligence Labs},
+  url       = {https://huggingface.co/MakiniAI/makini-v2},
+}
+```
+## Contact
+David Maina Wachira — dave@makini.tech — https://makini.tech