Xhenifer/sesotho-sentiment-afriberta

Sesotho Sentiment Analysis (AfriBERTa)

A fine-tuned sentiment classifier for Sesotho news headlines, built as part of the Wale Lab R2I Fellowship (Cohort 3). Classifies text into Positive, Negative, or Neutral.

🚀 Try it live

An interactive browser-based demo (no setup required, runs entirely client-side) is available here: 👉 Sesotho Sentiment Analysis Space

The demo runs this model directly in your browser via Transformers.js and WebAssembly — no server, no API calls, nothing sent off your device. It also includes an optional South African orthography converter, for testing cross-orthography generalization between Lesotho-standard and South African Sesotho spelling conventions.

Model description

This model is a fine-tuned version of castorini/afriberta_large, adapted for 3-class sentiment classification on Sesotho news headlines.

• Base model: castorini/afriberta_large
• Task: Text classification (sentiment)
• Labels: NEGATIVE, NEUTRAL, POSITIVE
• Language: Sesotho (st)

Training data

Fine-tuned on a labeled dataset of 2,209 Sesotho news headlines. The source data is significantly imbalanced across classes:

Label	Count	Proportion
Negative	1,552	70.3%
Positive	551	24.9%
Neutral	106	4.8%

To address this imbalance, training used a class-weighted cross-entropy loss (weights: Negative 1.0×, Neutral 6.0×, Positive 3.0×), which meaningfully improved the model's ability to recognize the minority Neutral class without a corresponding drop in Negative/Positive performance.

Training procedure

• 3 epochs, learning rate 2e-5, batch size 8, warmup ratio 0.1
• 80/20 stratified train/validation split, seed 43
• Full training script and data preprocessing available on request

Evaluation results

Evaluated on a held-out validation split (442 examples):

Metric	Score
Macro F1	0.661
Matthews Correlation Coefficient	0.440
Neutral-class F1	0.605
Neutral-class recall	0.619

Limitations

• The Neutral class remains the hardest to classify reliably, given only 106 examples in the source dataset — class weighting helps but cannot fully substitute for more labeled data.
• The model was trained primarily on standard-case text; performance may vary on ALL CAPS or heavily stylized input.
• As with any sentiment classifier trained on news headlines, performance on informal or conversational Sesotho text (social media, spoken transcription, etc.) has not been evaluated and may differ.

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="Xhenifer/sesotho-sentiment-afriberta", top_k=None)
clf("TONAKHOLO O FETOLA NAHA KA BEKE")

For browser-based / client-side usage via Transformers.js, see the demo Space or its source for a working implementation.

Citation

Xhenifer. (2026). Sesotho Sentiment Analysis (AfriBERTa). Wale Lab R2I Fellowship, Cohort 3. Hugging Face. https://huggingface.co/Xhenifer/sesotho-sentiment-afriberta