Lighthouse Keeper 0.1

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+keeper.spm filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,144 @@
+---
+license: apache-2.0
+language:
+- en
+- pcm
+- yo
+- ha
+- ig
+library_name: onnx
+pipeline_tag: text-classification
+tags:
+- child-safety
+- content-moderation
+- nigeria
+- on-device
+---
+
+# Lighthouse Keeper 0.1
+
+Keeper is the text safety model inside Lighthouse, a child safety app from Prosperity Tech. It reads short messages on a child's phone and scores them for 13 kinds of risk. It is small enough to run on a cheap Android phone with no internet connection, so the message never has to leave the device.
+
+We built it for Nigerian children. It understands Nigerian English, Pidgin, Yoruba, Hausa and Igbo, including the way teenagers actually type: mixed languages, slang, no tone marks, typos.
+
+This is our first version. It is useful, it is not finished, and the weak spots are listed below.
+
+## What it does
+
+Give it one message. It returns a score for each of these 13 categories:
+
+bullying, eating disorders, gambling, graphic content, grooming, hate speech, privacy threats, scams, self harm, sextortion, sexual content, substance use, violence.
+
+For each category you get four numbers that add up to 1: how likely the message is at level none, low, review or high.
+
+- **high**: act now. A stranger asking a child to keep a secret and send photos.
+- **review**: a parent should look. A friend pushing a betting app.
+- **low**: worth counting, not worth an alert. A mild insult between classmates.
+- **none**: nothing there, even if the message uses the same words.
+
+In the Lighthouse app we take the category with the highest level and only flag it when the chance of any risk is at least 0.8. When two categories tie, the more serious one wins, so a threat to share nudes shows as sextortion and not bullying. The order is in `config.json`. You can pick your own cutoff.
+
+## Size and speed
+
+| | |
+|---|---|
+| Parameters | 8.3 million |
+| `keeper.onnx` (int8, what we ship) | 8.5 MB |
+| `keeper-fp32.onnx` | 33 MB |
+| `keeper.spm` (word pieces) | 0.6 MB |
+| Longest input | 128 word pieces, longer messages are cut |
+| Speed | under 1 ms per message on a laptop CPU |
+
+The model is a 4 layer encoder, 256 wide, with its own 20,000 piece SentencePiece vocabulary. It takes any length up to 128.
+
+## How to use it
+
+```bash
+pip install onnxruntime sentencepiece huggingface_hub numpy
+python example.py "how far, wetin be your house address and which school you dey"
+```
+
+`example.py` in this repo shows the whole thing in about 30 lines: split the message with `keeper.spm`, put id 2 in front, run `keeper.onnx`, read back 13 rows of 4 numbers. The category order and level names are in `config.json`.
+
+## How we built it
+
+We did not have a big labelled dataset for Nigerian languages, so we made one.
+
+1. We wrote a short rule sheet for each category: what counts, what doesn't, and examples of each level. The rules come from public child safety guidance and Nigerian hate speech research.
+2. We collected about 300,000 messages from public datasets and had gpt-oss-120b write about 100,000 more in Nigerian English and Pidgin.
+3. A large open safety model, [gpt-oss-safeguard-20b](https://huggingface.co/openai/gpt-oss-safeguard-20b), read every message against every rule sheet and gave a score from 0 to 3. We kept its full probabilities, not just its answer.
+4. Keeper learned to copy those scores. This is called distillation: the big model teaches, the small one learns.
+5. The first versions were weak on Yoruba and Hausa. So we had [AfriqueQwen 9B](https://huggingface.co/McGill-NLP/AfriqueQwen3.5-9B-50Langs-Instruct-v1) write 68,000 new messages in Yoruba, Hausa and Igbo, each with an English meaning. The teacher scored the English meaning and Keeper learned that score for the original message. We also let human labels from AfriHate win over the teacher, and we read 2.4 million lines of Yoruba, Hausa, Igbo, Pidgin and English web text before training.
+
+The final training set is about 460,000 messages. Training takes under an hour on one GPU.
+
+## How well it works
+
+We test on 746 messages that Keeper never saw in training: real AfriHate tweets, messages from OpenAI's teen safety test pack, and messages we wrote across all five languages and all 13 categories. A message passes when Keeper puts it at the right level for its category.
+
+| | Keeper 0.1 |
+|---|---|
+| All 746 messages | 80% |
+| Risky messages caught | 84% |
+| Safe messages flagged by mistake | 12% |
+| Pidgin | 87% |
+| Igbo | 81% |
+| Yoruba | 78% |
+| English | 76% |
+| Hausa | 74% |
+| Real AfriHate tweets | 74% |
+
+For comparison, the phrase word list that Lighthouse used before scores 37% on the same test and catches 22% of risky messages. The teacher model itself scores 85%.
+
+The 557 messages we wrote are published as a test set: [Prosperitytech/keeper-nigerian-safety-test](https://huggingface.co/datasets/Prosperitytech/keeper-nigerian-safety-test). Keeper gets 84% on that part. The tweets and the OpenAI pack are not ours to republish.
+
+## Where it falls short
+
+- **False alarms.** Look across all 13 categories and Keeper flags something in about 28% of ordinary safe messages. Most of those are at level low. With a 0.8 cutoff that drops to about 14%, and it then catches about 77% of what the teacher calls risky. Pick the cutoff for your use.
+- **It reacts to the style of AI-written text.** Many of the new Yoruba, Hausa and Igbo examples were written by a model, and Keeper partly learned that style as a risk signal. Real Yoruba and Hausa chat may score lower than our test suggests. Fixing this is the main goal for the next version.
+- **The test is small.** About 100 messages per African language. A single score can move by 5 to 10 points from luck.
+- **Hausa and Igbo are behind Pidgin and English.**
+- **Text only.** It does not look at images, voice notes or links.
+- **One message at a time.** Grooming and bullying often only show up across a whole conversation.
+
+Keeper is meant to help a parent notice something. It should not make decisions about a child on its own.
+
+## Data we trained on
+
+All of it allows commercial use. Thank you to everyone who made these public.
+
+| Dataset | License |
+|---|---|
+| [AfriHate](https://huggingface.co/datasets/afrihate/afrihate) | Apache 2.0 |
+| [HausaHate](https://huggingface.co/datasets/franciellevargas/HausaHate) | Apache 2.0 |
+| [BRIGHTER emotion](https://huggingface.co/datasets/brighter-dataset/BRIGHTER-emotion-categories) | CC BY 4.0 |
+| [Nigerian hate speech (Mendeley)](https://data.mendeley.com/datasets/r5jsynhxsx/1) | CC BY 4.0 |
+| [Nigerian Twitter cyberbullying (Mendeley)](https://data.mendeley.com/datasets/5d3y8v68sd/1) | CC BY 4.0 |
+| [Hausa English code switched (Mendeley)](https://data.mendeley.com/datasets/3xjyjsf4sb/1) | CC BY 4.0 |
+| [SMS phishing (Mendeley)](https://data.mendeley.com/datasets/f45bkkt8pr) | CC BY 4.0 |
+| [NaijaScamBench](https://huggingface.co/datasets/Eterniti/NaijaScamBench) | CC BY 4.0 |
+| [Africa smishing SMS](https://huggingface.co/datasets/electricsheepafrica/africa-smishing-sms-phishing) | MIT |
+| [NaijaWeb](https://huggingface.co/datasets/saheedniyi/naijaweb) | MIT |
+| [FineWeb 2](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2) | ODC-BY |
+| [Civil Comments](https://huggingface.co/datasets/google/civil_comments) | CC0 |
+| [Measuring Hate Speech](https://huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech) | CC BY 4.0 |
+| [Aegis 2.0](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) | CC BY 4.0 |
+| [Nemotron Safety Guard v3](https://huggingface.co/datasets/nvidia/Nemotron-Safety-Guard-Dataset-v3) | CC BY 4.0 |
+| [WildGuardMix](https://huggingface.co/datasets/allenai/wildguardmix) | ODC-BY |
+| [XGuard Train Open](https://huggingface.co/datasets/Alibaba-AAIG/XGuard-Train-Open-200K) | Apache 2.0 |
+| [Aya red teaming](https://huggingface.co/datasets/CohereLabs/aya_redteaming) | Apache 2.0 |
+| [OpenPII](https://huggingface.co/datasets/ai4privacy/pii-masking-openpii-1.5m) | CC BY 4.0 |
+| [Scam dialogue](https://huggingface.co/datasets/BothBosu/scam-dialogue) | Apache 2.0 |
+| [SMS Spam Collection](https://archive.ics.uci.edu/dataset/228) | CC BY 4.0 |
+
+Models used to build it: gpt-oss-safeguard-20b and gpt-oss-120b from OpenAI (Apache 2.0), and AfriqueQwen 9B from McGill NLP (CC BY 4.0).
+
+We are not publishing the training set itself. Parts of it are written to look like grooming and sextortion, and we don't want that text floating around on its own.
+
+## License
+
+Apache 2.0. You can use Keeper in your own products, paid ones included. Please credit Prosperity Tech and link back here.
+
+## Contact
+
+Prosperity Tech, makers of Lighthouse. Open an issue on this page or reach us through [lighthouse](https://github.com/prosperitytechoss/lighthouse) on GitHub. If Keeper gets something wrong in your language, we want to hear about it.

config.json

@@ -0,0 +1,45 @@
+{
+  "name": "lighthouse-keeper",
+  "version": "0.1",
+  "parameters": 8330000,
+  "max_tokens": 128,
+  "bos_id": 2,
+  "pad_id": 0,
+  "categories": [
+    "bullying",
+    "eating-disorders",
+    "gambling",
+    "graphic-content",
+    "grooming",
+    "hate-speech",
+    "privacy-threats",
+    "scams",
+    "self-harm",
+    "sextortion",
+    "sexual-content",
+    "substance-use",
+    "violence"
+  ],
+  "levels": [
+    "none",
+    "low",
+    "review",
+    "high"
+  ],
+  "suggested_min_risk": 0.8,
+  "tie_break_order": [
+    "self-harm",
+    "sextortion",
+    "grooming",
+    "sexual-content",
+    "violence",
+    "graphic-content",
+    "eating-disorders",
+    "substance-use",
+    "hate-speech",
+    "scams",
+    "privacy-threats",
+    "gambling",
+    "bullying"
+  ]
+}

example.py

@@ -0,0 +1,36 @@
+import json
+import sys
+
+import numpy as np
+import onnxruntime as ort
+import sentencepiece as spm
+from huggingface_hub import hf_hub_download
+
+REPO = "Prosperitytech/lighthouse-keeper"
+
+cfg = json.load(open(hf_hub_download(REPO, "config.json")))
+sp = spm.SentencePieceProcessor(model_file=hf_hub_download(REPO, "keeper.spm"))
+session = ort.InferenceSession(hf_hub_download(REPO, "keeper.onnx"), providers=["CPUExecutionProvider"])
+
+
+def score(text):
+    ids = [cfg["bos_id"]] + sp.encode(text)[: cfg["max_tokens"] - 1]
+    probs = session.run(None, {"input_ids": np.array([ids], dtype=np.int64)})[0][0]
+    return {cat: dict(zip(cfg["levels"], map(float, p))) for cat, p in zip(cfg["categories"], probs)}
+
+
+def flag(text, min_risk=cfg["suggested_min_risk"]):
+    priority = cfg["tie_break_order"]
+    best = None
+    for cat, p in score(text).items():
+        level = max(p, key=p.get)
+        risk = 1 - p["none"]
+        if level != "none" and risk >= min_risk:
+            key = (cfg["levels"].index(level), p[level] + 0.06 * (1 - priority.index(cat) / len(priority)))
+            if best is None or key > best[1]:
+                best = (cat, key, p[level])
+    return None if best is None else {"category": best[0], "level": cfg["levels"][best[1][0]], "confidence": round(best[2], 3)}
+
+
+if __name__ == "__main__":
+    print(flag(" ".join(sys.argv[1:]) or "send me your BVN make I help you collect the money"))

keeper-fp32.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aaddc7c67d9eb09eaf4a442605717c9d4e96c8a0caf93c44856b20f2caf57c0a
+size 33357151

keeper.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f290ae65968cd380ce09e075ee2931d0a0230cd007d3493492bbf4a1ffc9d78f
+size 8465401

keeper.spm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9becb67f3537c7cea9674216493f0b2de1916c2007382b9d2c8e5a46b22c41e
+size 588662