Update model card
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ Keeper is the text safety model inside Lighthouse, a child safety app from Prosp
|
|
| 21 |
|
| 22 |
We built it for Nigerian children. It understands Nigerian English, Pidgin, Yoruba, Hausa and Igbo, including the way teenagers actually type: mixed languages, slang, no tone marks, typos.
|
| 23 |
|
| 24 |
-
This is our first version.
|
| 25 |
|
| 26 |
## What it does
|
| 27 |
|
|
@@ -92,17 +92,6 @@ For comparison, the phrase word list that Lighthouse used before scores 37% on t
|
|
| 92 |
|
| 93 |
The 557 messages we wrote are published as a test set: [Prosperitytech/keeper-nigerian-safety-test](https://huggingface.co/datasets/Prosperitytech/keeper-nigerian-safety-test). Keeper gets 84% on that part. The tweets and the OpenAI pack are not ours to republish.
|
| 94 |
|
| 95 |
-
## Where it falls short
|
| 96 |
-
|
| 97 |
-
- **False alarms.** Look across all 13 categories and Keeper flags something in about 28% of ordinary safe messages. Most of those are at level low. With a 0.8 cutoff that drops to about 14%, and it then catches about 77% of what the teacher calls risky. Pick the cutoff for your use.
|
| 98 |
-
- **It reacts to the style of AI-written text.** Many of the new Yoruba, Hausa and Igbo examples were written by a model, and Keeper partly learned that style as a risk signal. Real Yoruba and Hausa chat may score lower than our test suggests. Fixing this is the main goal for the next version.
|
| 99 |
-
- **The test is small.** About 100 messages per African language. A single score can move by 5 to 10 points from luck.
|
| 100 |
-
- **Hausa and Igbo are behind Pidgin and English.**
|
| 101 |
-
- **Text only.** It does not look at images, voice notes or links.
|
| 102 |
-
- **One message at a time.** Grooming and bullying often only show up across a whole conversation.
|
| 103 |
-
|
| 104 |
-
Keeper is meant to help a parent notice something. It should not make decisions about a child on its own.
|
| 105 |
-
|
| 106 |
## Data we trained on
|
| 107 |
|
| 108 |
All of it allows commercial use. Thank you to everyone who made these public.
|
|
|
|
| 21 |
|
| 22 |
We built it for Nigerian children. It understands Nigerian English, Pidgin, Yoruba, Hausa and Igbo, including the way teenagers actually type: mixed languages, slang, no tone marks, typos.
|
| 23 |
|
| 24 |
+
This is our first version.
|
| 25 |
|
| 26 |
## What it does
|
| 27 |
|
|
|
|
| 92 |
|
| 93 |
The 557 messages we wrote are published as a test set: [Prosperitytech/keeper-nigerian-safety-test](https://huggingface.co/datasets/Prosperitytech/keeper-nigerian-safety-test). Keeper gets 84% on that part. The tweets and the OpenAI pack are not ours to republish.
|
| 94 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
## Data we trained on
|
| 96 |
|
| 97 |
All of it allows commercial use. Thank you to everyone who made these public.
|