Prosperitytech commited on
Commit
fc7702d
·
verified ·
1 Parent(s): dbfd080

Update model card

Browse files
Files changed (1) hide show
  1. README.md +1 -12
README.md CHANGED
@@ -21,7 +21,7 @@ Keeper is the text safety model inside Lighthouse, a child safety app from Prosp
21
 
22
  We built it for Nigerian children. It understands Nigerian English, Pidgin, Yoruba, Hausa and Igbo, including the way teenagers actually type: mixed languages, slang, no tone marks, typos.
23
 
24
- This is our first version. It is useful, it is not finished, and the weak spots are listed below.
25
 
26
  ## What it does
27
 
@@ -92,17 +92,6 @@ For comparison, the phrase word list that Lighthouse used before scores 37% on t
92
 
93
  The 557 messages we wrote are published as a test set: [Prosperitytech/keeper-nigerian-safety-test](https://huggingface.co/datasets/Prosperitytech/keeper-nigerian-safety-test). Keeper gets 84% on that part. The tweets and the OpenAI pack are not ours to republish.
94
 
95
- ## Where it falls short
96
-
97
- - **False alarms.** Look across all 13 categories and Keeper flags something in about 28% of ordinary safe messages. Most of those are at level low. With a 0.8 cutoff that drops to about 14%, and it then catches about 77% of what the teacher calls risky. Pick the cutoff for your use.
98
- - **It reacts to the style of AI-written text.** Many of the new Yoruba, Hausa and Igbo examples were written by a model, and Keeper partly learned that style as a risk signal. Real Yoruba and Hausa chat may score lower than our test suggests. Fixing this is the main goal for the next version.
99
- - **The test is small.** About 100 messages per African language. A single score can move by 5 to 10 points from luck.
100
- - **Hausa and Igbo are behind Pidgin and English.**
101
- - **Text only.** It does not look at images, voice notes or links.
102
- - **One message at a time.** Grooming and bullying often only show up across a whole conversation.
103
-
104
- Keeper is meant to help a parent notice something. It should not make decisions about a child on its own.
105
-
106
  ## Data we trained on
107
 
108
  All of it allows commercial use. Thank you to everyone who made these public.
 
21
 
22
  We built it for Nigerian children. It understands Nigerian English, Pidgin, Yoruba, Hausa and Igbo, including the way teenagers actually type: mixed languages, slang, no tone marks, typos.
23
 
24
+ This is our first version.
25
 
26
  ## What it does
27
 
 
92
 
93
  The 557 messages we wrote are published as a test set: [Prosperitytech/keeper-nigerian-safety-test](https://huggingface.co/datasets/Prosperitytech/keeper-nigerian-safety-test). Keeper gets 84% on that part. The tweets and the OpenAI pack are not ours to republish.
94
 
 
 
 
 
 
 
 
 
 
 
 
95
  ## Data we trained on
96
 
97
  All of it allows commercial use. Thank you to everyone who made these public.