xbian-rblx commited on
Commit
5996110
·
1 Parent(s): 2d32af4

update the score cutoffs

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -18,7 +18,7 @@ The model classifies text into two PII-related categories in a multi-label fashi
18
  - **PRIVACY_ASKING_FOR_PII**: Attempting to obtain personal identifying information (PII) through direct questions or insinuation.
19
  - **PRIVACY_GIVING_PII**: Sharing or threatening to share someone's personal identifying information (PII), including but not limited to telephone numbers, email addresses, government ID numbers, social media handles, and account passwords/credentials. This category also includes attempts to direct a user off-platform to an external platform or real-world location (DUOP).
20
 
21
- The classifier expects text input with a maximum sequence length of 512 tokens. The outputs are uncalibrated scores. The recommended cutoff for pii detection is when the sum of both categories `max(privacy_asking_for_pii, privacy_giving_pii) >= 0.26911312`, which achives the optimal F1 score on Roblox English anonymized chat. For per-category cutoffs we recommend `privacy_asking_for_pii >= 0.2` and `privacy_giving_pii >= 0.3`.
22
 
23
  The table below displays evaluation metrics on internal held-out datasets and comparisons with other state-of-the-art models.
24
 
 
18
  - **PRIVACY_ASKING_FOR_PII**: Attempting to obtain personal identifying information (PII) through direct questions or insinuation.
19
  - **PRIVACY_GIVING_PII**: Sharing or threatening to share someone's personal identifying information (PII), including but not limited to telephone numbers, email addresses, government ID numbers, social media handles, and account passwords/credentials. This category also includes attempts to direct a user off-platform to an external platform or real-world location (DUOP).
20
 
21
+ The classifier expects text input with a maximum sequence length of 512 tokens. The outputs are uncalibrated scores. The recommended cutoff for pii detection is when the sum of both categories `max(privacy_asking_for_pii, privacy_giving_pii) >= 0.2691`, which achives the optimal F1 score on Roblox English anonymized chat. For per-category cutoffs we recommend `privacy_asking_for_pii >= 0.2` and `privacy_giving_pii >= 0.3`.
22
 
23
  The table below displays evaluation metrics on internal held-out datasets and comparisons with other state-of-the-art models.
24