Update README.md

README.md

@@ -11,9 +11,7 @@ base_model: FacebookAI/xlm-roberta-large
 datasets:
 - Roblox/roblox-pii-safety-for-chat-benchmark
 - ai4privacy/pii-masking-openpii-1.5m
-- ai4privacy/pii-masking-health-phi-400k
 - nvidia/Nemotron-PII
-- PKU-Alignment/BeaverTails
 ---
 
 # Roblox PII Classifier v2.0 — Model Card
@@ -29,7 +27,7 @@ Unlike its predecessor and other NER models, **v2 evaluates text within its surr
 * **RAG-Augmented LLM Labeling:** Primarily trained on data labeled by LLMs leveraging Retrieval-Augmented Generation (RAG) on a curated corpus of labeled samples, outperforming human expert baselines in both precision and recall on complex PII cases.
 * **Massive Multilingual Expansion:** Broadened synthetic data generation and fine-tuning capabilities to expand language coverage from **17 to 189 languages**.
 * **Automated Red-Teaming & Clustering:** Applied automated adversarial red-teaming and cluster visualization to systematically surface, isolate, and patch model vulnerabilities.
-* **Significant Performance Leap:** Overall F1 score increased from **63.41% to 90.52%**, with Recall at 1% False Positive Rate (FPR) jumping from **18.51% to 87.10%** on our refreshed internal evaluation set.
+* **Significant Performance Leap:** Overall F1 score increased from **63.41% to 90.52%** on our refreshed internal evaluation set, with a f1 score increase from **64.74% to 88.78%** on the recently released roblox-pii-safety-for-chat-benchmark.
 
 ## Model Architecture & Classification Scope
 
@@ -66,13 +64,12 @@ emitted in this fixed index order:
 
 The table below illustrates best f1 performance across internal and open-source evaluation benchmarks (full conversational data used when possible):
 
-| Dataset / Benchmark | Roblox PII v2 | *Roblox PII v1 | *OpenAI Privacy Filter | *GLiNER2 | Qwen3Guard | LlamaGuard v3 1B | LlamaGuard v3 8B | LlamaGuard v4 12B | NemoGuard 8B | *Piiranha NER | Shieldstral
+| Dataset / Benchmark | Roblox PII v2 | *Roblox PII v1 | *OpenAI Privacy Filter | *GLiNER2 | Qwen3Guard Gen 8B | LlamaGuard v3 1B | LlamaGuard v3 8B | LlamaGuard v4 12B | NemoGuard 8B | *Piiranha NER | Shieldstral
 | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 | **roblox-pii-safety-for-chat** | **88.78%** | 64.74% | 58.10% | 54.42% | 66.15% | 27.80% | 56.06% | 54.30% | 56.35% | 58.34% | 57.66 |
 | **Roblox Internal Evaluation** | **90.52%** | 63.41% | 20.98% | 28.21% | 15.62% | 8.59% | 12.90% | 21.92% | 14.77% | 20.98% | 31.87 |
 | **Nemotron-PII** | **99.22%** | 70.07% | 65.22% | 62.92% | 37.46% | 53.32% | 46.07% | 35.05% | 56.16% | 68.52% | 56.69 |
 | **PII Masking OpenPII 1.5M** | **99.76%** | 86.79% | 84.23% | 83.07% | 61.94% | 65.10% | 64.23% | 56.16% | 54.01% | 86.35% | 85.33 |
-| **PII Masking 400k** | **97.64%** | 97.44% | 72.66% | 72.28% | 48.83% | 52.41% | 49.64% | 42.48% | 49.82% | 97.09% | 67.13 |
 
 \* These models all were supplied with only the target-user text as other speaker texts are meant to be supplemental contextual information. roblox-pii-classifier-v2 performs best when supplied with conversational context, however it still outperforms roblox-pii-classifier when supplied only with target-user text.