Create Model Card

README.md

@@ -0,0 +1,79 @@
+---
+license: apache-2.0
+language: multilingual
+tags:
+- privacy
+- pii-detection
+- text-classification
+- xlm-roberta
+library_name: transformers
+base_model: FacebookAI/xlm-roberta-large
+---
+
+## Roblox PII Classifier
+
+# Roblox PII Classifier v2.0 — Model Card
+
+## Overview
+We present `Roblox/roblox-pii-classifier-v2`, an upgraded, context-aware PII (Personally Identifiable Information) detection model. Built on the [XLM-RoBERTa-Large](https://huggingface.co/xlm-roberta-large) architecture, v2.0 is designed to identify attempts to share or solicit personal information across complex, multi-user conversations. 
+
+Unlike roblox-pii-classifier v1, **v2.0 evaluates text within its surrounding conversational context**, allowing it to detect sophisticated, adversarial bypass attempts (e.g., collaborative PII exchange, creative spelling, character substitution, and implicit references) while drastically reducing false positives in casual conversation.
+
+## What’s New in v2.0?
+
+* **Conversational Context Integration:** Evaluates target messages within the broader multi-user chat context, eliminating ambiguity inherent in isolated messages.
+* **RAG-Augmented LLM Labeling:** Primarily trained on data labeled by LLMs leveraging Retrieval-Augmented Generation (RAG) on a curated corpus of labeled samples, outperforming human expert baselines in both precision and recall on complex PII cases.
+* **Massive Multilingual Expansion:** Broadened synthetic data generation and fine-tuning capabilities to expand language coverage from **17 to 189 languages**.
+* **Automated Red-Teaming & Clustering:** Applied automated adversarial red-teaming and cluster visualization to systematically surface, isolate, and patch model vulnerabilities.
+* **Significant Performance Leap:** Overall F1 score increased from **63.41% to 90.52%**, with Recall at 1% False Positive Rate (FPR) jumping from **18.51% to 87.10%**.
+
+## Model Architecture & Classification Scope
+
+### Input & Output Specs
+* **Input Format:** Text input incorporating the target message alongside surrounding multi-turn chat history (up to a max sequence length of 512 tokens).
+* **Outputs:** Uncalibrated multi-label prediction scores for PII intent.
+For per-category cutoffs we recommend privacy_asking_for_pii >= 0.6 and privacy_giving_pii >= 0.5, and directing_users_off_platform >= 0.06.
+
+### Class Labels
+The model performs multi-label classification across three PII categories:
+
+1. **`PRIVACY_ASKING_FOR_PII`**: Attempting to obtain personal identifying information through direct queries, stealth tactics, or subtle insinuations.
+2. **`PRIVACY_GIVING_PII`**: Sharing or threatening to share PII—including phone numbers, email addresses, government IDs, social media handles, and credentials—or attempting to direct a user off-platform (DUOP) to external apps or real-world locations.
+
+---
+
+## Training & Evaluation Data Pipeline
+
+1. **Contextual Chat Data:** Training sets utilize anonymized internal chat logs formatted with preceding conversational context.
+2. **RAG-Enhanced LLM Annotations:** Synthetic and real chat datasets were labeled using an advanced LLM pipeline augmented with RAG to reference policy-specific edge cases.
+3. **Adversarial Synthetic Data:** Target generation specifically tailored toward masking, leetspeak, split-message bypasses, and low-resource language edge cases.
+4. **Vulnerability Patching:** Automated red-teaming cycles identified weak clusters, triggering targeted synthetic data injections to patch blind spots.
+
+---
+
+## Benchmark Comparisons
+
+The table below illustrates performance across internal and open-source evaluation benchmarks:
+
+| Dataset / Benchmark | PII v2.0 | PII v1.1 | OpenAI Privacy Filter | GLiNER2 | Qwen3Guard | LlamaGuard v3 1B | LlamaGuard v3 8B | LlamaGuard v4 12B | NemoGuard 8B | Piiranha NER |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **PII v2 Eval Synthetic/Redteam** | **86.04%** | 64.21% | 54.36% | 49.90% | 41.77% | 26.37% | 46.89% | 41.15% | 38.83% | 55.45% |
+| **PII v2 Eval (Primary)** | **90.52%** | 63.41% | 20.98% | 28.21% | 5.17% | 3.96% | 10.38% | 12.81% | 1.31% | 20.98% |
+| **Nemotron-PII** | **99.22%** | 70.07% | 56.10% | 62.90% | 37.50% | 53.32% | 46.10% | 37.50% | 56.20% | 68.50% |
+| **PII Masking OpenPII 1.5M** | **99.76%** | 86.79% | 84.23% | 83.07% | 61.94% | 65.10% | 64.23% | 56.16% | 54.02% | 86.35% |
+| **PII Masking 400k** | **97.64%** | 97.44% | 72.66% | 72.28% | 48.83% | 52.41% | 49.64% | 42.48% | 49.82% | 97.09% |
+| **Beavertails** | 63.24% | 60.54% | 59.36% | 27.83% | 66.74% | 64.03% | 64.95% | **68.65%** | 65.45% | 61.76% |
+
+
+## Usage
+The dependencies for the inference file can be installed as follows:
+```
+pip install -r requirements.txt
+```
+The provided Python file demonstrates how to use the classifier with text input.
+To run the inference, please run the following command:
+```
+python inference.py --input_file <your text file path> --model_path <path to Huggingface model>
+```
+
+If `model_path` isn't specified, the model will be loaded directly from HuggingFace.