Upload README.md with huggingface_hub

README.md

@@ -1,199 +1,153 @@
 ---
-library_name: transformers
-tags: []
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
+language:
+- multilingual
+tags:
+- prompt-injection
+- toxicity-detection
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+base_model: jhu-clsp/mmBERT-base
+---
 
-## Technical Specifications [optional]
+# modernBERT – Prompt Injection + Toxicity Classifier (v3.5)
 
-### Model Architecture and Objective
+Fine-tuned from [**jhu-clsp/mmBERT-base**](https://huggingface.co/jhu-clsp/mmBERT-base) for **2-head prompt-injection and toxicity detection**.
 
-[More Information Needed]
+This model outputs two scores: `prompt_injection` (index 0) and `toxic` (index 1). A **tiered detection strategy** combines both heads to achieve higher recall than a single PI threshold alone.
 
-### Compute Infrastructure
+**Usage:**  
+For a single text input, tokenize and split into overlapping chunks of ≤512 tokens (overlap=200, stride=312), run them in a batch, and take the **maximum logit across chunks** per head before applying sigmoid. Apply the tiered rule to the resulting PI and toxic probabilities.
 
-[More Information Needed]
+---
 
-#### Hardware
+## Tiered Detection Strategy
 
-[More Information Needed]
+```
+flag = (pi >= pi_thresh) OR (pi >= pi_lower_bound AND toxic >= toxic_thresh)
+```
 
-#### Software
+### Thresholds at 0.5% FPR
 
-[More Information Needed]
+| Parameter | Value |
+|:----------|------:|
+| `pi_thresh` | 0.990000 |
+| `pi_lower_bound` | 0.50 |
+| `toxic_thresh` | 0.95 |
 
-## Citation [optional]
+| Dataset | Recall | FPR |
+|:--------|-------:|----:|
+| test (262K) | 69.83% | 0.644% |
+| customer_test (1.4M) | 69.83% | 2.977% |
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+### Thresholds at 1% FPR
 
-**BibTeX:**
+| Parameter | Value |
+|:----------|------:|
+| `pi_thresh` | 0.981736 |
+| `pi_lower_bound` | 0.50 |
+| `toxic_thresh` | 0.90 |
 
-[More Information Needed]
+| Dataset | Recall | FPR |
+|:--------|-------:|----:|
+| test (262K) | 75.26% | 1.008% |
+| customer_test (1.4M) | 76.30% | 3.271% |
 
-**APA:**
+---
 
-[More Information Needed]
+## Evaluation Data
 
-## Glossary [optional]
+The datasets used to compute the metrics above are available on S3:
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+| Dataset | S3 URI |
+|:--------|:-------|
+| **test** (262K) | `s3://cisco-sbg-ai-nonprod-45f676d4/voyager/data/pi_modeling/v5/dataset/test_raw/` |
+| **customer_test** (1.4M) | `s3://cisco-sbg-ai-nonprod-45f676d4/voyager/data/pi_modeling/v5/dataset/customer_test_raw/` |
 
-[More Information Needed]
+Download the parquet data, tokenize, and run inference using the code snippet below.
 
-## More Information [optional]
+---
 
-[More Information Needed]
+## W&B Model Comparison
 
-## Model Card Authors [optional]
+Interactive ROC curves and recall/FPR tables comparing **pi-mmbert-v2** and **pi-mmbert-v3.5**:
 
-[More Information Needed]
+🔗 [**W&B Report: pi-model-comparison**](https://cisco-sbgai.wandb.io/cisco-sbg-ai-nonprod/pi-model-comparison?nw=nwuserkarthkal)
 
-## Model Card Contact
+---
 
-[More Information Needed]
+## 🚀 Example Usage
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+# --- Load model and tokenizer ---
+model_name = "robustintelligence/pi-mmbert-v3.5"
+model = AutoModelForSequenceClassification.from_pretrained(model_name, trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+
+# --- Inference parameters ---
+max_length = 512
+chunk_overlap = 200
+stride = max_length - chunk_overlap  # 312
+
+# --- Tiered thresholds (0.5% FPR) ---
+pi_thresh = 0.990
+pi_lower_bound = 0.5
+toxic_thresh = 0.95
+# --- Tiered thresholds (1% FPR) ---
+# pi_thresh = 0.9817
+# pi_lower_bound = 0.5
+# toxic_thresh = 0.90
+
+# --- Example input ---
+text = "This is a user message that may or may not contain prompt injection."
+
+encoded = tokenizer(
+    text,
+    add_special_tokens=True,
+    truncation=False,
+)
+input_ids = encoded["input_ids"]
+
+# --- Split into overlapping chunks ---
+if len(input_ids) <= max_length:
+    chunks = [input_ids]
+else:
+    chunks = []
+    for start in range(0, len(input_ids), stride):
+        end = min(start + max_length, len(input_ids))
+        chunks.append(input_ids[start:end])
+        if end == len(input_ids):
+            break
+
+# --- Pad and stack ---
+input_tensors = [torch.tensor(chunk, dtype=torch.long) for chunk in chunks]
+attention_masks = [torch.ones_like(t) for t in input_tensors]
+input_ids_batch = torch.nn.utils.rnn.pad_sequence(input_tensors, batch_first=True, padding_value=0)
+attention_mask_batch = torch.nn.utils.rnn.pad_sequence(attention_masks, batch_first=True, padding_value=0)
+
+# --- Run inference ---
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model = model.to(device)
+model.eval()
+
+with torch.no_grad(), torch.autocast(device_type=device, dtype=torch.bfloat16):
+    logits = model(
+        input_ids=input_ids_batch.to(device),
+        attention_mask=attention_mask_batch.to(device),
+    ).logits  # [num_chunks, 2]
+
+# --- Aggregate: max logit across chunks, then sigmoid ---
+max_logits = logits.max(dim=0).values  # [2]
+probs = torch.sigmoid(max_logits)
+
+pi_prob = probs[0].item()
+toxic_prob = probs[1].item()
+
+# --- Apply tiered detection rule ---
+is_flagged = (pi_prob >= pi_thresh) or (pi_prob >= pi_lower_bound and toxic_prob >= toxic_thresh)
+
+print(f"PI probability:    {pi_prob:.4f}")
+print(f"Toxic probability: {toxic_prob:.4f}")
+print(f"Prompt injection detected? {'FLAG' if is_flagged else 'ALLOW'}")
+```