Update model card for v2-guardrail-clean

README.md

@@ -15,18 +15,44 @@ pipeline_tag: text-classification
 
 # ReframeBot-Guardrail-DistilBERT
 
-A 3-class text classifier fine-tuned from `distilbert-base-uncased` that
-routes conversation turns to one of three task modes:
+A 3-class DistilBERT classifier for routing ReframeBot user turns:
 
 | Label | Meaning |
 |---|---|
-| `TASK_1` | CBT / academic stress — engage with Socratic questioning |
-| `TASK_2` | Crisis / self-harm signal — redirect to emergency hotlines |
-| `TASK_3` | Out-of-scope — validate feeling, pivot back to academics |
+| `TASK_1` | CBT / academic stress |
+| `TASK_2` | Crisis / self-harm signal |
+| `TASK_3` | Out-of-scope |
 
-This model is the guardrail component of the ReframeBot system. Its output
-is combined with a dual-signal crisis detector (regex + cosine similarity)
-before the final routing decision is made.
+This version was retrained on `data/guardrail_dataset_clean.jsonl`, which
+merges the original guardrail data with curated hard cases for CBT/Crisis
+boundaries, Vietnamese text, pills/overdose language, and OOS work/mental
+health informational prompts.
+
+## Current System Threshold
+
+The ReframeBot runtime uses the classifier's full probability vector and
+routes to `TASK_2` when:
+
+```text
+P(TASK_2) >= 0.10
+```
+
+after academic-context/follow-up overrides and after the regex + semantic
+crisis detector has already run.
+
+## Evaluation
+
+Hard out-of-domain eval set (`data/evaluation_test_data.json`, 60 samples):
+
+| Mode | Accuracy | TASK_2 Precision | TASK_2 Recall | TASK_2 F1 |
+|---|---:|---:|---:|---:|
+| Argmax only | 0.9667 | 1.0000 | 0.9048 | 0.9500 |
+| Tuned `P(TASK_2) >= 0.10` | 0.9833 | 0.9545 | 1.0000 | 0.9767 |
+
+Threshold sweep artifact in the project repo:
+
+- `reports/guardrail_threshold_sweep.csv`
+- `reports/guardrail_threshold_sweep.png`
 
 ## Usage
 
@@ -36,54 +62,20 @@ from transformers import pipeline
 classifier = pipeline(
     "text-classification",
     model="Nhatminh1234/ReframeBot-Guardrail-DistilBERT",
+    revision="v2-guardrail-clean",
 )
 
-classifier("I'm really stressed about my finals next week")
-# [{'label': 'TASK_1', 'score': 0.97}]
-
-classifier("What's a good recipe for pasta?")
-# [{'label': 'TASK_3', 'score': 0.94}]
+classifier("I'm stressed about my final exam")
 ```
 
-## Training Details
-
-| Hyperparameter | Value |
-|---|---|
-| Base model | distilbert-base-uncased |
-| Number of labels | 3 (TASK_1, TASK_2, TASK_3) |
-| Learning rate | 2e-6 |
-| Batch size | 16 |
-| Max epochs | 20 (early stopping, patience=3) |
-| Weight decay | 0.01 |
-| Max token length | 128 |
-| Best model criterion | macro F1 |
-| Hardware | NVIDIA RTX 5070 (laptop, 8 GB VRAM) |
-
-**Dataset:** 1,674 labelled samples (80/20 train/val split). Includes hard
-negatives — benign metaphors that superficially resemble crisis language
-(e.g., "dying of embarrassment after that presentation").
-
-## Evaluation
-
-Per-class results on the validation split (335 samples):
+For full class probabilities:
 
-| Class | Precision | Recall | F1 | Support |
-|---|---|---|---|---|
-| TASK_1 | 0.99 | 1.00 | 1.00 | 107 |
-| TASK_2 | 0.98 | 1.00 | 0.99 | 91 |
-| TASK_3 | 1.00 | 0.98 | 0.99 | 137 |
-| **macro avg** | **0.99** | **0.99** | **0.99** | **335** |
-
-Accuracy on a separate, harder held-out test set: **88.3%** (includes
-boundary cases not present in the training distribution).
-
-## Intended Use
-
-Designed as a routing component in the ReframeBot system. The TASK_2
-output alone is not sufficient for crisis intervention — the full system
-also applies a regex + semantic similarity layer before acting on a
-crisis signal.
+```python
+classifier("I bought pills to overdose", top_k=None)
+```
 
-## Project
+## Safety Note
 
-GitHub: [ReframeBot](https://github.com/minhnghiem32131024429/ReframeBot)
+This classifier is a routing component, not a standalone crisis intervention
+system. ReframeBot also uses regex + semantic crisis detection and crisis
+response handling around this model.