Update README.md
Browse files
README.md
CHANGED
|
@@ -25,7 +25,14 @@ Detects **blue-team / defensive security** text (English), with a focus on **tec
|
|
| 25 |
|
| 26 |
> **Recommended cut-off:** `prob >= 0.579` (arg-max on the validation split)
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
## Intended uses & limits
|
| 31 |
|
|
@@ -33,8 +40,6 @@ Detects **blue-team / defensive security** text (English), with a focus on **tec
|
|
| 33 |
* **Input language:** English
|
| 34 |
* **No external test set** yet → treat numbers as optimistic
|
| 35 |
|
| 36 |
-
---
|
| 37 |
-
|
| 38 |
## Training data
|
| 39 |
|
| 40 |
| Label | Rows |
|
|
@@ -44,8 +49,6 @@ Detects **blue-team / defensive security** text (English), with a focus on **tec
|
|
| 44 |
| Other | 130 000 |
|
| 45 |
| **Total** | **180 296** |
|
| 46 |
|
| 47 |
-
---
|
| 48 |
-
|
| 49 |
## Model details
|
| 50 |
|
| 51 |
| Field | Value |
|
|
@@ -56,8 +59,6 @@ Detects **blue-team / defensive security** text (English), with a focus on **tec
|
|
| 56 |
| Hardware | 1× RTX 4090 (≈ 41 min) |
|
| 57 |
| Inference dtype| FP16-safe |
|
| 58 |
|
| 59 |
-
---
|
| 60 |
-
|
| 61 |
## Training Data License
|
| 62 |
|
| 63 |
- **Source**: [trendmicro-ailab/Primus-FineWeb](https://huggingface.co/datasets/trendmicro-ailab/Primus-FineWeb)
|
|
@@ -66,21 +67,59 @@ Detects **blue-team / defensive security** text (English), with a focus on **tec
|
|
| 66 |
- Preserve all original copyright/license notices
|
| 67 |
- Honor [Common Crawl ToU](https://commoncrawl.org/terms-of-use/)
|
| 68 |
|
| 69 |
-
---
|
| 70 |
-
|
| 71 |
## Quick start
|
| 72 |
|
| 73 |
```python
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
> **Recommended cut-off:** `prob >= 0.579` (arg-max on the validation split)
|
| 27 |
|
| 28 |
+
## Demo
|
| 29 |
+
|
| 30 |
+
| Phrase | Blue Score |
|
| 31 |
+
|--------|------------|
|
| 32 |
+
| To exfiltrate sensitive data, launch a phishing campaign that tricks employees into revealing their VPN credentials. | 0.066 |
|
| 33 |
+
| We should deploy an EDR solution, monitor all endpoints for intrusion attempts, and enforce strict password policies. | 0.557 |
|
| 34 |
+
| Our marketing team will unveil the new cybersecurity branding materials at next Tuesday’s antivirus product launch | 0.256 |
|
| 35 |
+
| I'm excited about the company picnic. There's no cybersecurity topic—just burgers and games. | 0.272 |
|
| 36 |
|
| 37 |
## Intended uses & limits
|
| 38 |
|
|
|
|
| 40 |
* **Input language:** English
|
| 41 |
* **No external test set** yet → treat numbers as optimistic
|
| 42 |
|
|
|
|
|
|
|
| 43 |
## Training data
|
| 44 |
|
| 45 |
| Label | Rows |
|
|
|
|
| 49 |
| Other | 130 000 |
|
| 50 |
| **Total** | **180 296** |
|
| 51 |
|
|
|
|
|
|
|
| 52 |
## Model details
|
| 53 |
|
| 54 |
| Field | Value |
|
|
|
|
| 59 |
| Hardware | 1× RTX 4090 (≈ 41 min) |
|
| 60 |
| Inference dtype| FP16-safe |
|
| 61 |
|
|
|
|
|
|
|
| 62 |
## Training Data License
|
| 63 |
|
| 64 |
- **Source**: [trendmicro-ailab/Primus-FineWeb](https://huggingface.co/datasets/trendmicro-ailab/Primus-FineWeb)
|
|
|
|
| 67 |
- Preserve all original copyright/license notices
|
| 68 |
- Honor [Common Crawl ToU](https://commoncrawl.org/terms-of-use/)
|
| 69 |
|
|
|
|
|
|
|
| 70 |
## Quick start
|
| 71 |
|
| 72 |
```python
|
| 73 |
+
import torch
|
| 74 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 75 |
+
|
| 76 |
+
def classify_texts(model_name, phrases, threshold=0.515):
|
| 77 |
+
"""
|
| 78 |
+
Returns a list of (probability_offensive, label) tuples for each phrase
|
| 79 |
+
given a model_name and threshold.
|
| 80 |
+
"""
|
| 81 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 82 |
+
model = AutoModelForSequenceClassification.from_pretrained(model_name)
|
| 83 |
+
model.eval()
|
| 84 |
+
|
| 85 |
+
inputs = tokenizer(phrases, padding=True, truncation=True, return_tensors="pt")
|
| 86 |
+
|
| 87 |
+
with torch.no_grad():
|
| 88 |
+
logits = model(**inputs).logits # shape: (batch_size, 2)
|
| 89 |
+
probs_offensive = torch.softmax(logits, dim=1)[:, 1] # Probability of the "Offensive" class
|
| 90 |
+
|
| 91 |
+
results = []
|
| 92 |
+
for p_val in probs_offensive:
|
| 93 |
+
p_val = p_val.item()
|
| 94 |
+
label = "Offensive (red-team)" if p_val >= threshold else "Not Offensive"
|
| 95 |
+
results.append((p_val, label))
|
| 96 |
+
return results
|
| 97 |
+
|
| 98 |
+
def main():
|
| 99 |
+
# Example phrases: Offensive (red-team), Defensive (blue-team), Non-technical
|
| 100 |
+
phrases = [
|
| 101 |
+
# 1) Cybersecurity Offensive / red-team
|
| 102 |
+
"To exfiltrate sensitive data, launch a phishing campaign that tricks employees into revealing their VPN credentials.",
|
| 103 |
+
# 2) Cybersecurity Defensive / blue-team
|
| 104 |
+
"We should deploy an EDR solution, monitor all endpoints for intrusion attempts, and enforce strict password policies.",
|
| 105 |
+
# 5) Cybersecruity Marketing
|
| 106 |
+
"“Our marketing team will unveil the new cybersecurity branding materials at next Tuesday’s antivirus product launch",
|
| 107 |
+
# 5) Non Cybersecruity related
|
| 108 |
+
"I'm excited about the company picnic. There's no cybersecurity topic—just burgers and games."
|
| 109 |
+
]
|
| 110 |
+
|
| 111 |
+
# Classify with both models
|
| 112 |
+
threshold = 0.515
|
| 113 |
+
blue_results = classify_texts("HagalazAI/BlueSecureBERT", phrases, threshold)
|
| 114 |
+
red_results = classify_texts("HagalazAI/RedSecureBERT", phrases, threshold)
|
| 115 |
+
|
| 116 |
+
# Print a Markdown table
|
| 117 |
+
print("| # | Phrase | Blue Score | Blue Label | Red Score | Red Label |")
|
| 118 |
+
print("|---|--------|-----------|-----------|----------|----------|")
|
| 119 |
+
for i, text in enumerate(phrases, start=1):
|
| 120 |
+
blue_score, blue_label = blue_results[i - 1]
|
| 121 |
+
red_score, red_label = red_results[i - 1]
|
| 122 |
+
print(f"| {i} | {text} | {blue_score:.3f} | {blue_label} | {red_score:.3f} | {red_label} |")
|
| 123 |
+
|
| 124 |
+
if __name__ == "__main__":
|
| 125 |
+
main()
|