| # ModernBERT-IDS | |
| A fine-tuned ModernBERT-based multi-class classifier for detecting DDoS and related attack categories directly from raw network log strings. This model is part of the ModernBERT-DoS/IDS project, focused on high-accuracy intrusion detection using transformer-based architectures. | |
| --- | |
| ## Model Summary | |
| - **Model Type:** Transformer-based multi-class text classifier | |
| - **Base Model:** answerdotai/ModernBERT-base | |
| - **Task:** Intrusion Detection / Log Classification | |
| - **Input:** Raw network log text (e.g., packet captures converted to text format) | |
| - **Output:** Predicted attack class label | |
| - **Training Objective:** Weighted cross-entropy (handles class imbalance) | |
| This model was trained to identify several traffic types present in datasets such as CIC-DDoS2019 and custom SSL logs. | |
| --- | |
| ## Intended Use | |
| ### Primary Use Cases | |
| - Intrusion Detection Systems (IDS) | |
| - DDoS and attack pattern classification | |
| - Network monitoring and security research | |
| - Automated analysis of raw pcap-derived logs | |
| - Multi-class traffic categorization in SOC workflows | |
| ### Not Intended For | |
| - Real-time blocking without further validation | |
| - Use on unprocessed binary packet captures (requires conversion to text logs) | |
| - Detection of malware not represented in the training data | |
| --- | |
| ## Training Details | |
| ### Training Pipeline | |
| The model was trained using: | |
| - ModernBERT fine-tuning with mean pooling | |
| - 3× dropout layers for regularization | |
| - Two fully-connected layers with GELU activation | |
| - LayerNorm for stable optimization | |
| - Tokenization up to 512 tokens to support large logs | |
| ### Data Handling | |
| - Strict train/validation/test split with no data leakage | |
| - No manual feature removal required; model learns directly from raw logs | |
| - Stratified sampling to preserve class distribution | |
| ### Baselines (for comparison) | |
| The training repository includes benchmarks against: | |
| - Random Forest | |
| - Linear SVM | |
| - Logistic Regression | |
| - CNN | |
| - BiLSTM with Attention | |
| ModernBERT-IDS consistently outperformed all baselines in macro-F1 scoring. | |
| --- | |
| ## Evaluation Metrics | |
| The following metrics were computed on the test split (unseen logs): | |
| - Accuracy | |
| - Macro F1 | |
| - Weighted F1 | |
| - Per-class F1 | |
| - Precision/Recall | |
| - Confusion matrix | |
| This model achieved macro-F1 performance in the 0.95–0.97 range depending on dataset variation. | |
| --- | |
| ## Supported Attack Classes | |
| The model dynamically adapts to classes found in the training dataset, typically including: | |
| - DDoS | |
| - BENIGN | |
| - LDAP | |
| - NetBIOS | |
| - MSSQL | |
| - Portmap | |
| - UDP | |
| - SSL | |
| Additional classes may be present depending on the uploaded dataset. | |
| --- | |
| ## Example Usage | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification | |
| import torch | |
| model_name = "ccaug/modernbert-IDS" | |
| tokenizer = AutoTokenizer.from_pretrained(model_name) | |
| model = AutoModelForSequenceClassification.from_pretrained(model_name) | |
| log_line = "Frame 144: 98 bytes on wire (784 bits), 98 bytes captured ..." | |
| inputs = tokenizer(log_line, return_tensors="pt", truncation=True, max_length=512) | |
| with torch.no_grad(): | |
| outputs = model(**inputs) | |
| predicted_class = outputs.logits.argmax(dim=1).item() | |
| print("Predicted class:", predicted_class) |