| --- |
| license: mit |
| language: en |
| tags: [cvss, cve, vulnerability, security, text-classification] |
| datasets: [nvd] |
| base_model: microsoft/deberta-v3-base |
| --- |
| |
| # CVEscore: CVSS v3.1 vector prediction from CVE descriptions |
|
|
| Fine-tuned `microsoft/deberta-v3-base` with 8 classification heads (AV, AC, PR, UI, S, C, I, A). |
| The base score and severity are computed from the predicted vector with the official CVSS v3.1 formula. |
|
|
| ## Input format |
| Prefix the description with who will score the CVE, as in training: |
|
|
| - `[NVD] <description>` predicts the vector NVD would give. |
| - `[CNA: <org>] <description>` predicts the vector that CNA would give. `<org>` is the domain of the CNA's |
| email without the TLD (`security@wordfence.com` -> `wordfence`), or `cisa-adp` for CISA's ADP. |
|
|
| Remove CVSS vectors and scores quoted in the description first (`strip_cvss` in `src/cvss_text.py`): |
| the model was trained without them. |
|
|
| With every CVE prompted as `[NVD]`, exact match is 0.393 overall (0.552 on NVD-scored CVEs). |
| Tagging each CVE with its real scorer raises CNA-scored exact match from 0.339 to 0.438. |
| Ablation: https://github.com/atomic-crafter/CVEscore/blob/master/results/ablation.md |
|
|
| ## Training data and split |
| NVD CVE JSON 2.0 feeds, CVEs with a CVSS v3.1 vector (NVD score preferred, CNA score otherwise). |
| **Time-based split**: train on CVEs published before 2024-10-01, validate up to the end of 2024, |
| test on CVEs published from 2025 onwards. No random split, so no leakage from the future. |
|
|
| ## Test results (123,459 CVEs published in 2025+, each tagged with its scorer) |
| | Metric | Value | |
| |---|---| |
| | Exact vector match | 0.467 | |
| | Severity accuracy | 0.691 | |
| | Base score MAE | 0.860 | |
| | Inference speed | 1.01 ms/CVE (RTX 5080, batched) | |
|
|
| Per-metric macro F1: AV 0.782, AC 0.707, PR 0.746, UI 0.903, S 0.851, C 0.819, I 0.819, A 0.796 |
|
|
| ## Intended use |
| Triage aid: pre-score new CVEs before official scoring is available, and route low-confidence |
| predictions (product of the 8 heads' top probabilities) to a human analyst. |
| Not a replacement for analyst scoring. |
|
|
| ## Limitations |
| - English descriptions only; vague descriptions ("unspecified vulnerability") are scored poorly. |
| - NVD and vendor (CNA) scores disagree on most CVEs scored by both (18% exact agreement in 2025+), |
| which caps achievable accuracy. |
| - CVSS v3.1 only (no v4.0). |
|
|
| ## Usage |
| Code, training pipeline and the `CVSSModel` class: https://github.com/atomic-crafter/CVEscore |
|
|
| ```python |
| from model import CVSSModel, predict_proba # src/model.py in the GitHub repo |
| model, tokenizer, config = CVSSModel.load("path/to/this/repo", device="cuda") |
| ``` |
|
|
