End of training

README.md

@@ -1,67 +1,50 @@
 ---
 library_name: transformers
-license: cc-by-4.0
+license: mit
 base_model: roberta-base
-metrics:
-- accuracy
 tags:
 - generated_from_trainer
-- text-classification
-- classification
-- nlp
-- vulnerability
+metrics:
+- accuracy
 model-index:
 - name: vulnerability-severity-classification-roberta-base
   results: []
-datasets:
-- CIRCL/vulnerability-scores
 ---
 
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
 
-# VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification
-
-# Severity classification
-
-This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the dataset [CIRCL/vulnerability-scores](https://huggingface.co/datasets/CIRCL/vulnerability-scores).
-
-The model was presented in the paper [VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification](https://huggingface.co/papers/2507.03607) [[arXiv](https://arxiv.org/abs/2507.03607)].
-
-**Abstract:** VLAI is a transformer-based model that predicts software vulnerability severity levels directly from text descriptions. Built on RoBERTa, VLAI is fine-tuned on over 600,000 real-world vulnerabilities and achieves over 82% accuracy in predicting severity categories, enabling faster and more consistent triage ahead of manual CVSS scoring. The model and dataset are open-source and integrated into the Vulnerability-Lookup service.
-
-You can read [this page](https://www.vulnerability-lookup.org/user-manual/ai/) for more information.
+# vulnerability-severity-classification-roberta-base
 
+This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.0079
+- Accuracy: 0.8187
+- F1 Macro: 0.7495
+- Low Precision: 0.6490
+- Low Recall: 0.5059
+- Low F1: 0.5686
+- Medium Precision: 0.8468
+- Medium Recall: 0.8712
+- Medium F1: 0.8588
+- High Precision: 0.8140
+- High Recall: 0.8114
+- High F1: 0.8127
+- Critical Precision: 0.7671
+- Critical Recall: 0.7488
+- Critical F1: 0.7579
 
 ## Model description
 
-It is a classification model and is aimed to assist in classifying vulnerabilities by severity based on their descriptions.
-
-## How to get started with the model
-
-```python
-from transformers import AutoModelForSequenceClassification, AutoTokenizer
-import torch
+More information needed
 
-labels = ["low", "medium", "high", "critical"]
+## Intended uses & limitations
 
-model_name = "CIRCL/vulnerability-severity-classification-roberta-base"
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForSequenceClassification.from_pretrained(model_name)
-model.eval()
+More information needed
 
-test_description = "SAP NetWeaver Visual Composer Metadata Uploader is not protected with a proper authorization, allowing unauthenticated agent to upload potentially malicious executable binaries \
-that could severely harm the host system. This could significantly affect the confidentiality, integrity, and availability of the targeted system."
-inputs = tokenizer(test_description, return_tensors="pt", truncation=True, padding=True)
+## Training and evaluation data
 
-# Run inference
-with torch.no_grad():
-    outputs = model(**inputs)
-    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
-
-# Print results
-print("Predictions:", predictions)
-predicted_class = torch.argmax(predictions, dim=-1).item()
-print("Predicted severity:", labels[predicted_class])
-```
+More information needed
 
 ## Training procedure
 
@@ -76,37 +59,20 @@ The following hyperparameters were used during training:
 - lr_scheduler_type: linear
 - num_epochs: 5
 
-It achieves the following results on the evaluation set:
-- Loss: 2.0549
-- Accuracy: 0.8168
-- F1 Macro: 0.7486
-- Low Precision: 0.6931
-- Low Recall: 0.4992
-- Low F1: 0.5804
-- Medium Precision: 0.8449
-- Medium Recall: 0.8722
-- Medium F1: 0.8584
-- High Precision: 0.8083
-- High Recall: 0.8124
-- High F1: 0.8103
-- Critical Precision: 0.7617
-- Critical Recall: 0.7299
-- Critical F1: 0.7455
-
 ### Training results
 
 | Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1 Macro | Low Precision | Low Recall | Low F1 | Medium Precision | Medium Recall | Medium F1 | High Precision | High Recall | High F1 | Critical Precision | Critical Recall | Critical F1 |
 |:-------------:|:-----:|:-----:|:---------------:|:--------:|:--------:|:-------------:|:----------:|:------:|:----------------:|:-------------:|:---------:|:--------------:|:-----------:|:-------:|:------------------:|:---------------:|:-----------:|
-| 2.4224        | 1.0   | 16352 | 2.5629          | 0.7378   | 0.6209   | 0.7021        | 0.2288     | 0.3451 | 0.7902           | 0.8129        | 0.8014    | 0.7011         | 0.7546      | 0.7269  | 0.6479             | 0.5765          | 0.6101      |
-| 2.1704        | 2.0   | 32704 | 2.3004          | 0.7693   | 0.6756   | 0.6116        | 0.3487     | 0.4442 | 0.8013           | 0.8491        | 0.8245    | 0.7664         | 0.7493      | 0.7577  | 0.6794             | 0.6727          | 0.6760      |
-| 2.1241        | 3.0   | 49056 | 2.1680          | 0.7906   | 0.7040   | 0.7357        | 0.3666     | 0.4894 | 0.8024           | 0.8852        | 0.8417    | 0.8044         | 0.7481      | 0.7752  | 0.7111             | 0.7082          | 0.7097      |
-| 1.5991        | 4.0   | 65408 | 2.0822          | 0.8083   | 0.7326   | 0.7214        | 0.4372     | 0.5444 | 0.8378           | 0.8689        | 0.8531    | 0.8028         | 0.7983      | 0.8005  | 0.7279             | 0.7370          | 0.7324      |
-| 1.6372        | 5.0   | 81760 | 2.0549          | 0.8168   | 0.7486   | 0.6931        | 0.4992     | 0.5804 | 0.8449           | 0.8722        | 0.8584    | 0.8083         | 0.8124      | 0.8103  | 0.7617             | 0.7299          | 0.7455      |
+| 2.8382        | 1.0   | 16475 | 2.5695          | 0.7351   | 0.6552   | 0.4964        | 0.4244     | 0.4576 | 0.7986           | 0.7994        | 0.7990    | 0.7393         | 0.6933      | 0.7156  | 0.5868             | 0.7245          | 0.6484      |
+| 2.3037        | 2.0   | 32950 | 2.3201          | 0.7709   | 0.6774   | 0.6294        | 0.3490     | 0.4490 | 0.8025           | 0.8541        | 0.8275    | 0.7629         | 0.7502      | 0.7565  | 0.6908             | 0.6630          | 0.6766      |
+| 2.1765        | 3.0   | 49425 | 2.1006          | 0.7905   | 0.7077   | 0.6790        | 0.3867     | 0.4928 | 0.8246           | 0.8568        | 0.8404    | 0.7894         | 0.7659      | 0.7775  | 0.6903             | 0.7524          | 0.7201      |
+| 1.7249        | 4.0   | 65900 | 2.0247          | 0.8091   | 0.7329   | 0.6677        | 0.4528     | 0.5396 | 0.8236           | 0.8874        | 0.8543    | 0.8136         | 0.7828      | 0.7979  | 0.7669             | 0.7144          | 0.7397      |
+| 1.3227        | 5.0   | 82375 | 2.0079          | 0.8187   | 0.7495   | 0.6490        | 0.5059     | 0.5686 | 0.8468           | 0.8712        | 0.8588    | 0.8140         | 0.8114      | 0.8127  | 0.7671             | 0.7488          | 0.7579      |
 
 
 ### Framework versions
 
-- Transformers 5.8.0
-- Pytorch 2.11.0+cu130
+- Transformers 5.8.1
+- Pytorch 2.12.0+cu130
 - Datasets 4.8.5
 - Tokenizers 0.22.2

config.json

@@ -34,7 +34,7 @@
   "pad_token_id": 1,
   "problem_type": "single_label_classification",
   "tie_word_embeddings": true,
-  "transformers_version": "5.8.0",
+  "transformers_version": "5.8.1",
   "type_vocab_size": 1,
   "use_cache": true,
   "vocab_size": 50265

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a02fca88e4cc69505091129c6eacd435a26eedafe6a4d35755ba9c3ddc692600
+oid sha256:425da81b964907c943a9fd042b0ac9f4fbf58f2a82baee3448fba8432fbf76cc
 size 498618976