Commit
·
7221211
1
Parent(s):
db25331
Update README.md
Browse files
README.md
CHANGED
|
@@ -5,13 +5,16 @@ language:
|
|
| 5 |
tags:
|
| 6 |
- cybersecurity
|
| 7 |
---
|
|
|
|
| 8 |
This model represents an improved version of the [SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT) model, trained on a corpus eight times larger than its predecessor, leveraging the computational power of 8xA100 GPUs. This version, known as SecureBERT+, brings forth an average improvment of 9% in the performance of the Masked Language Model (MLM) task. This advancement signifies a substantial stride towards achieving heightened proficiency in language understanding and representation learning within the cybersecurity domain.
|
| 9 |
|
| 10 |
|
| 11 |
SecureBERT is a domain-specific language model based on RoBERTa which is trained on a huge amount of cybersecurity data and fine-tuned/tweaked to understand/represent cybersecurity textual data.
|
| 12 |
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
[SecureGPT](https://huggingface.co/ehsanaghaei/SecureGPT)
|
| 17 |
|
|
|
|
| 5 |
tags:
|
| 6 |
- cybersecurity
|
| 7 |
---
|
| 8 |
+
# SecureBERT+
|
| 9 |
This model represents an improved version of the [SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT) model, trained on a corpus eight times larger than its predecessor, leveraging the computational power of 8xA100 GPUs. This version, known as SecureBERT+, brings forth an average improvment of 9% in the performance of the Masked Language Model (MLM) task. This advancement signifies a substantial stride towards achieving heightened proficiency in language understanding and representation learning within the cybersecurity domain.
|
| 10 |
|
| 11 |
|
| 12 |
SecureBERT is a domain-specific language model based on RoBERTa which is trained on a huge amount of cybersecurity data and fine-tuned/tweaked to understand/represent cybersecurity textual data.
|
| 13 |
|
| 14 |
+
## Dataset
|
| 15 |
+

|
| 16 |
|
| 17 |
+
Other model variants:
|
| 18 |
|
| 19 |
[SecureGPT](https://huggingface.co/ehsanaghaei/SecureGPT)
|
| 20 |
|