Commit
·
2ff2376
1
Parent(s):
8736038
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,19 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- cybersecurity
|
| 7 |
---
|
| 8 |
+
This model represents an enhanced iteration of the [SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT) model, trained on a corpus eight times larger than its predecessor, over the course of 400 hours, leveraging the computational power of 8xA100 GPUs. The innovation, known as SecureBERT_Plus, brings forth an average improvment of 9% in the performance of the Masked Language Model (MLM) task. This advancement signifies a substantial stride towards achieving heightened proficiency in language understanding and representation learning within the cybersecurity knowledge.
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
SecureBERT is a domain-specific language model based on RoBERTa which is trained on a huge amount of cybersecurity data and fine-tuned/tweaked to understand/represent cybersecurity textual data.
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
Related cybersecurity language models:
|
| 15 |
+
[SecureGPT](https://huggingface.co/ehsanaghaei/SecureGPT)
|
| 16 |
+
[SecureDeBERTa](https://huggingface.co/ehsanaghaei/SecureDeBERTa)
|
| 17 |
+
[SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT)
|
| 18 |
+
|
| 19 |
+
|