MiST: Mid-Training LLMs for Cybersecurity
Paper • 2609.18496 • Published
MiST-30B-A3B
MiST (Mid-trained Security Transformer) 30B-A3B is a cybersecurity-specialized language model developed by Dream. It is derived from NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16, a 30B-parameter Mixture-of-Experts model with approximately 3B active parameters per forward pass. MiST-30B-A3B is adapted for cybersecurity knowledge and analysis through a two-stage pipeline: cybersecurity mid-training on a curated synthetic corpus followed by supervised fine-tuning. While MiST-30B-A3B can be used directly for cybersecurity knowledge and analysis tasks such as cyber threat intelligence, CVE-to-CWE mapping, MITRE ATT&CK interpretation, and vulnerability analysis, we highly recommend fine-tuning it on your specific use case for best results. It is designed to serve as a strong starting point for building custom cybersecurity systems through further task-specific fine-tuning or reinforcement learning.
MiST-30B-A3B is released alongside our EMNLP paper "MiST: Mid-Training LLMs for Cybersecurity." The paper itself focuses on Qwen-based MiST variants (MiST-8B and MiST-32B); MiST-30B-A3B applies the same mid-training pipeline and training data to NVIDIA's Nemotron 3.5 Lightning 30B base. It is not intended for autonomous decision-making or for deployment as an autonomous security agent.
Model Details
- Methodology paper: https://arxiv.org/abs/2609.18496
- Developer: Dream
- Base model: NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16 (OpenMDW-1.1)
- Released checkpoint: MiST-30B-A3B-SFT
- License: Apache 2.0
- Language: English
- Seed corpus: released separately alongside this model
License notice. MiST-30B-A3B is released under the Apache 2.0 license. The NVIDIA-originated model materials inherited from the base model remain subject to the OpenMDW-1.1 license. Under OpenMDW-1.1, any party that initiates litigation claiming that the model materials infringe a patent or copyright loses their rights to use those materials.
Quick start
MiST-30B-A3B uses the same inference setup as its NVIDIA base model. For code snippets covering model loading with the transformers library, recommended inference settings, and the chat template, refer to the Quick Start section of the Nemotron-3.5-Lightning model card:
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
The same loading pattern applies to MiST-30B-A3B; substitute this repository's identifier for the base model identifier in the code examples.
What MiST is and how it was built
Cybersecurity is a knowledge-intensive domain where general-purpose LLMs often lack the precise terminology, taxonomies, and structured reasoning needed for reliable analysis. MiST addresses this through mid-training as a corpus-design problem. Rather than continued pre-training on very large volumes of raw domain text, we curate a compact, expert-vetted seed corpus of high-quality cybersecurity sources and transform it into synthetic training data via multiple generation flows.
Starting from NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16, MiST-30B-A3B is adapted through two stages:
- Cybersecurity mid-training on our curated corpus (~976M tokens), combining synthetic cybersecurity content with a small subset of general instruction data to preserve broad capabilities.
- Supervised fine-tuning (SFT) on a mixture of synthetic cybersecurity conversations and general instruction data (~1.55B tokens total).
The training data and the mid-training and SFT stages of the pipeline are identical to those used for our Qwen-based MiST-8B release; the two differences for MiST-30B-A3B are the base model and the omission of the final DPO alignment stage. Full methodology, training data statistics, and hyperparameters are described in the paper.
Training data
MiST-30B-A3B is trained on a combination of curated cybersecurity sources, synthetic data derived from them, and third-party general instruction data.
Seed corpus (~65M tokens): expert-vetted authoritative cybersecurity sources organized into four families. The seed corpus is released separately alongside this model.
- Vulnerability and threat intelligence: NVD CVE records; curated cyber threat intelligence reports (used during training only; not included in the released seed corpus).
- Security knowledge bases and taxonomies: MITRE ATT&CK, CAPEC, D3FEND; CWE.
- Operational security artifacts: Sigma rules, Atomic Red Team tests, Splunk ESCU detections, MISP Galaxy clusters.
- Defensive guidance and platform documentation: OWASP, NIST publications, vendor and platform documentation, security-focused subset of English Wikipedia.
Synthetic mid-training data (~976M tokens): generated from the seed corpus using Qwen3-30B-A3B through several complementary flows (paraphrasing, educational transformation, factual Q&A, and analytical explanation), followed by LLM-based quality filtering. A small subset of general instruction data (AllenAI's Olmo 3 Dolci-Instruct-SFT) is included to preserve general capabilities.
SFT data: synthetic cybersecurity conversations combined with the full AllenAI Dolci-Instruct-SFT dataset (~1.55B tokens total).
Full data statistics, licensing notes, and the decontamination pipeline are described in the paper.
Attribution. MiST-30B-A3B training used material from the following AI2 dataset:
- Dolci-Instruct-SFT: https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT
The Dolci-Instruct-SFT dataset is licensed under the Open Data Commons Attribution License (ODC-By).
Evaluation
MiST-30B-A3B is evaluated on a broad public cybersecurity benchmark suite and on standard general-capability benchmarks. All evaluations use non-reasoning inference whenever a reasoning or thinking mode is available on the evaluated model. MiST-30B-A3B and the Nemotron-3.5-Lightning 30B-A3B baseline results are reported as the mean over three evaluation runs; results for the other comparison models are taken from the MiST paper's evaluation.
Cybersecurity benchmark suite
Comparison of MiST-30B-A3B against its Nemotron-3.5-Lightning 30B-A3B base model and prior open cybersecurity-specialized models. Scores are accuracy or the benchmark's reported metric; bold indicates the best score in each column.
| Model | CTI-MCQ | CTI-RCM | SECURE-MAET | SECURE-CWET | SECURE-KCV | Athena-CKT | Athena-ATE | Athena-RMS | Athena-TAA | PrimeVul | PV-CoT | MMLU-Cyber | SecEval | Mean |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepHat-V1-7B | 54.7 | 67.2 | 82.7 | 83.1 | 86.2 | 69.2 | 9.8 | 3.1 | 14.8 | 12.0 | 3.9 | 79.8 | 62.4 | 48.4 |
| Foundation-Sec-8B | 58.2 | 67.9 | 84.7 | 83.3 | 80.1 | 76.9 | 37.3 | 18.7 | 26.4 | 0.0 | 17.8 | 78.6 | 65.0 | 53.5 |
| Lily-Cybersecurity-7B | 42.1 | 42.9 | 55.8 | 54.4 | 43.2 | 67.0 | 3.0 | 2.1 | 12.4 | 0.0 | 7.1 | 68.8 | 48.9 | 34.4 |
| Qwen3-8B-Primus | 66.8 | 64.4 | 85.9 | 87.0 | 75.8 | 76.4 | 33.9 | 11.5 | 24.6 | 0.3 | 17.1 | 86.0 | 61.8 | 53.2 |
| RedSage-8B-DPO | 62.0 | 73.2 | 89.6 | 91.2 | 81.1 | 78.8 | 51.6 | 18.8 | 22.8 | 1.6 | 14.8 | 82.4 | 67.4 | 56.5 |
| Primus-8B | 54.7 | 67.2 | 81.9 | 82.6 | 82.0 | 73.7 | 30.6 | 7.3 | 21.4 | 0.3 | 13.4 | 82.2 | 60.9 | 50.6 |
| Primus-70B | 67.7 | 65.1 | 91.4 | 93.7 | 88.1 | 82.1 | 52.7 | 14.8 | 2.2 | 1.0 | 17.6 | 87.6 | 70.5 | 56.5 |
| CyberPal 2.0-20B | 73.8 | 72.8 | 91.4 | 92.5 | 83.6 | 82.7 | 57.0 | 23.3 | 19.6 | 0.8 | 14.1 | 82.2 | 67.1 | 58.5 |
| Qwen3-32B | 68.4 | 65.9 | 89.5 | 91.9 | 88.8 | 80.2 | 26.0 | 6.3 | 15.2 | 4.0 | 16.0 | 84.8 | 72.7 | 54.6 |
| Nemotron-3.5-Lightning 30B-A3B (baseline) | 62.7 | 66.2 | 89.5 | 90.7 | 87.6 | 78.7 | 22.1 | 6.5 | 16.3 | 2.6 | 8.9 | 75.3 | 62.4 | 51.5 |
| MiST-30B-A3B | 71.8 | 74.6 | 93.4 | 92.3 | 84.7 | 86.1 | 63.6 | 33.3 | 24.3 | 2.1 | 12.1 | 74.0 | 71.8 | 60.3 |
MiST-30B-A3B improves over the Nemotron-3.5-Lightning baseline by +8.8 absolute percentage points (+17.1% relative) on the suite mean, and achieves the highest mean score of any open cybersecurity-specialized model in this comparison, outperforming peer releases from Cisco (Foundation-Sec-8B), Trend Micro (Primus family), and IBM Research (CyberPal 2.0-20B). The benchmark suite covers CTI-Bench (MCQ, RCM), SECURE (MAET, CWET, KCV), AthenaBench (CKT, ATE, RMS, TAA), PrimeVul (direct and chain-of-thought, scored by paired accuracy), MMLU Computer Security, and SecEval. Benchmark descriptions and prompting strategies are in the paper (Section 4).
General capabilities
To quantify the effect of cybersecurity specialization on general capabilities, MiST-30B-A3B is evaluated on standard reasoning, instruction-following, and knowledge benchmarks:
| Model | ARC-C | GSM8K | MMLU | IFEval | Mean |
|---|---|---|---|---|---|
| Nemotron-3.5-Lightning 30B-A3B (baseline) | 94.8 | 15.0 | 77.0 | 84.2 | 67.8 |
| MiST-30B-A3B | 78.7 | 81.5 | 76.0 | 87.7 | 81.0 |
On the four general-capability benchmarks, MiST-30B-A3B shows substantial gains on GSM8K and IFEval while declining on ARC-C and remaining roughly flat on MMLU, resulting in a higher overall suite mean than the base model.
Downstream adaptation
Beyond out-of-the-box benchmark performance, MiST-30B-A3B is designed to serve as a strong starting point for building custom cybersecurity systems through further adaptation.
As demonstrated in our paper for the Qwen-based MiST-8B variant (which uses the same mid-training recipe applied here), MiST models serve as a stronger starting point for both reinforcement learning and task-specific supervised fine-tuning on cybersecurity tasks. The figures below are reproduced from the paper's MiST-8B experiments:
Under GRPO reinforcement learning on verifiable cybersecurity tasks (CVE→CWE mapping, CVSS vector prediction, and vulnerable-code-to-CWE mapping), MiST-8B reaches higher validation accuracy than the Qwen3-8B baseline throughout training while requiring smaller policy movement from the initial model.
Under task-specific supervised fine-tuning on PrimeVul vulnerability detection, MiST gains substantially more from fine-tuning than the Qwen3 baseline, under both direct and chain-of-thought prompting.
Intended use
MiST-30B-A3B is designed for cybersecurity knowledge, analysis, and defense tasks where structured reasoning over security concepts is valuable. Intended applications include:
- Defensive cybersecurity analysis: supporting security analysts, incident responders, and defenders in interpreting, prioritizing, and reasoning about security information.
- Research and downstream adaptation: MiST-30B-A3B provides a strong starting checkpoint for further task-specific fine-tuning or reinforcement learning on cybersecurity tasks (as shown in the paper for CVE→CWE mapping, CVSS vector prediction, and vulnerable-code classification using the Qwen-based MiST-8B variant).
- Cyber threat intelligence (CTI): answering CTI questions, explaining MITRE ATT&CK tactics and techniques, mapping campaigns and adversary techniques.
- Vulnerability and weakness analysis: correlating CVE evidence to CWE root causes, reasoning about CVSS metrics, explaining vulnerability impact.
- Detection and mitigation analysis: explaining and evaluating security controls, detection logic, and mitigation strategies.
- Cybersecurity knowledge tasks: answering questions grounded in standards and taxonomies such as NIST, OWASP, CWE, CAPEC, and D3FEND.
MiST-30B-A3B is intended for use by security researchers, analysts, and developers building cybersecurity-focused applications, with appropriate human oversight.
Out of scope
MiST-30B-A3B should not be used for:
- Any form of wrongdoing, intrusion, malware development, or instructions intended to enable harm to systems, networks, or individuals.
- High-stakes operational decisions without human review. Model outputs should be treated as advisory, not authoritative.
- Autonomous agent deployment or long-horizon multi-step agentic workflows. Agentic and tool-use capabilities have not been specifically trained or evaluated in this release, and independent testing indicates that models in this class require external prompt-injection defenses when deployed in tool-using settings.
- Live security operations (SOC triage, incident response, automated remediation, log and telemetry analysis at scale, secure code assistance) without human-in-the-loop oversight. These operational settings were not evaluated for this release.
- Reasoning-specialized tasks. MiST-30B-A3B is not a reasoning-optimized model; it is designed for knowledge and analysis.
- Generating content or advice in domains outside cybersecurity where domain-specific expertise is required.
Users deploying MiST-30B-A3B in security-related products should implement appropriate guardrails, monitoring, access controls, and human oversight consistent with their industry, jurisdiction, and use case.
Ethical considerations
Dream views responsible AI development as a shared responsibility between model providers and deployers. MiST-30B-A3B has been developed and released with the following considerations.
Dual-use risk. Improved performance on vulnerability analysis, threat-intelligence analysis, and security understanding may lower the barrier to misuse. This risk is addressed through the deliberate exclusion of agentic reinforcement learning and long-horizon agentic training, the inclusion of safety and refusal data in the training mixture, and a preliminary safety evaluation. The safety evaluation is not comprehensive and does not establish safety under adaptive, multi-turn, or tool-assisted attacks.
Data governance. Training data consists of public and third-party cybersecurity sources plus synthetic transformations grounded in those sources. We treat synthetic samples as governed derivatives of their source material. Source provenance, applicable licenses, and known restrictions are documented in the paper and in the released dataset documentation. If you believe the released materials contain content that should not be included, please contact us through the reporting mechanism below.
Deployer responsibility. Developers integrating MiST-30B-A3B into cybersecurity products should add application-specific safeguards and guardrails appropriate to the deployment context, evaluate the system end-to-end in the intended deployment environment before production use, preserve human oversight for high-stakes decisions, implement appropriate logging and monitoring, and comply with applicable industry regulations and legal requirements in their jurisdiction.
Scope caveats. Results should be interpreted as evidence of improved cybersecurity knowledge and structured analysis on the evaluated benchmarks, not as a direct measure of deployment readiness in live security operations. MiST-30B-A3B is trained and evaluated primarily in English on Western-published cybersecurity sources; performance in other languages or on cybersecurity practices specific to other regulatory environments has not been evaluated.
Citation
If you use MiST in your work, please cite:
@inproceedings{ovadia2026mist,
title = {MiST: Mid-Training LLMs for Cybersecurity},
author = {Oded Ovadia and Elad Ben Zaken and Elad Guttman and Orly Moreno Kadosh},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
year = {2026},
publisher = {Association for Computational Linguistics}
}
Contact and reporting
For issues, questions, or requests related to the release, please open a discussion on this Hugging Face repository or contact the authors at the addresses provided in the paper.
To report a concern regarding released data or model behavior, please use the discussion feature on this repository.
For copyright or other legal matters, please contact legal@dreamgroup.com. Dream’s copyright policy is available at https://dreamgroup.com/copyright-policy, and the public summaries of training content at https://dreamgroup.com/public-summaries.
- Downloads last month
- -