MiST-8B

MiST (Mid-trained Security Transformer) is an 8B-parameter cybersecurity-specialized language model developed by Dream. It is derived from Qwen3-8B-Base and adapted for cybersecurity knowledge and analysis through a three-stage pipeline: cybersecurity mid-training on a curated synthetic corpus, supervised fine-tuning, and preference optimization. While MiST-8B can be used directly for cybersecurity knowledge and analysis tasks such as cyber threat intelligence, CVE-to-CWE mapping, MITRE ATT&CK interpretation, and vulnerability analysis, we highly recommend fine-tuning it on your specific use case for best results. It is designed to serve as a strong starting point for building custom cybersecurity systems through further task-specific fine-tuning or reinforcement learning.

MiST-8B is released as an open research contribution accompanying our EMNLP paper "MiST: Mid-Training LLMs for Cybersecurity." It is not intended for autonomous decision-making or for deployment as an autonomous security agent.

Model Details

• Paper: https://arxiv.org/abs/2609.18496
• Developer: Dream
• Base model: Qwen3-8B-Base (Apache 2.0)
• Released checkpoint: MiST-8B-DPO (reported as MiST-8B-DPO in the paper)
• License: Apache 2.0
• Language: English
• Seed corpus: released separately alongside this model

---

Quick start

MiST-8B uses the same inference setup as Qwen3-8B. For code snippets covering model loading with the transformers library, recommended inference settings, and the chat template, refer to the Quick Start section of the Qwen3-8B model card:

https://huggingface.co/Qwen/Qwen3-8B

The same loading pattern applies to MiST-8B; substitute this repository's identifier for the base model identifier in the code examples.

---

What MiST is and how it was built

Cybersecurity is a knowledge-intensive domain where general-purpose LLMs often lack the precise terminology, taxonomies, and structured reasoning needed for reliable analysis. MiST addresses this through mid-training as a corpus-design problem. Rather than continued pre-training on very large volumes of raw domain text, we curate a compact, expert-vetted seed corpus of high-quality cybersecurity sources and transform it into synthetic training data via multiple generation flows.

Starting from Qwen3-8B-Base, MiST is adapted through three stages:

1. Cybersecurity mid-training on our curated corpus (~976M tokens), combining synthetic cybersecurity content with a small subset of general instruction data to preserve broad capabilities.
2. Supervised fine-tuning (SFT) on a mixture of synthetic cybersecurity conversations and general instruction data (~1.55B tokens total).
3. Direct preference optimization (DPO) using general (non-cybersecurity) preference data as a final alignment step to improve general response quality while preserving cybersecurity capabilities acquired earlier.

All training was conducted on a single node of 8× NVIDIA B200 GPUs. Full methodology, training data statistics, and hyperparameters are described in the paper.

---

Training data

MiST is trained on a combination of curated cybersecurity sources, synthetic data derived from them, and third-party general instruction data.

Seed corpus (~65M tokens): expert-vetted authoritative cybersecurity sources organized into four families. The seed corpus is released separately alongside this model.

• Vulnerability and threat intelligence: NVD CVE records; curated cyber threat intelligence reports (used during training only; not included in the released seed corpus).
• Security knowledge bases and taxonomies: MITRE ATT&CK, CAPEC, D3FEND; CWE.
• Operational security artifacts: Sigma rules, Atomic Red Team tests, Splunk ESCU detections, MISP Galaxy clusters.
• Defensive guidance and platform documentation: OWASP, NIST publications, vendor and platform documentation, security-focused subset of English Wikipedia.

Synthetic mid-training data (~976M tokens): generated from the seed corpus using Qwen3-30B-A3B through several complementary flows (paraphrasing, educational transformation, factual Q&A, and analytical explanation), followed by LLM-based quality filtering. A small subset of general instruction data (AllenAI's Olmo 3 Dolci-Instruct-SFT) is included to preserve general capabilities.

SFT data: synthetic cybersecurity conversations combined with the full AllenAI Dolci-Instruct-SFT dataset (~1.55B tokens total).

DPO data: AllenAI's Dolci-Instruct-DPO (general, non-cybersecurity preference data).

Full data statistics, licensing notes, and the decontamination pipeline are described in the paper.

Attribution. MiST-8B training used material from the following AI2 datasets:

• Dolci-Instruct-SFT: https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT
• Dolci-Instruct-DPO: https://huggingface.co/datasets/allenai/Dolci-Instruct-DPO

The Dolci datasets are licensed under the Open Data Commons Attribution License (ODC-By).

---

Evaluation

MiST-8B is evaluated on a broad public cybersecurity benchmark suite and on standard general-capability benchmarks. All evaluations use non-reasoning inference whenever a reasoning or thinking mode is available on the evaluated model. Full per-benchmark results, evaluation methodology, and comparison to additional models (including large proprietary systems) are described in the paper.

Cybersecurity benchmark suite

Comparison of MiST-8B against the Qwen3-8B baseline and prior open cybersecurity-specialized models of comparable size (plus IBM Research's CyberPal 2.0-20B, included as a direct peer despite its larger parameter count). Scores are accuracy or the benchmark's reported metric.

Model	CTI-MCQ	CTI-RCM	SECURE-MAET	SECURE-CWET	SECURE-KCV	Athena-CKT	Athena-ATE	Athena-RMS	Athena-TAA	PrimeVul	PV-CoT	MMLU-Cyber	SecEval	Mean
Qwen3-8B (baseline)	58.2	53.5	83.8	87.5	84.5	75.2	9.4	5.5	15.0	2.0	13.0	78.0	66.7	48.6
CyberPal 2.0-20B	73.8	72.8	91.4	92.5	83.6	82.7	57.0	23.3	19.6	0.8	14.1	82.2	67.1	58.5
DeepHat-V1-7B	54.7	67.2	82.7	83.1	86.2	69.2	9.8	3.1	14.8	12.0	3.9	79.8	62.4	48.4
Foundation-Sec-8B	58.2	67.9	84.7	83.3	80.1	76.9	37.3	18.7	26.4	0.0	17.8	78.6	65.0	53.5
Lily-Cybersecurity-7B	42.1	42.9	55.8	54.4	43.2	67.0	3.0	2.1	12.4	0.0	7.1	68.8	48.9	34.4
Primus-8B	54.7	67.2	81.9	82.6	82.0	73.7	30.6	7.3	21.4	0.3	13.4	82.2	60.9	50.6
Qwen3-8B-Primus	66.8	64.4	85.9	87.0	75.8	76.4	33.9	11.5	24.6	0.3	17.1	86.0	61.8	53.2
RedSage-8B-DPO	62.0	73.2	89.6	91.2	81.1	78.8	51.6	18.8	22.8	1.6	14.8	82.4	67.4	56.5
MiST-8B	79.3	73.9	94.3	93.7	87.3	86.0	57.3	25.7	24.6	10.0	15.0	83.4	71.5	61.7

MiST-8B improves over the Qwen3-8B baseline by +13.1 absolute percentage points (+27.0% relative) on the suite mean, and achieves the highest mean score of any open cybersecurity-specialized model in this comparison, outperforming peer releases from Cisco (Foundation-Sec-8B), Trend Micro (Primus family), IBM Research (CyberPal 2.0-20B), and others. The benchmark suite covers CTI-Bench (MCQ, RCM), SECURE (MAET, CWET, KCV), AthenaBench (CKT, ATE, RMS, TAA), PrimeVul (direct and chain-of-thought), MMLU Computer Security, and SecEval. Benchmark descriptions and prompting strategies are in the paper (Section 4).

General capabilities

To quantify catastrophic forgetting from cybersecurity specialization, MiST-8B is evaluated on standard reasoning, instruction-following, and knowledge benchmarks:

Model	ARC-C	GSM8K	MMLU	IFEval	Mean
Qwen3-8B (baseline)	77.1	89.7	73.4	90.0	82.5
MiST-8B	89.8	90.0	74.9	88.5	85.8

MiST-8B retains or improves on the general capabilities of its base model, indicating that cybersecurity specialization did not degrade broad reasoning, instruction following, or knowledge.

Downstream adaptation

Beyond out-of-the-box benchmark performance, MiST-8B is designed to serve as a strong starting point for building custom cybersecurity systems through further adaptation.

Under GRPO reinforcement learning on verifiable cybersecurity tasks (CVE→CWE mapping, CVSS vector prediction, and vulnerable-code-to-CWE mapping), MiST-8B reaches higher validation accuracy than the Qwen3-8B baseline throughout training while requiring smaller policy movement from the initial model.

Under task-specific supervised fine-tuning on PrimeVul vulnerability detection, MiST gains substantially more from fine-tuning than the Qwen3 baseline, under both direct and chain-of-thought prompting.

See the paper (Section 5.4) for full downstream-adaptation results.

---

Intended use

MiST-8B is designed for cybersecurity knowledge, analysis, and defense tasks where structured reasoning over security concepts is valuable. Intended applications include:

• Defensive cybersecurity analysis: supporting security analysts, incident responders, and defenders in interpreting, prioritizing, and reasoning about security information.
• Research and downstream adaptation: MiST provides a strong starting checkpoint for further task-specific fine-tuning or reinforcement learning on cybersecurity tasks (as shown in the paper for CVE→CWE mapping, CVSS vector prediction, and vulnerable-code classification).
• Cyber threat intelligence (CTI): answering CTI questions, explaining MITRE ATT&CK tactics and techniques, mapping campaigns and adversary techniques.
• Vulnerability and weakness analysis: correlating CVE evidence to CWE root causes, reasoning about CVSS metrics, explaining vulnerability impact.
• Detection and mitigation analysis: explaining and evaluating security controls, detection logic, and mitigation strategies.
• Cybersecurity knowledge tasks: answering questions grounded in standards and taxonomies such as NIST, OWASP, CWE, CAPEC, and D3FEND.

MiST is intended for use by security researchers, analysts, and developers building cybersecurity-focused applications, with appropriate human oversight.

---

Out of scope

MiST-8B should not be used for:

• Any form of wrongdoing, intrusion, malware development, or instructions intended to enable harm to systems, networks, or individuals.
• High-stakes operational decisions without human review. Model outputs should be treated as advisory, not authoritative.
• Autonomous agent deployment or long-horizon multi-step agentic workflows. Agentic and tool-use capabilities have not been specifically trained or evaluated in this release, and independent testing indicates that models in this class require external prompt-injection defenses when deployed in tool-using settings.
• Live security operations (SOC triage, incident response, automated remediation, log and telemetry analysis at scale, secure code assistance) without human-in-the-loop oversight. These operational settings were not evaluated for this release.
• Reasoning-specialized tasks. MiST is not a reasoning-optimized model; it is designed for knowledge and analysis.
• Generating content or advice in domains outside cybersecurity where domain-specific expertise is required.

Users deploying MiST in security-related products should implement appropriate guardrails, monitoring, access controls, and human oversight consistent with their industry, jurisdiction, and use case.

---

Ethical considerations

Dream views responsible AI development as a shared responsibility between model providers and deployers. MiST has been developed and released with the following considerations.

Dual-use risk. Improved performance on vulnerability analysis, threat-intelligence analysis, and security understanding may lower the barrier to misuse. This risk is addressed through the deliberate exclusion of agentic reinforcement learning and long-horizon agentic training, the inclusion of safety and refusal data in the training mixture, and a preliminary safety evaluation reported in the paper (Section O). The safety evaluation is not comprehensive and does not establish safety under adaptive, multi-turn, or tool-assisted attacks.

Data governance. Training data consists of public and third-party cybersecurity sources plus synthetic transformations grounded in those sources. We treat synthetic samples as governed derivatives of their source material. Source provenance, applicable licenses, and known restrictions are documented in the paper and in the released dataset documentation. If you believe the released materials contain content that should not be included, please contact us through the reporting mechanism below.

Deployer responsibility. Developers integrating MiST into cybersecurity products should add application-specific safeguards and guardrails appropriate to the deployment context, evaluate the system end-to-end in the intended deployment environment before production use, preserve human oversight for high-stakes decisions, implement appropriate logging and monitoring, and comply with applicable industry regulations and legal requirements in their jurisdiction.

Scope caveats. Results should be interpreted as evidence of improved cybersecurity knowledge and structured analysis on the evaluated benchmarks, not as a direct measure of deployment readiness in live security operations. MiST is trained and evaluated primarily in English on Western-published cybersecurity sources; performance in other languages or on cybersecurity practices specific to other regulatory environments has not been evaluated.

---

Citation

If you use MiST in your work, please cite:

@inproceedings{ovadia2026mist,
  title     = {MiST: Mid-Training LLMs for Cybersecurity},
  author    = {Oded Ovadia and Elad Ben Zaken and Elad Guttman and Orly Moreno Kadosh},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2026},
  publisher = {Association for Computational Linguistics}
}

---

Contact and reporting

For issues, questions, or requests related to the release, please open a discussion on this Hugging Face repository or contact the authors at the addresses provided in the paper.

To report a concern regarding released data or model behavior, please use the discussion feature on this repository.

For copyright or other legal matters, please contact legal@dreamgroup.com. Dream’s copyright policy is available at https://dreamgroup.com/copyright-policy, and the public summaries of training content at https://dreamgroup.com/public-summaries.