cyber-pal-security/CyberOss-2.0-20B

CyberPal-2.0-20B

CyberPal-2.0-20B is a cybersecurity-expert 20B-parameter Small Language Model (SLM) fine-tuned for security operations and threat-management workflows (e.g., CTI Q&A, vulnerability-to-weakness mapping, detection/mitigation recommendations). It is part of the CyberPal 2.0 model family (4B–20B), trained using the SecKnowledge 2.0 data enrichment + formatting pipeline to produce higher-fidelity, task-grounded reasoning traces for cybersecurity tasks. (https://arxiv.org/html/2510.14113v1)

---

Model Details

• Model type: Decoder-only language model (instruction-tuned)
• Parameters: ~20B
• Base model: gpt-oss-20b (fine-tuned)
• Domain focus: Cybersecurity (CTI, SOC/IR, appsec, IAM, governance/compliance)
• Context length: 7168
• Training data: SecKnowledge 2.0 (see below)
• Paper: “Toward Cybersecurity-Expert Small Language Models” (arXiv:2510.14113)

---

Intended Use

CyberPal-2.0-20B is intended for defensive cybersecurity use-cases, such as:

• Cyber Threat Intelligence (CTI): answering CTI questions; mapping campaigns/actors/techniques; explaining ATT&CK concepts
• Vulnerability & weakness analysis: correlating CVE evidence / bug tickets → CWE root causes
• Detection & mitigation guidance: proposing detections/mitigations for tactics/techniques/weaknesses/vulnerabilities
• Security operations support: incident summarization, investigation assistance, hypothesis-driven triage, response recommendations

The paper’s stated goal is a domain-specialized backbone for “threat management and security operations” that is practical for enterprise/on-prem deployments.

Out-of-scope / not recommended

• Any form of wrongdoing, intrusion, malware development, or instructions intended to enable harm.
• High-stakes decisions without human review (treat outputs as advisory, not authoritative).
• Agentic/Tool use is currently not tested and support will be released in newer versions.

---

Training Data (SecKnowledge 2.0)

CyberPal-2.0-20B is fine-tuned on SecKnowledge 2.0, produced by an enrichment pipeline that:

• uses expert-in-the-loop schema/format steering (task-specific reasoning formats),
• performs multi-step grounding using documents and/or web search,
• and uses LLM-based judging for readability/factuality checks in the pipeline.

SecKnowledge (the starting dataset) is described as:

• 153k instructions in the first stage (from structured public security sources),
• expanded to a 403k-example cybersecurity corpus in the second stage via synthetic generation.

The SecKnowledge 2.0 pipeline uses gpt-oss-120b (Medium reasoning effort) as the backbone LLM for dataset generation/enrichment.

Note: For full dataset composition/task breakdowns and templates, see the paper’s SecKnowledge 2.0 section and appendices.

---

How to Use

Tips:

• Trained on both short and long reasoning traces, as described in the paper. For harder questions, prompt it to “think step-by-step” before answering.
• vLLM inference is not fully test, thus HF use is currently recommended.

Transformers (example)

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "matanle/CyberPal-2.0-20B"

tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = """You are a security analyst.
Question: Given this CVE description and bug ticket excerpt, map the likely root cause CWE(s) and explain why."""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        temperature=0.0,
    )

print(tokenizer.decode(out[0], skip_special_tokens=True))

## Paper

This model is associated with the paper:
[Toward Cybersecurity-Expert Small Language Models](https://arxiv.org/abs/2510.14113)

## Citation

If you use this model, please cite:

@misc{levi2025towardcybersecurityexpertsmall,
  title={Toward Cybersecurity-Expert Small Language Models},
  author={Matan Levi and Daniel Ohayon and Ariel Blobstein and Ravid Sagi and Ian Molloy and Yair Allouche},
  year={2025},
  eprint={2510.14113},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  doi={10.48550/arXiv.2510.14113},
  url={https://arxiv.org/abs/2510.14113}
}
---
license: apache-2.0
---