pki/nova-24b-cybersec

nova:24b - Cybersecurity Domain LLM

24B parameter LLM fine-tuned on 40,000+ cybersecurity examples. Built on Dolphin3.0-R1-Mistral-24B.

Training Data (40,075 examples)

Dataset	Examples	Source
SecurityGPT	16,000	407 security PDFs
PKI Context	18,997	Security Q&A pairs
Energy Sector Threats	3,386	ICS/SCADA scenarios
ISO 27001 Controls	1,116	93 Annex A controls
ISO 27005 Threats	576	48 threat categories

Domains Covered

• Threat modeling & risk assessment
• Incident response
• Cryptography
• Vulnerability management
• Compliance (ISO 27001/27005)
• Adversarial ML
• Secure coding
• ICS/SCADA security

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("pki/nova-24b-cybersec")
tokenizer = AutoTokenizer.from_pretrained("pki/nova-24b-cybersec")

messages = [
    {"role": "system", "content": "You are a cybersecurity expert."},
    {"role": "user", "content": "Explain MITRE ATT&CK framework"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1)
print(tokenizer.decode(outputs[0]))

Ollama (GGUF)

# Download Q8 GGUF and create Modelfile
ollama create nova:24b -f Modelfile
ollama run nova:24b

Model Details

Aspect	Detail
Base Model	Dolphin3.0-R1-Mistral-24B
Parameters	24 billion
Context Window	32,768 tokens
Training Examples	40,075
Training Method	LoRA (r=32, alpha=64)
Hardware	RTX 4090, ~26 hours

Important: Temperature Setting

Critical: Use temperature 0.05-0.1. Mistral-24B requires very low temperature for coherent output.

outputs = model.generate(inputs, temperature=0.08, top_p=0.9)

Files

• model-*.safetensors - Model weights (transformers format)
• nova-24b-q8.gguf - Quantized GGUF for Ollama/llama.cpp

Training Configuration

• LoRA rank: 32
• LoRA alpha: 64
• Learning rate: 5e-5
• Epochs: 5
• Batch size: 40 (effective)
• Optimizer: AdamW 8-bit

Limitations

• Trained primarily on English text
• Best for security-focused tasks
• Requires low temperature (0.05-0.1)
• Large model - needs significant VRAM

License

Apache 2.0

Citation

@misc{nova24b-cybersec-2024,
  author = {PKI},
  title = {nova:24b - Cybersecurity Domain LLM},
  year = {2024},
  publisher = {HuggingFace},
}