wagonbomb/sigmashift-suricata-v1

SigmaShift Suricata Rule Generator v1

A fine-tuned Mistral-7B model that generates production-ready Suricata IDS rules and Zeek detection scripts from security findings (CVEs, vulnerability descriptions, threat intelligence).

Model Details

Parameter	Value
Base model	mistralai/Mistral-7B-Instruct-v0.3
Method	QLoRA (rank=64, alpha=128, NF4 quantization)
Training	3 epochs, 1266 steps, batch 4, lr 2e-4
Final train loss	0.106
Best eval loss	0.141 (checkpoint 1200)
Quantization	Q4_K_M (4.1GB) via llama.cpp

Evaluation

Tested on 20 held-out CVE test cases (not in training data):

• Average score: 8.4/12
• Suricata: 100% structurally valid rules
• Zeek: Valid detection scripts with Notice framework integration

Scoring criteria: block presence (suricata/zeek/reasoning), valid action keyword, msg field, SID, classtype, rev, balanced parentheses, Notice redef, event handler.

Usage with Ollama

# Download the GGUF
# Create model from Modelfile.q4km (update the FROM path)
ollama create sigmashift-suricata:v1-q4km -f Modelfile.q4km

# Run
ollama run sigmashift-suricata:v1-q4km "Generate a Suricata rule for CVE-2021-44228 Log4Shell"

Output Format

The model outputs three tagged blocks:

<suricata>
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET EXPLOIT Apache Log4j RCE (CVE-2021-44228)"; ...)
</suricata>

<zeek>
@load base/frameworks/notice
redef enum Notice::Type += { Log4Shell_Attempt };
event http_message_done(c: connection, is_orig: bool) { ... }
</zeek>

<reasoning>
Detection approach explanation...
</reasoning>

Files

File	Description
sigmashift-suricata-v1-q4km.gguf	Q4_K_M quantized model (4.1GB)
Modelfile / Modelfile.q4km	Ollama model definitions with system prompt
adapter/	QLoRA adapter weights (for custom merging)
eval_report.md	Full 20-case evaluation report

Training Data

Built from:

• Emerging Threats Open ruleset (~40k Suricata rules)
• NVD API CVE descriptions joined to ET rules by CVE reference
• MITRE ATT&CK technique descriptions joined by metadata tags

Part of SigmaShift

This model powers the offline/air-gapped rule generation mode in SigmaShift, a security analysis platform that converts vulnerability scan output into detection rules.

License

Apache 2.0