|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: |
|
|
- Qwen/Qwen3-0.6B |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- cybersecurity |
|
|
- siem |
|
|
- log-analysis |
|
|
- field-extraction |
|
|
- security-automation |
|
|
- fine-tuned |
|
|
--- |
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/689df7f27100a16137c1ea74/O5bUR_i8GjNWqAB-nf15J.png" width="700"> |
|
|
|
|
|
|
|
|
# LLMSIEM/logem |
|
|
|
|
|
LLMSIEM/logem is a specialized language model fine-tuned for Security Information and Event Management (SIEM) tasks, particularly excelling at structured field extraction from security logs and events. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
LLMSIEM/logem is a fine-tuned version of Qwen3-0.6B, specifically optimized for cybersecurity applications. The model demonstrates that targeted fine-tuning can dramatically improve performance on domain-specific tasks, achieving superior results compared to much larger general-purpose models. |
|
|
|
|
|
- **Developed by:** [Hassan Shehata] |
|
|
- **Model type:** Causal Language Model (Fine-tuned) |
|
|
- **Language(s):** English |
|
|
- **License:** Apache 2.0 |
|
|
- **Finetuned from model:** Qwen/Qwen3-0.6B |
|
|
- **Model size:** 1.2 GB (FP16), 396 MB (Q4_K_M quantized) |
|
|
- **Parameters:** 0.6B |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Blog Post:** [LinkedIn/Blog Series Link] |
|
|
|
|
|
## Performance Highlights |
|
|
|
|
|
🏆 **Best-in-class performance** for SIEM field extraction tasks: |
|
|
- **66.7% perfect matches** (FP16 version) |
|
|
- **0.833 F1 score** - outperforms 12B parameter models |
|
|
- **1.00s average response time** - 3x faster than larger alternatives |
|
|
- **Zero complete failures** on standardized test suite |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
The model is designed for cybersecurity professionals and SIEM engineers who need to: |
|
|
- Extract structured fields from security logs |
|
|
- Parse and normalize security event data |
|
|
- Automate log analysis workflows |
|
|
- Generate structured outputs from unstructured security data |
|
|
|
|
|
### Example Use Cases |
|
|
|
|
|
```python |
|
|
# Example: Extract fields from a security log |
|
|
input_text = "Extract fields from: Failed login attempt from 192.168.1.100 for user admin at 2024-01-15T10:30:45Z" |
|
|
|
|
|
# Model will output structured JSON with relevant fields: |
|
|
# { |
|
|
# "event_type": "failed_login", |
|
|
# "source_ip": "192.168.1.100", |
|
|
# "username": "admin", |
|
|
# "timestamp": "2024-01-15T10:30:45Z" |
|
|
# } |
|
|
``` |
|
|
|
|
|
### Downstream Use |
|
|
|
|
|
- Integration into SIEM platforms (Splunk, ELK, QRadar) |
|
|
- Security orchestration and automated response (SOAR) workflows |
|
|
- Threat hunting and incident response automation |
|
|
- Security data lake processing pipelines |
|
|
|
|
|
### Out-of-Scope Use |
|
|
|
|
|
- General-purpose text generation |
|
|
- Non-security related field extraction |
|
|
- Real-time processing without proper input validation |
|
|
- Decision-making for critical security responses without human oversight |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
|
|
### Technical Limitations |
|
|
- Optimized specifically for security log formats seen during training |
|
|
- May struggle with completely novel log formats or schemas |
|
|
- Performance may degrade on logs with unusual encoding or formatting |
|
|
- Quantized version (Q4_K_M) shows 5% accuracy reduction vs FP16 |
|
|
|
|
|
### Security Considerations |
|
|
- Model outputs should be validated before use in automated security workflows |
|
|
- Not suitable for real-time critical security decisions without human oversight |
|
|
- Training data may contain biases from specific security environments |
|
|
- Should not be the sole source of truth for security incident classification |
|
|
|
|
|
### Recommendations |
|
|
|
|
|
- Always validate model outputs in production security environments |
|
|
- Implement fallback mechanisms for handling novel or malformed inputs |
|
|
- Regular retraining recommended as new log formats emerge |
|
|
- Use FP16 version for maximum accuracy, Q4_K_M for resource-constrained deployments |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
import torch |
|
|
|
|
|
# Load the model and tokenizer |
|
|
tokenizer = AutoTokenizer.from_pretrained("LLMSIEM/logem") |
|
|
model = AutoModelForCausalLM.from_pretrained("LLMSIEM/logem") |
|
|
|
|
|
# Example usage |
|
|
prompt = "Extract security fields from the following log: [your log here]" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
|
|
|
with torch.no_grad(): |
|
|
outputs = model.generate( |
|
|
inputs.input_ids, |
|
|
max_length=512, |
|
|
temperature=0.1, |
|
|
do_sample=False, |
|
|
pad_token_id=tokenizer.eos_token_id |
|
|
) |
|
|
|
|
|
result = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
print(result) |
|
|
``` |
|
|
|
|
|
### Using with Ollama (Recommended for Production) |
|
|
|
|
|
```bash |
|
|
# Pull the quantized version |
|
|
ollama pull LLMSIEM/logem |
|
|
|
|
|
# Run inference |
|
|
ollama run LLMSIEM/logem "Extract fields from: SSH login from 10.0.0.5 by root" |
|
|
``` |
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
The model was fine-tuned on a curated dataset of security logs and corresponding structured field extractions, including: |
|
|
- Network security events (firewall, IDS/IPS) |
|
|
- Authentication logs (successful/failed logins) |
|
|
- System security events (file access, process execution) |
|
|
- Application security logs (web servers, databases) |
|
|
|
|
|
Dataset characteristics: |
|
|
- 21 standardized test cases for evaluation |
|
|
- Diverse log formats and security event types |
|
|
- JSON-formatted target outputs for structured field extraction |
|
|
|
|
|
### Training Procedure |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
|
|
- **Base model:** Qwen3-0.6B |
|
|
- **Training regime:** Mixed precision (fp16) |
|
|
- **Fine-tuning approach:** Supervised fine-tuning on field extraction tasks |
|
|
- **Optimization:** Task-specific training for SIEM applications |
|
|
|
|
|
#### Model Variants |
|
|
|
|
|
- **FP16 Version:** 1.2 GB, maximum accuracy (0.833 F1) |
|
|
- **Q4_K_M Quantized:** 396 MB, production-optimized (0.800 F1) |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
|
|
#### Testing Data |
|
|
- 21 standardized security log parsing test cases |
|
|
- Diverse log formats from multiple security tools |
|
|
- Ground truth structured outputs for comparison |
|
|
|
|
|
#### Metrics |
|
|
- **Perfect Match Rate:** Percentage of test cases with 100% accurate field extraction |
|
|
- **F1 Score:** Harmonic mean of precision and recall for field detection |
|
|
- **Precision:** Accuracy of extracted fields |
|
|
- **Response Time:** Average inference latency |
|
|
|
|
|
### Results |
|
|
|
|
|
| Model | Perfect Matches | Avg F1 | Precision | Speed | Size | |
|
|
|-------|----------------|---------|-----------|-------|------| |
|
|
| **LLMSIEM/logem (FP16)** | **14/21 (66.7%)** | **0.833** | **0.848** | **1.00s** | **1.2 GB** | |
|
|
| LLMSIEM/logem (Q4_K_M) | 13/21 (61.9%) | 0.800 | 0.819 | 1.00s | 396 MB | |
|
|
| Gemma:12B | 15/21 (71.4%) | 0.790 | 0.788 | 3.06s | 5.0 GB | |
|
|
| Qwen3:0.6B (base) | 9/21 (42.9%) | 0.651 | 0.636 | 1.57s | 522 MB | |
|
|
|
|
|
#### Key Findings |
|
|
- **+28% F1 improvement** over base Qwen3-0.6B model |
|
|
- **Outperforms 12B models** in F1 score despite being 20x smaller |
|
|
- **3x faster** than comparable accuracy models |
|
|
- **12.6x smaller** than Gemma while maintaining superior performance |
|
|
|
|
|
## Environmental Impact |
|
|
|
|
|
Training a specialized 0.6B parameter model requires significantly less computational resources compared to training larger models from scratch: |
|
|
|
|
|
- **Hardware Type:** NVIDIA GPU (RTX3060) |
|
|
- **Training approach:** Fine-tuning (more efficient than training from scratch) |
|
|
- **Base model efficiency:** Starting from pre-trained Qwen3-0.6B reduces carbon footprint |
|
|
- **Production efficiency:** Smaller model size reduces inference energy consumption |
|
|
|
|
|
## Technical Specifications |
|
|
|
|
|
### Model Architecture |
|
|
- **Architecture:** Transformer decoder (Qwen3 family) |
|
|
- **Parameters:** 0.6 billion |
|
|
- **Context length:** [Inherited from Qwen3-0.6B] |
|
|
- **Vocabulary size:** [Inherited from Qwen3-0.6B] |
|
|
|
|
|
### Compute Infrastructure |
|
|
- **Training:** Fine-tuning on security-specific datasets |
|
|
- **Inference:** Optimized for CPU and GPU deployment |
|
|
- **Quantization:** GGML Q4_K_M for edge deployment |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you use this model in your research or applications, please cite: |
|
|
|
|
|
```bibtex |
|
|
@misc{llmsiem-logem-2025, |
|
|
title={LLMSIEM/logem: A Fine-tuned Language Model for Security Log Analysis}, |
|
|
author={[Hassan Shehata]}, |
|
|
year={2025}, |
|
|
url={https://huggingface.co/LLMSIEM/logem}, |
|
|
note={Fine-tuned from Qwen3-0.6B for SIEM applications} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Model Card Authors |
|
|
|
|
|
[Hassan Shehata/LLMSIEM] |
|
|
|
|
|
## Model Card Contact |
|
|
|
|
|
For questions about this model, please contact: |
|
|
- **Email:** [hassanshehata25895@gmail.com] |
|
|
- **LinkedIn:** [https://www.linkedin.com/in/hassan-shehata-503272172/] |
|
|
- **GitHub:** [[Your GitHub Profile](https://github.com/HassanShehata)] |
|
|
|
|
|
--- |
|
|
|
|
|
*This model is part of the LLMSIEM research series exploring the application of Large Language Models in cybersecurity and SIEM workflows.* |
