README.md

---
library_name: peft
base_model: mistralai/Mistral-7B-v0.1
tags:
- cybersecurity
- federated-learning
- security
- lora
- threat-intelligence
- incident-response
license: apache-2.0
---

# FedDAPT Security v1

A domain-adapted security LLM trained using federated learning across simulated multi-tenant security environments. Built on Mistral-7B with QLoRA adapters.

## What this model does

Specializes in cybersecurity tasks including incident summarization, alert triage, and threat intelligence analysis. Trained without centralizing any organization's private security data.

## Results

| Method | ROUGE-L |
|---|---|
| Zero-shot Mistral-7B | 0.367 |
| Centralized DAPT | 0.330 |
| **FedDAPT (this model)** | **0.707** |

FedDAPT achieved 2.1x improvement over centralized training on incident summarization.

## Quick Start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True, bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.bfloat16,
    ),
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "dsuyu1/FedDAPT-security-v1")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")

prompt = """### Instruction:
Summarize the following security incident in one sentence.

### Input:
Incident timeline: initial_access: phishing with macro attachment -> execution: PowerShell encoded command -> c2: Cobalt Strike HTTPS beacon -> lateral: SMB + PsExec lateral movement -> impact: ransomware across endpoints.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```

## Training Details

- **Framework:** FedDAPT (Federated Domain-Adaptive Pre-Training)
- **Base model:** Mistral-7B-v0.1
- **Adapter:** LoRA (r=16, alpha=32, targets: q_proj, v_proj)
- **Quantization:** QLoRA 4-bit NF4
- **Aggregation:** FedAvg with FedProx (mu=0.01)
- **Clients:** 3 (endpoint-focused, network-focused, cloud/CTI-focused)
- **Rounds:** 20
- **Corpus:** Curated from MITRE ATT&CK, SigmaHQ, NVD, CISA KEV, MITRE CAR (44,754 raw -> 4,265 curated)
- **Curation:** NVIDIA NeMo Curator (dedup, PII redaction, quality filtering)

## Limitations

- Trained on public proxy data, not real operational telemetry
- Triage accuracy (35%) has room for improvement
- Requires instruction format (### Instruction / ### Input / ### Response)
- 7B parameter model requires GPU for inference

## Citation

Villarreal, D. "Smarter SecOps: Leveraging Private, Federated Transfer Learning" BSides RGV 2026.