docs/threat_model.md

Threat Model — Explainable IDS

1. System Description

An ML-based Intrusion Detection System (IDS) monitors network traffic and classifies connections as Normal or one of four attack categories (DoS, Probe, R2L, U2R). The system uses post-hoc explainability methods (SHAP, LIME) to provide security analysts with interpretable justifications for each alert.

2. Assets Under Protection

Asset	Value	Sensitivity
Network integrity	High	Disruption → service outage
IDS model parameters	Medium	Leak → evasion knowledge
SHAP/LIME explanations	Medium	Leak → feature manipulation strategy
Training data statistics	Low-Medium	Leak → distribution knowledge for crafting attacks

3. Adversary Profiles

Adversary A: Network Attacker (External)

• Goal: Bypass IDS detection — send malicious traffic classified as "Normal"
• Capabilities: Can craft and modify network packets (control over src_bytes, dst_bytes, duration, protocol, count, etc.)
• Knowledge: Black-box (no model access) or Grey-box (knows model type + feature set)
• Constraints: Cannot modify all features — some are protocol-determined (e.g., protocol_type, flag) or network-infrastructure-bound (e.g., dst_host_count depends on actual connections)

Adversary B: Explanation Exploiter (Internal/External)

• Goal: Use SHAP/LIME output to learn which features the model relies on, then craft evasion attacks
• Capabilities: Can query the model and observe explanations (e.g., deployed as analyst dashboard)
• Knowledge: White-box on explanations, grey-box on model
• Attack: Query with diverse inputs → aggregate SHAP values → identify top features → manipulate those features in attack traffic

Adversary C: Training Data Poisoner (Supply Chain)

• Goal: Insert backdoor so model shows clean explanations but misclassifies triggered inputs
• Capabilities: Can inject samples into training set
• Relevance: Even explanations can be fooled if the model itself is compromised (Baniecki et al., 2022)

4. Feature Manipulability Analysis

Critical for realistic adversarial evaluation — not all 41 NSL-KDD features can be freely modified by an attacker.

Feature Category	Manipulable?	Examples	Justification
Packet content	✅ Yes	src_bytes, dst_bytes, hot, num_failed_logins	Attacker controls payload
Connection behavior	⚠️ Partially	duration, count, srv_count	Attacker can slow/speed connections but within limits
Protocol fields	⚠️ Constrained	protocol_type, flag	Must be valid TCP/UDP/ICMP; flag must match connection state
Network statistics	❌ No	dst_host_count, dst_host_srv_count	Aggregated by IDS sensor, not attacker-controlled
Error rates	⚠️ Partially	serror_rate, rerror_rate	Attacker can trigger errors but rates depend on overall traffic

Implication for SHAP/LIME: If the model relies heavily on non-manipulable features (dst_host_count, dst_host_same_srv_rate), it is more robust against evasion. If it relies on manipulable features (src_bytes, duration), evasion is easier.

5. Attack Scenarios

Scenario 1: Evasion via Explanation Leakage

1. Attacker queries IDS explanation API with known attack samples
2. SHAP reveals serror_rate (weight=0.45) and count (weight=0.32) are top features for DoS detection
3. Attacker crafts DoS traffic with low serror_rate (connection completion spoofing) and varied count
4. IDS misclassifies as Normal

Scenario 2: LIME Instability Exploitation

1. LIME produces different top features for the same input across runs (stochastic)
2. Analyst sees Feature A as top in run 1, Feature B in run 2
3. Inconsistent investigation → missed detections or wasted resources

Scenario 3: Backdoor with Clean Explanations

1. Poisoned training data contains trigger pattern (e.g., specific src_bytes + service combination)
2. Model correctly classifies and explains normal traffic
3. On triggered inputs: misclassifies as Normal AND SHAP shows plausible benign features
4. Analyst trusts explanation → attack goes undetected

6. Security Requirements

Requirement	Priority	Mitigation
Explanation access control	High	Rate-limit explanation API, log queries
Explanation consistency	High	Prefer SHAP (deterministic) over LIME for critical decisions
Model integrity verification	Medium	Track training data provenance, validate model fingerprints
Robust feature reliance	Medium	Verify model doesn't over-rely on manipulable features
Defense-in-depth	High	Explanations supplement (don't replace) rule-based IDS

7. Assumptions & Scope

• NSL-KDD is a benchmark dataset — real deployment would require domain-specific feature analysis
• We evaluate post-hoc explainability only (not inherently interpretable models)
• We focus on explanation reliability, not adversarial robustness of the classifier itself (that's Project 1)