docs/project_plan.md

Project Plan — Explainable IDS

1. Problem Statement

Intrusion Detection Systems (IDS) powered by deep learning achieve high detection rates but operate as black boxes. In security-critical environments, analysts need to understand why a connection is flagged as malicious — not just that it is. This project addresses three key questions:

1. Can we explain IDS decisions using post-hoc methods (SHAP, LIME)?
2. Are these explanations stable — do similar inputs produce similar explanations?
3. What are the security risks of making model decisions interpretable?

2. Methodology

Phase 1: Data Understanding & Preprocessing

• Load NSL-KDD dataset (41 features, binary + 5-class labels)
• Encode 3 categorical features (protocol_type, service, flag) via LabelEncoder
• Normalize all features to [0,1] via MinMaxScaler
• Analyze class distribution and document imbalance (especially U2R: ~52 samples, R2L: ~995)

Phase 2: Baseline Model Training

• Primary model: MLP (256→128→64→num_classes) with BatchNorm and Dropout
• Comparison models: LSTM (2-layer, hidden=64) and 1D-CNN (Conv64→Conv128→AvgPool→FC)
• Training: Adam optimizer, lr=1e-3, weight_decay=1e-4, 50 epochs
• Evaluation: Per-class Precision/Recall/F1, Weighted F1, PR-AUC, Confusion Matrix

Phase 3: Explainability Analysis

• SHAP: KernelExplainer (model-agnostic) — compute per-feature attributions for each class
  • Global summary plots (feature importance rankings)
  • Local force plots (individual predictions)
  • Class-specific analysis (which features drive anomaly detection)
• LIME: LimeTabularExplainer
  • Per-instance explanations with top-10 features
  • Compare LIME vs SHAP feature rankings

Phase 4: Explanation Stability Evaluation

• Perturbation stability (SENS_MAX): Add ε-bounded noise (ε=0.01, 0.03, 0.05), measure max attribution shift
• LIME stochastic stability: Run LIME 20 times per sample with different seeds, compute pairwise Spearman rank correlation
• Faithfulness: Mask top-k features identified by SHAP/LIME, measure prediction drop (higher drop = more faithful)
• Threshold: PCC > 0.6 = stable (per SAFARI framework, Huang et al. 2022)

Phase 5: Security Implications Analysis

• Can an attacker use SHAP output to identify which features to manipulate for evasion?
• Is LIME's stochasticity a security concern (inconsistent analyst decisions)?
• Risk of explanation manipulation attacks (backdoored models with clean explanations)

3. Experimental Design (≥3 variations required)

Experiment	Description	Metric
Baseline	MLP on binary NSL-KDD	Weighted F1, PR-AUC
Variation 1	MLP on 5-class NSL-KDD	Per-class F1
Variation 2	LSTM on binary NSL-KDD	Weighted F1 (compare to MLP)
Variation 3	1D-CNN on binary NSL-KDD	Weighted F1 (compare to MLP)
XAI Comparison	SHAP vs LIME feature rankings	Rank correlation, faithfulness
Stability	Explanation stability across ε values	SENS_MAX, PCC

4. Timeline

Phase	Duration	Status
Data preprocessing	1 day	✅ Done
Baseline training	1 day	🔄 In Progress
Explainability	2 days	Pending
Stability eval	1 day	Pending
Security analysis	1 day	Pending
Report writing	2 days	Pending

5. Deliverables

1. Explanation Analysis — SHAP/LIME visualizations with interpretation
2. Security Report — Adversarial risks of exposing explanations
3. Code + README — Fully reproducible pipeline
4. Report (max 10 pages PDF) — All design choices justified