Debashis commited on
Commit
94cbcd3
Β·
1 Parent(s): d5dc3a0

Complete README with full configuration guide, examples, and FAQ

Browse files
Files changed (1) hide show
  1. README.md +239 -91
README.md CHANGED
@@ -2,151 +2,299 @@
2
 
3
  An LLM-powered security log and incident analysis tool with a production-ready web interface.
4
 
5
- ## What It Does
6
 
7
- - **Paste logs or security alerts** β†’ LLM analyzes and explains
8
- - **Intelligence extraction**: What happened, severity level, remediation steps
9
- - **Director-level insights**: Clear, actionable narrative for decision-makers
10
- - **No training required**: Works out of the box with OpenAI, local LLMs, or mock inference
 
11
 
12
- ## πŸš€ Quick Start
13
 
14
- **Try it live:** https://huggingface.co/spaces/debashis2007/SecurityIncidentAnalyzer
15
 
16
- Or run locally:
 
 
17
 
18
  ```bash
19
- # Install dependencies
 
20
  pip install -r requirements.txt
21
-
22
- # Set up environment variables
23
  cp .env.example .env
24
- # Edit .env with your OpenAI API key or leave as mock for demo
25
-
26
- # Run the application
27
- python src/app.py
28
  ```
29
 
30
- Visit `http://localhost:7860` in your browser.
31
-
32
- ## Architecture
33
 
34
  ```
35
  src/
36
- β”œβ”€β”€ app.py # Gradio interface & main entry
37
- β”œβ”€β”€ llm/
38
- β”‚ β”œβ”€β”€ provider.py # LLM provider abstraction (OpenAI, local, mock)
39
- β”‚ └── prompts.py # Security analysis prompt templates
40
  β”œβ”€β”€ analyzer/
41
- β”‚ β”œβ”€β”€ security.py # Core incident analysis logic
42
- β”‚ └── models.py # Data models for analysis results
43
- β”œβ”€β”€ utils/
44
- β”‚ β”œβ”€β”€ logger.py # Logging configuration
45
- β”‚ └── config.py # Configuration management
46
- └── __init__.py
 
 
 
 
 
47
  ```
48
 
49
- ## Key Design Decisions
50
 
51
- 1. **Provider Abstraction**: LLM provider logic separated from analysis logic (see `src/llm/provider.py`)
52
- 2. **Structured Outputs**: Analysis results use Pydantic models for validation and CLI output
53
- 3. **Environment-based Config**: Support for API keys, model selection, and prompt templates
54
- 4. **Minimal Dependencies**: Gradio + requests/httpx only (no heavyweight frameworks)
55
 
56
- ## Testing
57
 
58
  ```bash
59
- pytest tests/
60
- pytest --cov=src tests/ # With coverage
61
  ```
62
 
63
- ## Deployment
64
 
65
- ### Hugging Face Spaces (Recommended)
66
- The app is automatically deployed! Just visit:
67
- πŸ‘‰ **https://huggingface.co/spaces/debashis2007/SecurityIncidentAnalyzer**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
 
69
- HF Spaces detects and auto-configures:
70
- - βœ… `spaces.yaml` β†’ Gradio app configuration
71
- - βœ… `Procfile` β†’ Launch command
72
- - βœ… `requirements.txt` β†’ Dependencies
 
 
73
 
74
- **Optional: Add OpenAI API Key to HF Spaces**
75
- 1. Go to Space Settings β†’ Secrets
76
- 2. Add `OPENAI_API_KEY` = your-key
77
- 3. Update `.env` locally: `LLM_PROVIDER=openai`
78
- 4. Push changes (Space auto-redeploys)
79
 
80
- ### Local Deployment
81
  ```bash
82
- # Set your provider
83
- export LLM_PROVIDER=openai # or mock, local
84
- export OPENAI_API_KEY=your-key
85
 
86
- # Start the app
87
- python src/app.py
 
 
 
 
 
 
88
  ```
89
 
90
- ## Environment Variables
91
 
92
- - `OPENAI_API_KEY`: Your OpenAI API key
93
- - `LLM_PROVIDER`: `openai`, `local`, or `mock` (default: `mock`)
94
- - `LLM_MODEL`: Model name (e.g., `gpt-4-turbo`, defaults provided per provider)
95
- - `DEBUG`: Set to `true` for verbose logging
96
 
97
- ### For HF Spaces
98
- Add secrets via Settings β†’ Secrets instead of `.env`:
99
- - `OPENAI_API_KEY` for OpenAI
100
- - `LLM_PROVIDER` (optional, defaults to mock)
101
- - `DEBUG` (optional)
 
 
102
 
103
- ### For Local Development
104
- Copy `.env.example` to `.env` and edit:
105
  ```bash
106
- cp .env.example .env
107
- # Edit with your keys/settings
108
- ```
109
 
110
- **Important:** `.env` is in `.gitignore` β€” never commit secrets!
 
111
 
112
- ## Example Security Incidents
 
 
 
 
113
 
114
- Try these in the app:
115
 
116
- ### Failed Authentication
117
  ```
118
  2025-12-21 14:32:15 AUTH_FAILURE - Failed login attempt from 192.168.1.100
119
- User account: admin@company.com
120
- Attempts: 15 in 2 minutes
 
121
  ```
122
 
123
- ### Ransomware Detection
124
  ```
125
  CRITICAL ALERT - File encryption detected
126
- Files Encrypted: 500+ files
127
- Process: unknown.exe running as SYSTEM
 
 
128
  Status: ACTIVE THREAT
129
  ```
130
 
131
- ### SQL Injection
132
  ```
133
  Web Application Firewall Alert
134
- Rule Triggered: SQL Injection Pattern
135
- Payload: ' OR '1'='1'
136
- Status: Blocked
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
137
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
 
139
- ## Architecture for AI Developers
 
 
 
140
 
141
- See `.github/copilot-instructions.md` for comprehensive AI agent onboardingβ€”covers:
142
- - Component architecture and data flows
143
- - Critical patterns (provider abstraction, config management, regex parsing)
144
- - Extension points for new providers and features
145
- - Testing strategy and troubleshooting
146
 
147
- ## License
148
 
149
- MIT
150
 
151
  ## Contributing
152
 
 
2
 
3
  An LLM-powered security log and incident analysis tool with a production-ready web interface.
4
 
5
+ ## 🎯 What It Does
6
 
7
+ Paste security logs or alerts β†’ Get immediate AI analysis explaining:
8
+ - **What happened** β€” Clear incident summary
9
+ - **Severity level** β€” CRITICAL, HIGH, MEDIUM, LOW, or INFO
10
+ - **Suggested remediation** β€” Actionable next steps
11
+ - **Key indicators** β€” IOCs and anomalies detected
12
 
13
+ No training required. Director-level insights in seconds.
14
 
15
+ ## πŸš€ Try It Now
16
 
17
+ **Live Demo:** https://huggingface.co/spaces/debashis2007/SecurityIncidentAnalyzer
18
+
19
+ Or run locally (5 minutes):
20
 
21
  ```bash
22
+ git clone https://github.com/Debashis2007/SecurityIncidentAnalyzer.git
23
+ cd SecurityIncidentAnalyzer
24
  pip install -r requirements.txt
 
 
25
  cp .env.example .env
26
+ python src/app.py # Visit http://localhost:7860
 
 
 
27
  ```
28
 
29
+ ## πŸ—οΈ Architecture
 
 
30
 
31
  ```
32
  src/
33
+ β”œβ”€β”€ app.py # Gradio web UI entry point
 
 
 
34
  β”œβ”€β”€ analyzer/
35
+ β”‚ β”œβ”€β”€ security.py # IncidentAnalyzer: core analysis logic
36
+ β”‚ └── models.py # RiskLevel enum, SecurityAnalysis dataclass
37
+ β”œβ”€β”€ llm/
38
+ β”‚ β”œβ”€β”€ provider.py # LLM provider abstraction
39
+ β”‚ β”‚ # - OpenAIProvider (gpt-4-turbo)
40
+ β”‚ β”‚ # - LocalLLMProvider (Ollama)
41
+ β”‚ β”‚ # - MockLLMProvider (demo)
42
+ β”‚ └── prompts.py # Security analysis prompt templates
43
+ └── utils/
44
+ β”œβ”€β”€ config.py # Environment-driven configuration
45
+ └── logger.py # Structured logging
46
  ```
47
 
48
+ ## βš™οΈ Configuration
49
 
50
+ ### Option 1: Mock LLM (Default - No API Key Required)
 
 
 
51
 
52
+ Perfect for demos and testing:
53
 
54
  ```bash
55
+ # No setup needed! Just run:
56
+ python src/app.py
57
  ```
58
 
59
+ The mock provider returns realistic-looking security analysis without any API calls.
60
 
61
+ ### Option 2: OpenAI GPT-4 Turbo
62
+
63
+ For production analysis with real LLM intelligence:
64
+
65
+ 1. **Get an API key:**
66
+ - Sign up: https://platform.openai.com
67
+ - Create key: https://platform.openai.com/account/api-keys
68
+ - Keep it safe!
69
+
70
+ 2. **Configure locally:**
71
+ ```bash
72
+ cp .env.example .env
73
+ # Edit .env:
74
+ # LLM_PROVIDER=openai
75
+ # OPENAI_API_KEY=sk-proj-YOUR_KEY_HERE
76
+ python src/app.py
77
+ ```
78
+
79
+ 3. **On Hugging Face Spaces:**
80
+ - Settings β†’ Secrets β†’ Add:
81
+ - `OPENAI_API_KEY` = your key
82
+ - `LLM_PROVIDER` = openai
83
+
84
+ ### Option 3: Local LLM (Ollama)
85
+
86
+ Run a local model without cloud API:
87
+
88
+ 1. **Install Ollama:** https://ollama.ai
89
+ 2. **Pull a model:**
90
+ ```bash
91
+ ollama pull mistral:7b
92
+ ollama serve # Keep running in another terminal
93
+ ```
94
 
95
+ 3. **Configure:**
96
+ ```bash
97
+ # .env
98
+ LLM_PROVIDER=local
99
+ LLM_MODEL=mistral:7b
100
+ ```
101
 
102
+ ## πŸ“‹ Environment Variables
103
+
104
+ Create a `.env` file (see `.env.example`):
 
 
105
 
 
106
  ```bash
107
+ # Provider selection (required)
108
+ LLM_PROVIDER=mock # mock, openai, or local
 
109
 
110
+ # OpenAI (only needed if LLM_PROVIDER=openai)
111
+ OPENAI_API_KEY=sk-proj-...
112
+
113
+ # Model override (optional, uses defaults if not set)
114
+ LLM_MODEL=gpt-4-turbo
115
+
116
+ # Debugging
117
+ DEBUG=false
118
  ```
119
 
120
+ **⚠️ Important:** `.env` is in `.gitignore` β€” secrets never get committed.
121
 
122
+ ### Provider Defaults
 
 
 
123
 
124
+ | Provider | Default Model | Notes |
125
+ |----------|---------------|-------|
126
+ | `mock` | mock-analyzer-v1 | No API required, deterministic |
127
+ | `openai` | gpt-4-turbo | Requires OPENAI_API_KEY |
128
+ | `local` | mistral:7b | Requires Ollama running on localhost:11434 |
129
+
130
+ ## πŸ§ͺ Testing
131
 
 
 
132
  ```bash
133
+ # Run all tests
134
+ pytest tests/ -v
 
135
 
136
+ # With coverage report
137
+ pytest tests/ --cov=src
138
 
139
+ # Test specific module
140
+ pytest tests/test_analyzer.py -v
141
+ ```
142
+
143
+ **Test Results:** 11/11 tests passing βœ…
144
 
145
+ ## πŸ“ Example Incidents to Test
146
 
147
+ ### 1. Failed Authentication
148
  ```
149
  2025-12-21 14:32:15 AUTH_FAILURE - Failed login attempt from 192.168.1.100
150
+ 2025-12-21 14:32:18 AUTH_FAILURE - Failed login attempt from 192.168.1.100
151
+ 2025-12-21 14:32:21 AUTH_FAILURE - Failed login attempt from 192.168.1.100
152
+ User: admin@company.com | Attempts: 15 in 2 minutes
153
  ```
154
 
155
+ ### 2. Ransomware Detection
156
  ```
157
  CRITICAL ALERT - File encryption detected
158
+ Directory: /mnt/backup/production
159
+ Files Encrypted: 500+
160
+ Process: unknown.exe (SYSTEM privilege)
161
+ Time: 2025-12-21 16:20:15 UTC
162
  Status: ACTIVE THREAT
163
  ```
164
 
165
+ ### 3. SQL Injection Attempt
166
  ```
167
  Web Application Firewall Alert
168
+ Rule: SQL Injection Pattern
169
+ URL: /api/users?id=1' OR '1'='1
170
+ Source IP: 203.0.113.45
171
+ Status: BLOCKED
172
+ Payload: ' OR '1'='1
173
+ ```
174
+
175
+ ### 4. Privilege Escalation
176
+ ```
177
+ Security Event: Privilege Escalation Detected
178
+ User: john.smith@company.com
179
+ Action: sudo su - (unauthorized)
180
+ Target System: prod-db-server-02
181
+ Time: 2025-12-21 17:02:45
182
+ Status: No approval ticket found
183
+ ```
184
+
185
+ ### 5. Suspicious Outbound Traffic
186
  ```
187
+ ALERT: Unusual outbound traffic detected
188
+ Destination: 10.0.0.1:4444
189
+ Source: internal-server-03
190
+ Data Volume: 2.3 GB in 45 minutes
191
+ Status: ONGOING
192
+ ```
193
+
194
+ ## πŸš€ Deployment
195
+
196
+ ### Hugging Face Spaces (Recommended)
197
+
198
+ Your app is live at:
199
+ πŸ‘‰ https://huggingface.co/spaces/debashis2007/SecurityIncidentAnalyzer
200
+
201
+ **How it works:**
202
+ - Detects `spaces.yaml` β†’ Gradio configuration
203
+ - Installs `requirements.txt` β†’ All dependencies
204
+ - Runs `src/app.py` β†’ Your app starts automatically
205
+
206
+ **To add OpenAI:**
207
+ 1. Settings β†’ Secrets
208
+ 2. Add `OPENAI_API_KEY=sk-proj-YOUR_KEY`
209
+ 3. Change `.env`: `LLM_PROVIDER=openai`
210
+ 4. Push to git (auto-redeploys)
211
+
212
+ ### Docker (Local or Cloud)
213
+
214
+ ```dockerfile
215
+ FROM python:3.9-slim
216
+ WORKDIR /app
217
+ COPY requirements.txt .
218
+ RUN pip install -r requirements.txt
219
+ COPY . .
220
+ ENV LLM_PROVIDER=mock
221
+ CMD python src/app.py
222
+ ```
223
+
224
+ ```bash
225
+ docker build -t security-analyzer .
226
+ docker run -p 7860:7860 security-analyzer
227
+ ```
228
+
229
+ ### Traditional Server
230
+
231
+ ```bash
232
+ # Production with gunicorn
233
+ pip install gunicorn
234
+ gunicorn -w 4 -b 0.0.0.0:7860 src.app:demo
235
+ ```
236
+
237
+ ## πŸ”§ For AI Developers
238
+
239
+ See `.github/copilot-instructions.md` for comprehensive agent onboarding:
240
+ - **Architecture patterns** β€” Provider abstraction, config management, regex parsing
241
+ - **Data flows** β€” How logs become analyzed incidents
242
+ - **Extension points** β€” Add new LLM providers, customize analysis
243
+ - **Testing strategy** β€” Unit, integration, E2E with mocks
244
+ - **Troubleshooting** β€” Common issues and solutions
245
+
246
+ ## πŸ“¦ Requirements
247
+
248
+ - **Python:** 3.9+
249
+ - **Core:** Gradio, Pydantic, httpx, python-dotenv
250
+ - **Optional:** OpenAI SDK (for OpenAI provider only)
251
+
252
+ See `requirements.txt` for exact versions.
253
+
254
+ ## 🧠 Key Design Decisions
255
+
256
+ 1. **Provider Abstraction** β€” LLM logic decoupled from analysis
257
+ 2. **Environment Config** β€” All settings via env vars, no hardcoding
258
+ 3. **Regex Parsing** β€” Flexible response extraction (works with any LLM format)
259
+ 4. **Async/Await** β€” Non-blocking I/O for responsive UI
260
+ 5. **Structured Types** β€” Pydantic models for validation and safety
261
+
262
+ ## βš–οΈ License
263
+
264
+ MIT β€” Use freely, modify, distribute.
265
+
266
+ ## 🀝 Contributing
267
+
268
+ ```bash
269
+ # Fork, create a branch, make changes
270
+ git checkout -b feature/my-improvement
271
+ pytest tests/ -v # Ensure tests pass
272
+ git push origin feature/my-improvement
273
+ # Create a Pull Request
274
+ ```
275
+
276
+ ## ❓ FAQ
277
+
278
+ **Q: Can I use this without an API key?**
279
+ A: Yes! The default `mock` provider needs no API key and is perfect for demos.
280
+
281
+ **Q: How accurate is the analysis?**
282
+ A: Depends on the LLM. Mock is deterministic for testing. OpenAI (GPT-4) provides production-grade analysis.
283
+
284
+ **Q: Can I run this offline?**
285
+ A: Yes! Use the `local` provider with Ollama on your own hardware.
286
 
287
+ **Q: Is my data safe?**
288
+ A: - **Mock provider:** No external calls, stays local
289
+ - **Local provider:** Stays on your machine
290
+ - **OpenAI provider:** Subject to OpenAI's privacy policy
291
 
292
+ **Q: How do I report bugs?**
293
+ A: Open an issue on GitHub with reproduction steps.
 
 
 
294
 
295
+ ---
296
 
297
+ **Questions?** Check `.github/copilot-instructions.md` or create an issue!
298
 
299
  ## Contributing
300