Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,262 +1,262 @@
|
|
| 1 |
-
---
|
| 2 |
-
title: SentinelScan WSS
|
| 3 |
-
emoji: π‘οΈ
|
| 4 |
-
colorFrom: indigo
|
| 5 |
-
colorTo: red
|
| 6 |
-
sdk: docker
|
| 7 |
-
pinned: true
|
| 8 |
-
---
|
| 9 |
-
# π‘οΈ SentinelScan β Website Security Scanner (WSS)
|
| 10 |
-
|
| 11 |
-
SentinelScan is a full-stack, enterprise-grade **Dynamic Application Security Testing (DAST)** platform designed to automate vulnerability detection across target domains and web APIs. Featuring a highly modular architecture, SentinelScan orchestrates a pipeline of custom security scanning agents concurrently, storing findings in a structured database and presenting them in a premium, real-time dashboard.
|
| 12 |
-
|
| 13 |
-
---
|
| 14 |
-
|
| 15 |
-
## π Key Features
|
| 16 |
-
|
| 17 |
-
* **Multi-Agent Concurrency**: Uses an asynchronous thread pool execution model (`ThreadPoolExecutor`) inside Celery tasks to run up to 17 specialized scanner modules in parallel.
|
| 18 |
-
* **Real-time Log Streaming**: Captures and exposes live, color-coded execution logs in-memory, enabling users to monitor active scans line-by-line.
|
| 19 |
-
* **Scheduled Scans**: Leverage Celery Beat to automate recurring scans (daily, weekly, monthly) for regular status monitoring.
|
| 20 |
-
* **Alert Webhooks**: Automatically dispatches security alerts to external services (e.g., Discord, custom webhooks) when critical or high vulnerabilities are discovered.
|
| 21 |
-
* **Authenticated Scanning**: Supports credentials/cookies injection via custom HTTP request headers, bypassing login perimeters to test deep backend routes.
|
| 22 |
-
* **Interactive Remediation**: Offers interactive, language-specific code remediation templates for each identified vulnerability type.
|
| 23 |
-
* **Dynamic PDF Reports**: Generates professional PDF summaries of completed scans, containing detailed risk score matrices and remediation guidelines.
|
| 24 |
-
|
| 25 |
-
---
|
| 26 |
-
|
| 27 |
-
## π Directory Structure
|
| 28 |
-
|
| 29 |
-
```text
|
| 30 |
-
Project-WSS/
|
| 31 |
-
βββ backend/ # Flask Backend Application
|
| 32 |
-
β βββ app/ # Main Flask Application Package
|
| 33 |
-
β β βββ routes/ # REST API Endpoints & Route Blueprints
|
| 34 |
-
β β β βββ auth.py # User Authentication (Login, Register)
|
| 35 |
-
β β β βββ reports.py # PDF Generation and Scan Reports
|
| 36 |
-
β β β βββ scans.py # Scan Configuration, Triggering, Logs
|
| 37 |
-
β β β βββ vulnerabilities.py # Remediation & Vulnerability Queries
|
| 38 |
-
β β βββ scanners/ # Security Engine Modules & Core Pipelines
|
| 39 |
-
β β β βββ __init__.py # Pipeline Definitions and Class Dispatcher
|
| 40 |
-
β β β βββ api_scanner.py # Exposed REST API Route Finder
|
| 41 |
-
β β β βββ base_scanner.py # Abstract Base Class and Shared Log Utilities
|
| 42 |
-
β β β βββ cloud_scanner.py # Public S3/Cloud Storage Auditor
|
| 43 |
-
β β β βββ cors_scanner.py # CORS Misconfigurations Tester
|
| 44 |
-
β β β βββ cve_scanner.py # Vulnerability Database Version Matcher
|
| 45 |
-
β β β βββ directory_scanner.py# Directory/File brute-forcer
|
| 46 |
-
β β β βββ fuzzer_scanner.py # SQLi & XSS Parameter Fuzzer
|
| 47 |
-
β β β βββ headers_scanner.py # HTTP Security Headers & Cache Poisoning
|
| 48 |
-
β β β βββ nmap_scanner.py # Port & Service Banner Scanner (via Nmap)
|
| 49 |
-
β β β βββ nuclei_scanner.py # Nuclei Template-based Scanner
|
| 50 |
-
β β β βββ robots_scanner.py # robots.txt Crawler
|
| 51 |
-
β β β βββ secrets_scanner.py # Page Secrets/API Key Scanner
|
| 52 |
-
β β β βββ sslyze_scanner.py # SSL/TLS Configurations & Ciphers Auditor
|
| 53 |
-
β β β βββ subdomain_scanner.py# Subdomain DNS Enumerator
|
| 54 |
-
β β β βββ tech_scanner.py # Technology Stack Fingerprinting
|
| 55 |
-
β β β βββ waf_scanner.py # WAF Detection & Fingerprinting
|
| 56 |
-
β β β βββ whois_scanner.py # Domain Registrar and Whois Lookup
|
| 57 |
-
β β β βββ zap_scanner.py # OWASP ZAP Active Spider Integration
|
| 58 |
-
β β βββ utils/ # Utility Scripts & Helpers
|
| 59 |
-
β β β βββ pdf_generator.py # ReportLab PDF Generation
|
| 60 |
-
β β β βββ webhook.py # Discord & Webhook Dispatcher
|
| 61 |
-
β β βββ database.py # SQLAlchemy Extension Instance
|
| 62 |
-
β β βββ extensions.py # Rate Limiter & Security Extensions
|
| 63 |
-
β β βββ models.py # SQLAlchemy Database Models (SQLite/PostgreSQL)
|
| 64 |
-
β β βββ scanner.py # Celery Tasks, Beat Schedules & Orchestration
|
| 65 |
-
β βββ celery_app.py # Celery Broker and Beat Scheduler Configuration
|
| 66 |
-
β βββ config.py # Environment Variable Parsing and App Constants
|
| 67 |
-
β βββ requirements.txt # Python Dependencies List
|
| 68 |
-
β βββ run.py # Flask Application Startup Launcher
|
| 69 |
-
β βββ .env # Local Environment Secret Key Configurations
|
| 70 |
-
β
|
| 71 |
-
βββ frontend/ # React Frontend Application (Vite-powered SPA)
|
| 72 |
-
β βββ src/ # React Application Source
|
| 73 |
-
β β ββ
|
| 74 |
-
β β βββ components/ # Reusable UI Components
|
| 75 |
-
β β β βββ AuthContext.jsx # Global JWT Login State & API Interceptor
|
| 76 |
-
β β β βββ CodeBlock.jsx # Syntax-highlighted Remediation Viewer
|
| 77 |
-
β β β βββ Layout.jsx # Dashboard App Shell & Navigation Sidebar
|
| 78 |
-
β β β βββ ProtectedRoute.jsx # Auth Check Router Wrapper
|
| 79 |
-
β β β βββ ThreatGauge.jsx # SVG Semi-circle Security Score Indicator
|
| 80 |
-
β β βββ pages/ # Top-level Routing View Pages
|
| 81 |
-
β β β βββ Dashboard.jsx # Overview, Scan Metrics, and Status Cards
|
| 82 |
-
β β β βββ LandingPage.jsx # Modern Dark Mode Promotional Marketing Page
|
| 83 |
-
β β β βββ Login.jsx # Clean Secure Authentication Portal
|
| 84 |
-
β β β βββ NewScan.jsx # Target, Pipeline and Cookie Configurations
|
| 85 |
-
β β β βββ Register.jsx # Account Creation Portal
|
| 86 |
-
β β β βββ ReportsHistory.jsx # Past Scan Lists and Export Center
|
| 87 |
-
β β β βββ ScanResults.jsx # Vulnerability breakdown & Live terminal logs
|
| 88 |
-
β β β βββ Settings.jsx # Notification threshold & webhook configuration
|
| 89 |
-
β β βββ App.css # Main Layout styling
|
| 90 |
-
β β βββ App.jsx # Routing configuration
|
| 91 |
-
β β βββ index.css # Global theme tokens, inputs, animations
|
| 92 |
-
β β βββ main.jsx # DOM Injection root
|
| 93 |
-
β β βββ mockApi.js # Standalone local frontend mock testing DB
|
| 94 |
-
β β βββ theme.css # Precision Sentinel palette values
|
| 95 |
-
β βββ vite.config.js # React Hot Module Reloading server options
|
| 96 |
-
β βββ package.json # Frontend NPM scripts & dependencies
|
| 97 |
-
β
|
| 98 |
-
βββ docker-compose.yml # Multi-container orchestrator (Redis service)
|
| 99 |
-
```
|
| 100 |
-
|
| 101 |
-
---
|
| 102 |
-
|
| 103 |
-
## ποΈ System Architecture
|
| 104 |
-
|
| 105 |
-
```mermaid
|
| 106 |
-
graph TD
|
| 107 |
-
User([Security Auditor]) -->|Browser| FE[React Frontend SPA]
|
| 108 |
-
FE -->|API Requests| BE[Flask Web Backend]
|
| 109 |
-
BE -->|Store Scans/Vulns| DB[(SQLite / PostgreSQL)]
|
| 110 |
-
BE -->|Enqueue Jobs| Redis[(Redis Broker)]
|
| 111 |
-
Celery[Celery Task Workers] -->|Dequeue Jobs| Redis
|
| 112 |
-
Celery -->|Write Live Logs| MemLog[(In-Memory Logs)]
|
| 113 |
-
Celery -->|Execute Scanners Concurrently| Scanners{Scanner Suite}
|
| 114 |
-
Scanners -->|Target Requests| Target[Target System]
|
| 115 |
-
Scanners -->|Persist Findings| DB
|
| 116 |
-
Celery -->|Trigger Alert| Webhook[Webhook Notification]
|
| 117 |
-
```
|
| 118 |
-
|
| 119 |
-
### Backend Components
|
| 120 |
-
1. **Flask (REST API)**: Exposes endpoints for managing accounts, starting scans, listing results, downloading PDFs, and tracking setting updates.
|
| 121 |
-
2. **Celery Worker**: Dequeues scan tasks and runs them asynchronously.
|
| 122 |
-
3. **ThreadPoolExecutor**: Multi-threads individual scanners inside a Celery task.
|
| 123 |
-
4. **Celery Beat**: Runs continuously to process scheduled periodic scans.
|
| 124 |
-
5. **Redis**: Acts as the fast in-memory message broker.
|
| 125 |
-
|
| 126 |
-
---
|
| 127 |
-
|
| 128 |
-
## ποΈ Database Schema
|
| 129 |
-
|
| 130 |
-
The database schema, defined in `backend/app/models.py`, includes five main tables:
|
| 131 |
-
|
| 132 |
-
1. **`User`**: Manages credential hashing (via `bcrypt`) and session links.
|
| 133 |
-
2. **`Scan`**: Details the target domain, scan mode (Quick, Standard, Deep), authorization headers, overall security score, scan status, and timings.
|
| 134 |
-
3. **`Vulnerability`**: Stores findings linked to a scan. Contains details like CVSS score, severity classification, category, description, and copy-pasteable remediation snippets.
|
| 135 |
-
4. **`ScheduledScan`**: Saves user-configured scanning intervals (daily, weekly, monthly) for targets.
|
| 136 |
-
5. **`AlertSettings`**: Manages notification flags, webhook URL destinations, and minimum severity thresholds.
|
| 137 |
-
|
| 138 |
-
---
|
| 139 |
-
|
| 140 |
-
## βοΈ Scan Pipelines
|
| 141 |
-
|
| 142 |
-
Pipeline routes are configured in `backend/app/scanners/__init__.py`. Depending on the target criticality and scan duration limits, auditors choose between:
|
| 143 |
-
|
| 144 |
-
| Pipeline | Target Speed | Underlying Scanner Suite | Description |
|
| 145 |
-
| :--- | :--- | :--- | :--- |
|
| 146 |
-
| **`Quick`** | ~30 seconds | Headers, Nmap (top 100 ports), SSLyze, Tech stack, WHOIS, WAF | Surface audit for standard misconfigurations |
|
| 147 |
-
| **`Standard`** | ~2β3 minutes | Quick + SQLi/XSS Fuzzer, Subdomains, API pathways, Cloud, Secrets, CVEs | Comprehensive assessment of application business logic |
|
| 148 |
-
| **`Deep`** | ~10β15 minutes| Standard + CORS, robots.txt, Directory brute-force, Nuclei, ZAP (active) | Deep crawling and automated vulnerability exploitation |
|
| 149 |
-
| **`SSL`** | ~15 seconds | SSLyze, Headers | SSL certificate validation and cipher security audit |
|
| 150 |
-
| **`Port`** | ~45 seconds | Nmap (standard 1000 ports) | Port and network service banner reconnaissance |
|
| 151 |
-
|
| 152 |
-
---
|
| 153 |
-
|
| 154 |
-
## π οΈ The Scanner Suite (17 Specialized Modules)
|
| 155 |
-
|
| 156 |
-
Each scanner inherits from `BaseScanner` (`backend/app/scanners/base_scanner.py`) which coordinates logging, domain parsing, and vulnerability formatting:
|
| 157 |
-
|
| 158 |
-
1. **Headers Scanner (`headers_scanner.py`)**: Checks HTTP security headers (HSTS, CSP, CORS, X-Frame-Options, permissions, Referrer policy) and runs a custom check for HTTP host parameter cache poisoning.
|
| 159 |
-
2. **Nmap Scanner (`nmap_scanner.py`)**: Fires `nmap` commands directly via sub-process, checking exposed network services and testing for vulnerabilities using script scanning banners.
|
| 160 |
-
3. **SSLyze Scanner (`sslyze_scanner.py`)**: Audits SSL certificates, verifying trust status, expiration, and highlighting weak legacy protocols (TLS 1.0, SSLv3).
|
| 161 |
-
4. **Tech Scanner (`tech_scanner.py`)**: Fingerprints backend technologies, libraries, servers, and frameworks.
|
| 162 |
-
5. **Whois Scanner (`whois_scanner.py`)**: Looks up registrar information, IP ownership, and registration details.
|
| 163 |
-
6. **WAF Scanner (`waf_scanner.py`)**: Detects the presence of firewalls (Cloudflare, AWS WAF, ModSecurity, etc.) by inspecting response indicators.
|
| 164 |
-
7. **CORS Scanner (`cors_scanner.py`)**: Audits cross-origin resource sharing declarations to prevent credential leaks.
|
| 165 |
-
8. **Robots Scanner (`robots_scanner.py`)**: Parses target `robots.txt` entries to extract hidden directories or disallowed routes.
|
| 166 |
-
9. **Directory Scanner (`directory_scanner.py`)**: Brute-forces directories using wordlists to discover hidden panels (`/admin`, `/phpmyadmin`, `/api/v1`).
|
| 167 |
-
10. **Fuzzer Scanner (`fuzzer_scanner.py`)**: Performs automated query parameter fuzzing, validating parameters against Cross-Site Scripting (XSS) and SQL Injection (SQLi) patterns.
|
| 168 |
-
11. **API Scanner (`api_scanner.py`)**: Maps routing interfaces, documenting open APIs and JSON payloads.
|
| 169 |
-
12. **Cloud Scanner (`cloud_scanner.py`)**: Audits exposed public storage assets (AWS S3 Buckets, Azure Blobs, etc.).
|
| 170 |
-
13. **Secrets Scanner (`secrets_scanner.py`)**: Scrapes source HTML code for exposed keys, AWS access IDs, and connection credentials.
|
| 171 |
-
14. **CVE Scanner (`cve_scanner.py`)**: Cross-references identified technology versions against public vulnerability registries.
|
| 172 |
-
15. **Nuclei Scanner (`nuclei_scanner.py`)**: Performs targeted scans using ProjectDiscovery's template engine.
|
| 173 |
-
16. **ZAP Scanner (`zap_scanner.py`)**: Coordinates deep active spider scanning via the OWASP ZAP API integration.
|
| 174 |
-
17. **CORS/API Helper Scanners**: Secondary scanners focused on validation and authorization testing.
|
| 175 |
-
|
| 176 |
-
---
|
| 177 |
-
|
| 178 |
-
## π Setup & Local Execution
|
| 179 |
-
|
| 180 |
-
### Prerequisites
|
| 181 |
-
* **Python 3.10+**
|
| 182 |
-
* **Node.js v18+**
|
| 183 |
-
* **Nmap** (must be added to system `PATH` environment variables)
|
| 184 |
-
* **Redis** (running locally on port `6379`)
|
| 185 |
-
|
| 186 |
-
---
|
| 187 |
-
|
| 188 |
-
### Step 1: Start Redis
|
| 189 |
-
You can run Redis using Docker:
|
| 190 |
-
```bash
|
| 191 |
-
docker-compose up -d
|
| 192 |
-
```
|
| 193 |
-
|
| 194 |
-
---
|
| 195 |
-
|
| 196 |
-
### Step 2: Configure and Start Backend
|
| 197 |
-
|
| 198 |
-
1. Navigate to the backend directory:
|
| 199 |
-
```bash
|
| 200 |
-
cd backend
|
| 201 |
-
```
|
| 202 |
-
2. Create a virtual environment and activate it:
|
| 203 |
-
```bash
|
| 204 |
-
python -m venv venv
|
| 205 |
-
# On Windows:
|
| 206 |
-
venv\Scripts\activate
|
| 207 |
-
# On Unix/macOS:
|
| 208 |
-
source venv/bin/activate
|
| 209 |
-
```
|
| 210 |
-
3. Install dependencies:
|
| 211 |
-
```bash
|
| 212 |
-
pip install -r requirements.txt
|
| 213 |
-
```
|
| 214 |
-
4. Verify your `.env` configuration. Ensure the keys and configurations are correct.
|
| 215 |
-
5. Seed the database and start the API server:
|
| 216 |
-
```bash
|
| 217 |
-
python run.py
|
| 218 |
-
```
|
| 219 |
-
*The Flask application will start on `http://127.0.0.1:5000`.*
|
| 220 |
-
|
| 221 |
-
---
|
| 222 |
-
|
| 223 |
-
### Step 3: Launch Celery Workers & Beat
|
| 224 |
-
Keep your backend running, open two new terminal sessions (with the virtual environment activated), and run:
|
| 225 |
-
|
| 226 |
-
1. **Celery Task Worker**:
|
| 227 |
-
```bash
|
| 228 |
-
celery -A celery_app.celery worker --loglevel=info
|
| 229 |
-
```
|
| 230 |
-
2. **Celery Beat Scheduler**:
|
| 231 |
-
```bash
|
| 232 |
-
celery -A celery_app.celery beat --loglevel=info
|
| 233 |
-
```
|
| 234 |
-
|
| 235 |
-
---
|
| 236 |
-
|
| 237 |
-
### Step 4: Configure and Run Frontend
|
| 238 |
-
|
| 239 |
-
1. Navigate to the frontend directory:
|
| 240 |
-
```bash
|
| 241 |
-
cd ../frontend
|
| 242 |
-
```
|
| 243 |
-
2. Install npm modules:
|
| 244 |
-
```bash
|
| 245 |
-
npm install
|
| 246 |
-
```
|
| 247 |
-
3. Start the Vite development server:
|
| 248 |
-
```bash
|
| 249 |
-
npm run dev
|
| 250 |
-
```
|
| 251 |
-
*The frontend application will boot on `http://localhost:5173`.*
|
| 252 |
-
|
| 253 |
-
---
|
| 254 |
-
|
| 255 |
-
## π§ͺ Seeding and Testing
|
| 256 |
-
|
| 257 |
-
On the first initialization, the database is pre-seeded with a default user and dummy mock security scan data so you can preview the platform immediately:
|
| 258 |
-
|
| 259 |
-
* **Mock Account Email**: `admin@gmail.com`
|
| 260 |
-
* **Mock Account Password**: `admin123`
|
| 261 |
-
|
| 262 |
-
You can log in with these credentials, explore the interactive remediation code windows, trigger new scans, check your live-updating terminal dashboard logs, and download auto-generated PDF reports directly from the history view.
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: SentinelScan WSS
|
| 3 |
+
emoji: π‘οΈ
|
| 4 |
+
colorFrom: indigo
|
| 5 |
+
colorTo: red
|
| 6 |
+
sdk: docker
|
| 7 |
+
pinned: true
|
| 8 |
+
---
|
| 9 |
+
# π‘οΈ SentinelScan β Website Security Scanner (WSS) by larxius
|
| 10 |
+
|
| 11 |
+
SentinelScan is a full-stack, enterprise-grade **Dynamic Application Security Testing (DAST)** platform designed to automate vulnerability detection across target domains and web APIs. Featuring a highly modular architecture, SentinelScan orchestrates a pipeline of custom security scanning agents concurrently, storing findings in a structured database and presenting them in a premium, real-time dashboard.
|
| 12 |
+
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
## π Key Features
|
| 16 |
+
|
| 17 |
+
* **Multi-Agent Concurrency**: Uses an asynchronous thread pool execution model (`ThreadPoolExecutor`) inside Celery tasks to run up to 17 specialized scanner modules in parallel.
|
| 18 |
+
* **Real-time Log Streaming**: Captures and exposes live, color-coded execution logs in-memory, enabling users to monitor active scans line-by-line.
|
| 19 |
+
* **Scheduled Scans**: Leverage Celery Beat to automate recurring scans (daily, weekly, monthly) for regular status monitoring.
|
| 20 |
+
* **Alert Webhooks**: Automatically dispatches security alerts to external services (e.g., Discord, custom webhooks) when critical or high vulnerabilities are discovered.
|
| 21 |
+
* **Authenticated Scanning**: Supports credentials/cookies injection via custom HTTP request headers, bypassing login perimeters to test deep backend routes.
|
| 22 |
+
* **Interactive Remediation**: Offers interactive, language-specific code remediation templates for each identified vulnerability type.
|
| 23 |
+
* **Dynamic PDF Reports**: Generates professional PDF summaries of completed scans, containing detailed risk score matrices and remediation guidelines.
|
| 24 |
+
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
## π Directory Structure
|
| 28 |
+
|
| 29 |
+
```text
|
| 30 |
+
Project-WSS/
|
| 31 |
+
βββ backend/ # Flask Backend Application
|
| 32 |
+
β βββ app/ # Main Flask Application Package
|
| 33 |
+
β β βββ routes/ # REST API Endpoints & Route Blueprints
|
| 34 |
+
β β β βββ auth.py # User Authentication (Login, Register)
|
| 35 |
+
β β β βββ reports.py # PDF Generation and Scan Reports
|
| 36 |
+
β β β βββ scans.py # Scan Configuration, Triggering, Logs
|
| 37 |
+
β β β βββ vulnerabilities.py # Remediation & Vulnerability Queries
|
| 38 |
+
β β βββ scanners/ # Security Engine Modules & Core Pipelines
|
| 39 |
+
β β β βββ __init__.py # Pipeline Definitions and Class Dispatcher
|
| 40 |
+
β β β βββ api_scanner.py # Exposed REST API Route Finder
|
| 41 |
+
β β β βββ base_scanner.py # Abstract Base Class and Shared Log Utilities
|
| 42 |
+
β β β βββ cloud_scanner.py # Public S3/Cloud Storage Auditor
|
| 43 |
+
β β β βββ cors_scanner.py # CORS Misconfigurations Tester
|
| 44 |
+
β β β βββ cve_scanner.py # Vulnerability Database Version Matcher
|
| 45 |
+
β β β βββ directory_scanner.py# Directory/File brute-forcer
|
| 46 |
+
β β β βββ fuzzer_scanner.py # SQLi & XSS Parameter Fuzzer
|
| 47 |
+
β β β βββ headers_scanner.py # HTTP Security Headers & Cache Poisoning
|
| 48 |
+
β β β βββ nmap_scanner.py # Port & Service Banner Scanner (via Nmap)
|
| 49 |
+
β β β βββ nuclei_scanner.py # Nuclei Template-based Scanner
|
| 50 |
+
β β β βββ robots_scanner.py # robots.txt Crawler
|
| 51 |
+
β β β βββ secrets_scanner.py # Page Secrets/API Key Scanner
|
| 52 |
+
β β β βββ sslyze_scanner.py # SSL/TLS Configurations & Ciphers Auditor
|
| 53 |
+
β β β βββ subdomain_scanner.py# Subdomain DNS Enumerator
|
| 54 |
+
β β β βββ tech_scanner.py # Technology Stack Fingerprinting
|
| 55 |
+
β β β βββ waf_scanner.py # WAF Detection & Fingerprinting
|
| 56 |
+
β β β βββ whois_scanner.py # Domain Registrar and Whois Lookup
|
| 57 |
+
β β β βββ zap_scanner.py # OWASP ZAP Active Spider Integration
|
| 58 |
+
β β βββ utils/ # Utility Scripts & Helpers
|
| 59 |
+
β β β βββ pdf_generator.py # ReportLab PDF Generation
|
| 60 |
+
β β β βββ webhook.py # Discord & Webhook Dispatcher
|
| 61 |
+
β β βββ database.py # SQLAlchemy Extension Instance
|
| 62 |
+
β β βββ extensions.py # Rate Limiter & Security Extensions
|
| 63 |
+
β β βββ models.py # SQLAlchemy Database Models (SQLite/PostgreSQL)
|
| 64 |
+
β β βββ scanner.py # Celery Tasks, Beat Schedules & Orchestration
|
| 65 |
+
β βββ celery_app.py # Celery Broker and Beat Scheduler Configuration
|
| 66 |
+
β βββ config.py # Environment Variable Parsing and App Constants
|
| 67 |
+
β βββ requirements.txt # Python Dependencies List
|
| 68 |
+
β βββ run.py # Flask Application Startup Launcher
|
| 69 |
+
β βββ .env # Local Environment Secret Key Configurations
|
| 70 |
+
β
|
| 71 |
+
βββ frontend/ # React Frontend Application (Vite-powered SPA)
|
| 72 |
+
β βββ src/ # React Application Source
|
| 73 |
+
β β ββοΏ½οΏ½ assets/ # SVGs, Fonts, and Static UI Elements
|
| 74 |
+
β β βββ components/ # Reusable UI Components
|
| 75 |
+
β β β βββ AuthContext.jsx # Global JWT Login State & API Interceptor
|
| 76 |
+
β β β βββ CodeBlock.jsx # Syntax-highlighted Remediation Viewer
|
| 77 |
+
β β β βββ Layout.jsx # Dashboard App Shell & Navigation Sidebar
|
| 78 |
+
β β β βββ ProtectedRoute.jsx # Auth Check Router Wrapper
|
| 79 |
+
β β β βββ ThreatGauge.jsx # SVG Semi-circle Security Score Indicator
|
| 80 |
+
β β βββ pages/ # Top-level Routing View Pages
|
| 81 |
+
β β β βββ Dashboard.jsx # Overview, Scan Metrics, and Status Cards
|
| 82 |
+
β β β βββ LandingPage.jsx # Modern Dark Mode Promotional Marketing Page
|
| 83 |
+
β β β βββ Login.jsx # Clean Secure Authentication Portal
|
| 84 |
+
β β β βββ NewScan.jsx # Target, Pipeline and Cookie Configurations
|
| 85 |
+
β β β βββ Register.jsx # Account Creation Portal
|
| 86 |
+
β β β βββ ReportsHistory.jsx # Past Scan Lists and Export Center
|
| 87 |
+
β β β βββ ScanResults.jsx # Vulnerability breakdown & Live terminal logs
|
| 88 |
+
β β β βββ Settings.jsx # Notification threshold & webhook configuration
|
| 89 |
+
β β βββ App.css # Main Layout styling
|
| 90 |
+
β β βββ App.jsx # Routing configuration
|
| 91 |
+
β β βββ index.css # Global theme tokens, inputs, animations
|
| 92 |
+
β β βββ main.jsx # DOM Injection root
|
| 93 |
+
β β βββ mockApi.js # Standalone local frontend mock testing DB
|
| 94 |
+
β β βββ theme.css # Precision Sentinel palette values
|
| 95 |
+
β βββ vite.config.js # React Hot Module Reloading server options
|
| 96 |
+
β βββ package.json # Frontend NPM scripts & dependencies
|
| 97 |
+
β
|
| 98 |
+
βββ docker-compose.yml # Multi-container orchestrator (Redis service)
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
---
|
| 102 |
+
|
| 103 |
+
## ποΈ System Architecture
|
| 104 |
+
|
| 105 |
+
```mermaid
|
| 106 |
+
graph TD
|
| 107 |
+
User([Security Auditor]) -->|Browser| FE[React Frontend SPA]
|
| 108 |
+
FE -->|API Requests| BE[Flask Web Backend]
|
| 109 |
+
BE -->|Store Scans/Vulns| DB[(SQLite / PostgreSQL)]
|
| 110 |
+
BE -->|Enqueue Jobs| Redis[(Redis Broker)]
|
| 111 |
+
Celery[Celery Task Workers] -->|Dequeue Jobs| Redis
|
| 112 |
+
Celery -->|Write Live Logs| MemLog[(In-Memory Logs)]
|
| 113 |
+
Celery -->|Execute Scanners Concurrently| Scanners{Scanner Suite}
|
| 114 |
+
Scanners -->|Target Requests| Target[Target System]
|
| 115 |
+
Scanners -->|Persist Findings| DB
|
| 116 |
+
Celery -->|Trigger Alert| Webhook[Webhook Notification]
|
| 117 |
+
```
|
| 118 |
+
|
| 119 |
+
### Backend Components
|
| 120 |
+
1. **Flask (REST API)**: Exposes endpoints for managing accounts, starting scans, listing results, downloading PDFs, and tracking setting updates.
|
| 121 |
+
2. **Celery Worker**: Dequeues scan tasks and runs them asynchronously.
|
| 122 |
+
3. **ThreadPoolExecutor**: Multi-threads individual scanners inside a Celery task.
|
| 123 |
+
4. **Celery Beat**: Runs continuously to process scheduled periodic scans.
|
| 124 |
+
5. **Redis**: Acts as the fast in-memory message broker.
|
| 125 |
+
|
| 126 |
+
---
|
| 127 |
+
|
| 128 |
+
## ποΈ Database Schema
|
| 129 |
+
|
| 130 |
+
The database schema, defined in `backend/app/models.py`, includes five main tables:
|
| 131 |
+
|
| 132 |
+
1. **`User`**: Manages credential hashing (via `bcrypt`) and session links.
|
| 133 |
+
2. **`Scan`**: Details the target domain, scan mode (Quick, Standard, Deep), authorization headers, overall security score, scan status, and timings.
|
| 134 |
+
3. **`Vulnerability`**: Stores findings linked to a scan. Contains details like CVSS score, severity classification, category, description, and copy-pasteable remediation snippets.
|
| 135 |
+
4. **`ScheduledScan`**: Saves user-configured scanning intervals (daily, weekly, monthly) for targets.
|
| 136 |
+
5. **`AlertSettings`**: Manages notification flags, webhook URL destinations, and minimum severity thresholds.
|
| 137 |
+
|
| 138 |
+
---
|
| 139 |
+
|
| 140 |
+
## βοΈ Scan Pipelines
|
| 141 |
+
|
| 142 |
+
Pipeline routes are configured in `backend/app/scanners/__init__.py`. Depending on the target criticality and scan duration limits, auditors choose between:
|
| 143 |
+
|
| 144 |
+
| Pipeline | Target Speed | Underlying Scanner Suite | Description |
|
| 145 |
+
| :--- | :--- | :--- | :--- |
|
| 146 |
+
| **`Quick`** | ~30 seconds | Headers, Nmap (top 100 ports), SSLyze, Tech stack, WHOIS, WAF | Surface audit for standard misconfigurations |
|
| 147 |
+
| **`Standard`** | ~2β3 minutes | Quick + SQLi/XSS Fuzzer, Subdomains, API pathways, Cloud, Secrets, CVEs | Comprehensive assessment of application business logic |
|
| 148 |
+
| **`Deep`** | ~10β15 minutes| Standard + CORS, robots.txt, Directory brute-force, Nuclei, ZAP (active) | Deep crawling and automated vulnerability exploitation |
|
| 149 |
+
| **`SSL`** | ~15 seconds | SSLyze, Headers | SSL certificate validation and cipher security audit |
|
| 150 |
+
| **`Port`** | ~45 seconds | Nmap (standard 1000 ports) | Port and network service banner reconnaissance |
|
| 151 |
+
|
| 152 |
+
---
|
| 153 |
+
|
| 154 |
+
## π οΈ The Scanner Suite (17 Specialized Modules)
|
| 155 |
+
|
| 156 |
+
Each scanner inherits from `BaseScanner` (`backend/app/scanners/base_scanner.py`) which coordinates logging, domain parsing, and vulnerability formatting:
|
| 157 |
+
|
| 158 |
+
1. **Headers Scanner (`headers_scanner.py`)**: Checks HTTP security headers (HSTS, CSP, CORS, X-Frame-Options, permissions, Referrer policy) and runs a custom check for HTTP host parameter cache poisoning.
|
| 159 |
+
2. **Nmap Scanner (`nmap_scanner.py`)**: Fires `nmap` commands directly via sub-process, checking exposed network services and testing for vulnerabilities using script scanning banners.
|
| 160 |
+
3. **SSLyze Scanner (`sslyze_scanner.py`)**: Audits SSL certificates, verifying trust status, expiration, and highlighting weak legacy protocols (TLS 1.0, SSLv3).
|
| 161 |
+
4. **Tech Scanner (`tech_scanner.py`)**: Fingerprints backend technologies, libraries, servers, and frameworks.
|
| 162 |
+
5. **Whois Scanner (`whois_scanner.py`)**: Looks up registrar information, IP ownership, and registration details.
|
| 163 |
+
6. **WAF Scanner (`waf_scanner.py`)**: Detects the presence of firewalls (Cloudflare, AWS WAF, ModSecurity, etc.) by inspecting response indicators.
|
| 164 |
+
7. **CORS Scanner (`cors_scanner.py`)**: Audits cross-origin resource sharing declarations to prevent credential leaks.
|
| 165 |
+
8. **Robots Scanner (`robots_scanner.py`)**: Parses target `robots.txt` entries to extract hidden directories or disallowed routes.
|
| 166 |
+
9. **Directory Scanner (`directory_scanner.py`)**: Brute-forces directories using wordlists to discover hidden panels (`/admin`, `/phpmyadmin`, `/api/v1`).
|
| 167 |
+
10. **Fuzzer Scanner (`fuzzer_scanner.py`)**: Performs automated query parameter fuzzing, validating parameters against Cross-Site Scripting (XSS) and SQL Injection (SQLi) patterns.
|
| 168 |
+
11. **API Scanner (`api_scanner.py`)**: Maps routing interfaces, documenting open APIs and JSON payloads.
|
| 169 |
+
12. **Cloud Scanner (`cloud_scanner.py`)**: Audits exposed public storage assets (AWS S3 Buckets, Azure Blobs, etc.).
|
| 170 |
+
13. **Secrets Scanner (`secrets_scanner.py`)**: Scrapes source HTML code for exposed keys, AWS access IDs, and connection credentials.
|
| 171 |
+
14. **CVE Scanner (`cve_scanner.py`)**: Cross-references identified technology versions against public vulnerability registries.
|
| 172 |
+
15. **Nuclei Scanner (`nuclei_scanner.py`)**: Performs targeted scans using ProjectDiscovery's template engine.
|
| 173 |
+
16. **ZAP Scanner (`zap_scanner.py`)**: Coordinates deep active spider scanning via the OWASP ZAP API integration.
|
| 174 |
+
17. **CORS/API Helper Scanners**: Secondary scanners focused on validation and authorization testing.
|
| 175 |
+
|
| 176 |
+
---
|
| 177 |
+
|
| 178 |
+
## π Setup & Local Execution
|
| 179 |
+
|
| 180 |
+
### Prerequisites
|
| 181 |
+
* **Python 3.10+**
|
| 182 |
+
* **Node.js v18+**
|
| 183 |
+
* **Nmap** (must be added to system `PATH` environment variables)
|
| 184 |
+
* **Redis** (running locally on port `6379`)
|
| 185 |
+
|
| 186 |
+
---
|
| 187 |
+
|
| 188 |
+
### Step 1: Start Redis
|
| 189 |
+
You can run Redis using Docker:
|
| 190 |
+
```bash
|
| 191 |
+
docker-compose up -d
|
| 192 |
+
```
|
| 193 |
+
|
| 194 |
+
---
|
| 195 |
+
|
| 196 |
+
### Step 2: Configure and Start Backend
|
| 197 |
+
|
| 198 |
+
1. Navigate to the backend directory:
|
| 199 |
+
```bash
|
| 200 |
+
cd backend
|
| 201 |
+
```
|
| 202 |
+
2. Create a virtual environment and activate it:
|
| 203 |
+
```bash
|
| 204 |
+
python -m venv venv
|
| 205 |
+
# On Windows:
|
| 206 |
+
venv\Scripts\activate
|
| 207 |
+
# On Unix/macOS:
|
| 208 |
+
source venv/bin/activate
|
| 209 |
+
```
|
| 210 |
+
3. Install dependencies:
|
| 211 |
+
```bash
|
| 212 |
+
pip install -r requirements.txt
|
| 213 |
+
```
|
| 214 |
+
4. Verify your `.env` configuration. Ensure the keys and configurations are correct.
|
| 215 |
+
5. Seed the database and start the API server:
|
| 216 |
+
```bash
|
| 217 |
+
python run.py
|
| 218 |
+
```
|
| 219 |
+
*The Flask application will start on `http://127.0.0.1:5000`.*
|
| 220 |
+
|
| 221 |
+
---
|
| 222 |
+
|
| 223 |
+
### Step 3: Launch Celery Workers & Beat
|
| 224 |
+
Keep your backend running, open two new terminal sessions (with the virtual environment activated), and run:
|
| 225 |
+
|
| 226 |
+
1. **Celery Task Worker**:
|
| 227 |
+
```bash
|
| 228 |
+
celery -A celery_app.celery worker --loglevel=info
|
| 229 |
+
```
|
| 230 |
+
2. **Celery Beat Scheduler**:
|
| 231 |
+
```bash
|
| 232 |
+
celery -A celery_app.celery beat --loglevel=info
|
| 233 |
+
```
|
| 234 |
+
|
| 235 |
+
---
|
| 236 |
+
|
| 237 |
+
### Step 4: Configure and Run Frontend
|
| 238 |
+
|
| 239 |
+
1. Navigate to the frontend directory:
|
| 240 |
+
```bash
|
| 241 |
+
cd ../frontend
|
| 242 |
+
```
|
| 243 |
+
2. Install npm modules:
|
| 244 |
+
```bash
|
| 245 |
+
npm install
|
| 246 |
+
```
|
| 247 |
+
3. Start the Vite development server:
|
| 248 |
+
```bash
|
| 249 |
+
npm run dev
|
| 250 |
+
```
|
| 251 |
+
*The frontend application will boot on `http://localhost:5173`.*
|
| 252 |
+
|
| 253 |
+
---
|
| 254 |
+
|
| 255 |
+
## π§ͺ Seeding and Testing
|
| 256 |
+
|
| 257 |
+
On the first initialization, the database is pre-seeded with a default user and dummy mock security scan data so you can preview the platform immediately:
|
| 258 |
+
|
| 259 |
+
* **Mock Account Email**: `admin@gmail.com`
|
| 260 |
+
* **Mock Account Password**: `admin123`
|
| 261 |
+
|
| 262 |
+
You can log in with these credentials, explore the interactive remediation code windows, trigger new scans, check your live-updating terminal dashboard logs, and download auto-generated PDF reports directly from the history view.
|