| # MODULES Directory | |
| ## Purpose: | |
| This directory houses the functional modules responsible for specific phases of the OSINT pipeline. Each subfolder represents a stage of processing from data ingestion to export. | |
| ## Subdirectories: | |
| - `ingestion/`: Connects to external data sources (web scrapers, APIs, sensors). | |
| - `preprocessing/`: Cleans, deduplicates, and normalizes raw data into structured input. | |
| - `ml_analysis/`: Applies machine learning models (e.g., classification, clustering, NLP). | |
| - `correlation/`: Cross-references data (e.g., STIX/IOC matching, pattern detection). | |
| - `export/`: Packages output into files, databases, APIs, or dashboards. | |
| ## Design Philosophy: | |
| Each module is atomic, reusable, and accepts standardized JSON inputs/outputs. Naming follows functional role, and new tools should be added under appropriate phase. | |