# MODULES Directory

## Purpose:
This directory houses the functional modules responsible for specific phases of the OSINT pipeline. Each subfolder represents a stage of processing from data ingestion to export.

## Subdirectories:
- `ingestion/`: Connects to external data sources (web scrapers, APIs, sensors).
- `preprocessing/`: Cleans, deduplicates, and normalizes raw data into structured input.
- `ml_analysis/`: Applies machine learning models (e.g., classification, clustering, NLP).
- `correlation/`: Cross-references data (e.g., STIX/IOC matching, pattern detection).
- `export/`: Packages output into files, databases, APIs, or dashboards.

## Design Philosophy:
Each module is atomic, reusable, and accepts standardized JSON inputs/outputs. Naming follows functional role, and new tools should be added under appropriate phase.