README.md

metadata

title: Lesion Dection
emoji: 🐢
colorFrom: blue
colorTo: green
sdk: streamlit
sdk_version: 1.45.1
app_file: app.py
pinned: false

ITC 6109 - Machine Vision: Skin Lesion Classification

Project: Automated Classification of Dermatoscopy Images (HAM10000) using Classical Computer Vision & Machine Learning.

👥 Team Members

Kolia Aimilia Kontoudakis Nikos Skiada Kiki Lampropoulou Nancy

📂 Project Structure

The project is organized into a modular Python package (src) to separate concerns between data handling, computer vision logic, and machine learning.

ham10000_project/
│
├── data/                    # Dataset (Input)
│   ├── images/              # Source dermatoscopy images
│   └── GroundTruth.csv      # Metadata and labels
│
├── models/                  # Artifacts (Output)
│   ├── skin_cancer_model.pkl # Trained Random Forest model
│   ├── scaler.pkl           # StandardScaler for normalization
│   ├── X_train_sample.npy   # Sample data for LIME initialization in App
│   └── [plots]              # ROC curves, Confusion Matrices, LIME plots
│
├── src/                     # Core Logic Package
│   ├── __init__.py          # Package initializer
│   ├── config.py            # Hyperparameters, constants, and paths
│   ├── data.py              # CSV parsing and metadata loading
│   ├── features.py          # The complete Computer Vision pipeline
│   ├── augmentation.py      # Keras-based image augmentation logic
│   ├── model.py             # Model training, evaluation, and plotting orchestration
│   ├── plots.py             # Visualization logic (ROC, Confusion Matrices)
│   └── explainability.py    # XAI logic (LIME, Feature Importance)
│
├── train_main.py            # MAIN SCRIPT: Orchestrates the training pipeline
├── app.py                   # WEB APP: Interactive Streamlit interface
└── requirements.txt         # Project dependencies

🔄 Sequence of Execution

When you run python train_main.py, the system follows this strict sequence to ensure data integrity (preventing leakage) and robust training.
Phase 1: Data Preparation
Load Metadata (src.data):
The script reads GroundTruth.csv.
It parses the one-hot encoded labels into a single target class (e.g., 'MEL', 'NV').
It constructs valid file paths for every image.
Train/Test Split:
CRITICAL STEP: The data is split into Training (80%) and Test (20%) sets before any image processing or augmentation occurs. This guarantees that no augmented version of a test image ever leaks into the training set.
Phase 2: Feature Extraction & Augmentation
The script processes the Test set and Training set differently:
Test Set Processing:
Iterates through test images.
Passes each image through the CV Pipeline (src.features) to extract a 1D feature vector (Color, Shape, Texture).
No augmentation is applied.
Training Set Processing (With Augmentation):
Iterates through training images.
Class Check: Checks if the image belongs to a minority class.
On-the-fly Augmentation (src.augmentation): If it's a minority class, Keras generates rotated, zoomed, or shifted versions of the image to balance the dataset.
Feature Extraction: Features are extracted for the original image and all generated augmented versions.
Phase 3: The Computer Vision Pipeline (src.features)
Every image (original or augmented) goes through these steps to generate numbers for the AI:
Preprocessing: Resize with padding (to keep aspect ratio), Grayscale conversion, CLAHE (Smart Contrast), and Gaussian Blur.
Segmentation: Otsu's Thresholding finds the lesion. Morphological operations (Opening/Dilation) clean noise and connect fragmented parts.
Shape Analysis: Calculates Area, Perimeter, and Compactness of the largest object.
Color Analysis: Calculates Mean/Std/Skew for RGB channels and a Color Histogram, strictly within the lesion mask.
Texture Analysis: Uses Canny Edge Detection to measure edge density inside the lesion.
Phase 4: Training & Evaluation (src.model)
Scaling: A StandardScaler is fit on the Training features and applied to Test features.
Training: Random Forest and SVM models are trained on the balanced feature set.
Evaluation:
Predictions are made on the Test Set.
Confusion Matrices (Raw and Normalized) are generated.
Multi-class ROC Curves are plotted.
Explainable AI (XAI):
Global: Feature Importance plot is generated for Random Forest.
Local (LIME): Individual explanations are generated for sample test instances to show why specific decisions were made.
Artifact Saving: The best model, scaler, class names, and a training sample (for the App) are saved to models/.

🚀 How to Run

Install Requirements:

pip install -r requirements.txt

Train the System:

python train_main.py

Launch the Web App:

streamlit run app.py