Building Defect AI
Deep Learning for Building Surface Defect Classification
Building Defect AI is a computer vision project that uses deep learning to classify building surface conditions from images. It explores transfer learning with ResNet18 and Grad-CAM explainability to support the interpretation of model predictions.
Project Overview
The project investigates how deep learning can help identify common building surface conditions through image classification.
The model classifies images into seven categories:
- Algae
- Major Crack
- Minor Crack
- Peeling
- Plain
- Spalling
- Stain
Best model: ResNet18 β Experiment 3
Test accuracy: 87.92%
Test images: 795
Correct predictions: 699
Dataset
The project uses the BD3 dataset.
| Dataset split | Images |
|---|---|
| Training | 2,378 |
| Validation | 792 |
| Testing | 795 |
| Total | 3,965 |
The dataset was split using a 60:20:20 ratio with random seed 42.
Model Development
Three experiments were conducted:
| Experiment | Approach | Test Accuracy |
|---|---|---|
| Experiment 1 | Frozen ResNet18 feature extractor | 83.14% |
| Experiment 2 | Fine-tuning Layer3, Layer4 and classifier | 79.75% |
| Experiment 3 | Fine-tuning Layer4 and classifier, with early stopping | 87.92% |
Experiment 3 produced the best test performance.
Evaluation Results
| Class | Precision | Recall | F1-score |
|---|---|---|---|
| Algae | 95.28% | 97.58% | 96.41% |
| Major Crack | 76.56% | 84.48% | 80.33% |
| Minor Crack | 88.78% | 69.05% | 77.68% |
| Peeling | 88.68% | 90.38% | 89.52% |
| Plain | 92.13% | 97.50% | 94.74% |
| Spalling | 81.58% | 93.00% | 86.92% |
| Stain | 93.68% | 84.76% | 89.00% |
Macro F1-score: 87.80%
Weighted F1-score: 87.77%
Minor crack detection remains an important area for improvement because its recall is comparatively low.
The actual test confusion matrix is available in evaluation/experiment3_test_confusion_matrix.png.
Explainability β Grad-CAM
Grad-CAM was implemented using the final convolutional block of ResNet18.
It generates heatmaps that highlight image regions contributing to a prediction. These visualisations provide a qualitative way to inspect model behaviour.
Grad-CAM does not provide precise defect boundaries, defect severity or a professional building condition assessment.
Repository Structure
building-defect-ai/
β
βββ README.md
βββ model/
β βββ building_defect_resnet18_experiment3.pth
β
βββ notebooks/
β βββ 01_dataset_exploration.ipynb
β βββ 02_image_preprocessing.ipynb
β βββ 03_resnet18_transfer_learning.ipynb
β βββ 04_model_evaluation.ipynb
β βββ 05_gradcam_explainability.ipynb
β
βββ src/
β βββ dataset.py
β βββ evaluate.py
β βββ gradcam.py
β βββ model.py
β βββ predict.py
β
βββ app/
β βββ app.py
β βββ README.md
β
βββ docs/
β βββ methodology.md
β βββ experiments.md
β βββ limitations.md
β
βββ evaluation/
β βββ experiment3_evaluation.json
β βββ experiment3_test_confusion_matrix.png
β
βββ assets/
βββ building_defect_ai_infographic.png
Intended Use
This project is an academic and experimental prototype for building surface image classification.
It is intended for:
- Exploring computer vision applications in architecture.
- Experimenting with transfer learning.
- Studying explainable AI techniques.
- Supporting research into automated building defect identification.
It is not intended to replace professional building inspections or engineering assessments.
Limitations
- Performance depends on the characteristics and distribution of the training dataset.
- Minor crack classification requires further improvement.
- Potential near-duplicate or augmented-image leakage between dataset splits has not been verified.
- Grad-CAM provides qualitative visual explanations, not defect segmentation.
- Results on external real-world images require further validation.
Future Development
- Expand and diversify the training dataset.
- Improve minor crack detection.
- Investigate object detection and segmentation.
- Evaluate generalisation on external building images.
- Explore integration with BIM-based building inspection workflows.
Project Links
GitHub Repository: https://github.com/fawadsaleem-gif/DL-Building-Defect-AI
Hugging Face Model: https://huggingface.co/fawad-archAI/building-defect-ai
Developed as part of an applied AI and deep learning project focused on architecture and the built environment.
