File size: 4,490 Bytes
e0054e4 8836a24 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 |
---
license: cc0-1.0
datasets:
- biglam/european_art
base_model:
- blesot/Mask-RCNN
pipeline_tag: object-detection
tags:
- cultural
---
# 🖼️ Saint George on a Bike – Mask R-CNN for Iconographic Object Detection
## Model Summary
This model uses the [Matterport Mask R-CNN](https://github.com/matterport/Mask_RCNN) implementation fine-tuned for detecting iconographic and symbolic elements in religious artworks. It is developed as part of the **Saint George on a Bike** project to enable semantic enrichment and understanding of historical imagery.
---
## 🧠 Model Details
- **Architecture**: Mask R-CNN with ResNet backbone
- **Framework**: TensorFlow 1.14.0 + Keras 2.2.5
- **Source**: https://github.com/matterport/Mask_RCNN
- **Configuration**:
- `NUM_CLASSES`: 69+1 (background)
- `DETECTION_MIN_CONFIDENCE`: 0.76
---
## 🎯 Use Cases
- Iconography detection in religious paintings
- Digital humanities and art historical research
- Training multimodal models for cultural heritage
- Enriching metadata in museum and archive collections
---
## 🏷️ Labels (Selected)
The model detects over 40 iconographic concepts including:
- `crucifixion`
- `angel`
- `crown of thorns`
- `monk`
- `sword`
- `chalice`
- `dove`
- `lion`, `shepherd`, `scroll`, `key of heaven`, `mitre`, and more
> Full class list is available in the source notebook.
---
## 📊 Training Data
- The model was trained on a DEArt dataset curated for the **Saint George on a Bike** project.
- Dataset contains annotated religious artworks with rich symbolic content.
- Format and exact size unspecified; annotations PascalXML structure.
---
## 🧪 Example Usage
```python
from mrcnn.config import Config
from mrcnn.model import MaskRCNN
from mrcnn.model import mold_image
from keras.preprocessing.image import load_img, img_to_array
from numpy import expand_dims
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
# Define class labels (shortened list)
classids=["BG","crucifixion","angel","person","crown of thorns", "horse", "dragon","bird","dog","boat","cat","book",
"sheep","shepherd","elephant","zebra","crown","tiara","camauro","zucchetto","mitre","saturno","skull",
"orange","apple","banana","nude","monk","lance","key of heaven", "banner","chalice","palm","sword","rooster",
"knight","scroll","lily","horn","prayer","tree","arrow","crozier","deer","devil","dove","eagle","hands",
"head","lion","serpent","stole","trumpet","judith","halo","helmet","shield","jug","holy shroud","god the father",
"swan", "butterfly", "bear", "centaur","pegasus","donkey","mouse","monkey","cow","unicorn"]
# Define the inference config
class PredictionConfig(Config):
NAME = "PREDICTION_cfg"
NUM_CLASSES = len(classids)
GPU_COUNT = 1
IMAGES_PER_GPU = 1
DETECTION_MIN_CONFIDENCE = 0.76
# Initialize model
cfg = PredictionConfig()
model = MaskRCNN(mode='inference', model_dir='./', config=cfg)
model.load_weights('<weights of model>', by_name=True)
# Load and process image
img = load_img("example.jpg")
image = img_to_array(img)
scaled_image = mold_image(image, cfg)
sample = expand_dims(scaled_image, 0)
# Run detection
yhat = model.detect(sample, verbose=0)[0]
# Visualize detections
fig = plt.figure(figsize=(12, 12))
ax = fig.add_subplot(111)
ax.imshow(img)
for i in range(len(yhat['rois'])):
y1, x1, y2, x2 = yhat['rois'][i]
width, height = x2 - x1, y2 - y1
rect = Rectangle((x1, y1), width, height, fill=False, color='red')
ax.add_patch(rect)
ax.text(x1 + 5, y1 + 10, classids[yhat['class_ids'][i]], fontsize=12, color='white')
plt.show()
```
---
## 📌 Limitations
- Accuracy on modern images or non-religious art is not guaranteed
- Requires legacy versions of TensorFlow and Keras
---
## 📚 Citation
If you use this model, please cite:
- The Matterport Mask R-CNN repository: https://github.com/matterport/Mask_RCNN
- DEArt Dataset
```
@misc{reshetnikov2022deartdataseteuropeanart,
title={DEArt: Dataset of European Art},
author={Artem Reshetnikov and Maria-Cristina Marinescu and Joaquim More Lopez},
year={2022},
eprint={2211.01226},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2211.01226},
}
```
---
## 🙏 Acknowledgements
This research has been supported by the Saint George on a Bike project 2018-EU-IA-0104, co-financed by the Connecting Europe Facility of the European Union. |