DeepTen Image Classifier

This model is an image classifier based on the DeepTen architecture, utilizing a ResNeSt50d feature extractor (backbone) and a custom Encoding Layer to capture contextual image information.

Links

Model Details

• Model Type: Image Classification (Convolutional Neural Network)
• Framework: PyTorch
• Weights Format: safetensors

Technical Architecture

The model replaces the traditional Global Average Pooling with a Deep Ten approach:

1. Backbone: resnest50d (via the timm library), instantiated without its original classification head.
2. Classification Head (DeepTen Head):
   • 1x1 Convolutional layer to reduce dimensionality to 128 channels.
   • Encoding Layer learning a dictionary of $K=32$ codewords.
   • L2 Normalization layer.
   • Final Linear (Fully Connected) layer adapted to the dataset's number of classes.

Training and Evaluation

The training pipeline is designed for distributed execution (Distributed Data Parallel - DDP) and comprehensive tracking:

• Validation: 5-Fold Cross-Validation strategy.
• Loss Function: CrossEntropyLoss.
• Optimizer: SGD (with Momentum and Weight Decay).
• Scheduler: ReduceLROnPlateau based on validation loss.
• Regularization: Configurable Early Stopping (patience and min delta).
• Experiment Tracking: Metrics (Loss, Accuracy, Precision, Recall, F1-Score) and confusion matrices are logged via Weights & Biases (W&B).

Preprocessing and Data Augmentation

• Training: Implements Principal Component Analysis-based color augmentation (PCAColorAugmentation) to modify RGB intensities using predefined eigenvectors and eigenvalues.
• Validation / Testing: Utilizes the TenCrop technique (extracting 10 sub-regions per image) with averaged predictions to maximize final evaluation robustness.

Performance

Fold	Best Epoch	Val Accuracy (%)	Test Accuracy (%)
1	46	79.42	79.71
2	54	81.02	81.50
3	43	80.78	82.11
4	58	80.47	80.25
5	56	80.88	79.79

Citations

ResNeSt: Split-Attention Networks [arXiv]
Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Zhi Zhang, Haibin Lin, Yue Sun, Tong He, Jonas Muller, R. Manmatha, Mu Li and Alex Smola

@article{zhang2020resnest,
title={ResNeSt: Split-Attention Networks},
author={Zhang, Hang and Wu, Chongruo and Zhang, Zhongyue and Zhu, Yi and Zhang, Zhi and Lin, Haibin and Sun, Yue and He, Tong and Muller, Jonas and Manmatha, R. and Li, Mu and Smola, Alexander},
journal={arXiv preprint},
year={2020}
}

Context Encoding for Semantic Segmentation [arXiv]
Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, Amit Agrawal

@InProceedings{Zhang_2018_CVPR,
author = {Zhang, Hang and Dana, Kristin and Shi, Jianping and Zhang, Zhongyue and Wang, Xiaogang and Tyagi, Ambrish and Agrawal, Amit},
title = {Context Encoding for Semantic Segmentation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}

Deep TEN: Texture Encoding Network [arXiv]
Hang Zhang, Jia Xue, Kristin Dana

@InProceedings{Zhang_2017_CVPR,
author = {Zhang, Hang and Xue, Jia and Dana, Kristin},
title = {Deep TEN: Texture Encoding Network},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {July},
year = {2017}
}