---
language: en
library_name: pytorch
tags:
- image-classification
- computer-vision
- resnest
- deepten
- timm
- safetensors
datasets:
- mcimpoi/minc-2500_split_1
metrics:
- accuracy
---

# DeepTen Image Classifier

This model is an image classifier based on the **DeepTen** architecture, utilizing a **ResNeSt50d** feature extractor (backbone) and a custom Encoding Layer to capture contextual image information.

## Links

<div style="display: flex; justify-content: center; align-items: center; gap: 15px;">
  <a href="https://wandb.ai/nbamine-fsdm/DeepTen/" target="_blank">
    <img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in W&B" style="margin: 0;">
  </a>
  <a href="https://github.com/NBAmine/deepten-resnest-texture-classification" target="_blank">
    <img src="https://img.shields.io/badge/GitHub-181717?style=for-the-badge&logo=github&logoColor=white" alt="Visualize in W&B" style="margin: 0;">
  </a>
</div>

## Model Details

* **Model Type:** Image Classification (Convolutional Neural Network)
* **Framework:** PyTorch
* **Weights Format:** `safetensors`

## Technical Architecture

The model replaces the traditional Global Average Pooling with a Deep Ten approach:
1. **Backbone:** `resnest50d` (via the `timm` library), instantiated without its original classification head.
2. **Classification Head (DeepTen Head):**
    * 1x1 Convolutional layer to reduce dimensionality to 128 channels.
    * Encoding Layer learning a dictionary of $K=32$ codewords.
    * L2 Normalization layer.
    * Final Linear (Fully Connected) layer adapted to the dataset's number of classes.

## Training and Evaluation

The training pipeline is designed for distributed execution (Distributed Data Parallel - DDP) and comprehensive tracking:

* **Validation:** 5-Fold Cross-Validation strategy.
* **Loss Function:** `CrossEntropyLoss`.
* **Optimizer:** SGD (with Momentum and Weight Decay).
* **Scheduler:** `ReduceLROnPlateau` based on validation loss.
* **Regularization:** Configurable Early Stopping (patience and min delta).
* **Experiment Tracking:** Metrics (Loss, Accuracy, Precision, Recall, F1-Score) and confusion matrices are logged via **Weights & Biases (W&B)**.

## Preprocessing and Data Augmentation

* **Training:** Implements Principal Component Analysis-based color augmentation (`PCAColorAugmentation`) to modify RGB intensities using predefined eigenvectors and eigenvalues.
* **Validation / Testing:** Utilizes the **TenCrop** technique (extracting 10 sub-regions per image) with averaged predictions to maximize final evaluation robustness.

## Performance

| Fold | Best Epoch | Val Accuracy (%) | Test Accuracy (%) |
| :---: | :---: | :---: | :---: |
| 1 | 46 | 79.42 | 79.71 |
| 2 | 54 | 81.02 | 81.50 |
| 3 | 43 | 80.78 | 82.11 |
| 4 | 58 | 80.47 | 80.25 |
| 5 | 56 | 80.88 | 79.79 |

## Citations

**ResNeSt: Split-Attention Networks** [[arXiv]](https://arxiv.org/abs/2004.08955)  
  [Hang Zhang](http://hangzh.com/), Chongruo Wu, Zhongyue Zhang, Yi Zhu, Zhi Zhang, Haibin Lin, Yue Sun, Tong He, Jonas Muller, R. Manmatha, Mu Li and Alex Smola
```
@article{zhang2020resnest,
title={ResNeSt: Split-Attention Networks},
author={Zhang, Hang and Wu, Chongruo and Zhang, Zhongyue and Zhu, Yi and Zhang, Zhi and Lin, Haibin and Sun, Yue and He, Tong and Muller, Jonas and Manmatha, R. and Li, Mu and Smola, Alexander},
journal={arXiv preprint},
year={2020}
}
```

**Context Encoding for Semantic Segmentation** [[arXiv]](https://arxiv.org/pdf/1803.08904.pdf)  
 [Hang Zhang](http://hangzh.com/), [Kristin Dana](http://eceweb1.rutgers.edu/vision/dana.html), [Jianping Shi](http://shijianping.me/), [Zhongyue Zhang](http://zhongyuezhang.com/), [Xiaogang Wang](http://www.ee.cuhk.edu.hk/~xgwang/), [Ambrish Tyagi](https://scholar.google.com/citations?user=GaSWCoUAAAAJ&hl=en), [Amit Agrawal](http://www.amitkagrawal.com/)
```
@InProceedings{Zhang_2018_CVPR,
author = {Zhang, Hang and Dana, Kristin and Shi, Jianping and Zhang, Zhongyue and Wang, Xiaogang and Tyagi, Ambrish and Agrawal, Amit},
title = {Context Encoding for Semantic Segmentation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}
```

**Deep TEN: Texture Encoding Network** [[arXiv]](https://arxiv.org/pdf/1612.02844.pdf)  
  [Hang Zhang](http://hangzh.com/), [Jia Xue](http://jiaxueweb.com/), [Kristin Dana](http://eceweb1.rutgers.edu/vision/dana.html)
```
@InProceedings{Zhang_2017_CVPR,
author = {Zhang, Hang and Xue, Jia and Dana, Kristin},
title = {Deep TEN: Texture Encoding Network},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {July},
year = {2017}
}
```