File size: 6,807 Bytes
faa5063 2b7de45 faa5063 a18a1ea faa5063 9b057aa faa5063 019d5b1 faa5063 019d5b1 faa5063 4846471 0189224 4846471 0189224 4846471 faa5063 4846471 faa5063 4846471 faa5063 4846471 faa5063 9b057aa faa5063 4846471 faa5063 4846471 faa5063 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
license: mit
language:
- en
---
# Model Details
Agglomerative Token Clustering (ATC), a novel hierarchical hard-merging based token reduction method.
ATC has been demonstrated to outperform all prior state-of-the-art token reduction methods across image classification, image synthesis, and object detection & segementation tasks.
We provide DeiT checkpoints (Tiny, Small, and Base) at four reduction rates (0.9, 0.7, 0.5, and 0.25) for four classification datasets: ImageNet-1K, NABirds, COCO 2014, and NUS-WIDE.
We also provide ViT-Adapter checkpoints (Tiny and Small) for the COCO 2017 object detection and segementation task, for both the ATC and ToMe token reduction methods.
### Model Description
- **Developed by:** Joakim Bruslund Haurum, Sergio Escalera, Graham W. Taylor, and Thomas B. Moeslund
- **Model type:** Vision Transformers
- **License:** MIT License
### More Resources
- **Repository:** [https://github.com/JoakimHaurum/ATC](https://github.com/JoakimHaurum/ATC)
- **Paper:** [https://arxiv.org/abs/2409.11923](https://arxiv.org/abs/2409.11923)
- **Project Page:** [https://vap.aau.dk/atc](https://vap.aau.dk/atc)
- **HuggingFace Collection:** [https://huggingface.co/collections/joakimbh/agglomerative-token-clustering-66e94dfb313e85ec97590fe4](https://huggingface.co/collections/joakimbh/agglomerative-token-clustering-66e94dfb313e85ec97590fe4)
## Model Zoo
**Note: This repository does not host any checkpoints but contains links to all the model repositories. Each token reduction method repository contains the checkpoints for the four considered keep rates.**
Image Classification Models:
| Model Name | Dataset | Linkage Function | Weights |
|:-----------|:--------:|:-----------:|:--------:|
| atc_base-im1k-average | ImageNet-1K | Average | [link](https://huggingface.co/joakimbh/atc_base-im1k-average) |
| atc_base-im1k-single | ImageNet-1K | Single | [link](https://huggingface.co/joakimbh/atc_base-im1k-single) |
| atc_base-im1k-complete | ImageNet-1K | Complete | [link](https://huggingface.co/joakimbh/atc_base-im1k-complete) |
| atc_small-im1k-average | ImageNet-1K | Average | [link](https://huggingface.co/joakimbh/atc_small-im1k-average) |
| atc_small-im1k-single | ImageNet-1K | Single | [link](https://huggingface.co/joakimbh/atc_small-im1k-single) |
| atc_small-im1k-complete | ImageNet-1K | Complete | [link](https://huggingface.co/joakimbh/atc_small-im1k-complete) |
| atc_tiny-im1k-average | ImageNet-1K | Average | [link](https://huggingface.co/joakimbh/atc_tiny-im1k-average) |
| atc_tiny-im1k-single | ImageNet-1K | Single | [link](https://huggingface.co/joakimbh/atc_tiny-im1k-single) |
| atc_tiny-im1k-complete | ImageNet-1K | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-im1k-complete) |
<br>
| atc_base-nab-average | NABirds | Average | [link](https://huggingface.co/joakimbh/atc_base-nab-average) |
| atc_base-nab-single | NABirds | Single | [link](https://huggingface.co/joakimbh/atc_base-nab-single) |
| atc_base-nab-complete | NABirds | Complete | [link](https://huggingface.co/joakimbh/atc_base-nab-complete) |
| atc_small-nab-average | NABirds | Average | [link](https://huggingface.co/joakimbh/atc_small-nab-average) |
| atc_small-nab-single | NABirds | Single | [link](https://huggingface.co/joakimbh/atc_small-nab-single) |
| atc_small-nab-complete | NABirds | Complete | [link](https://huggingface.co/joakimbh/atc_small-nab-complete) |
| atc_tiny-nab-average | NABirds | Average | [link](https://huggingface.co/joakimbh/atc_tiny-nab-average) |
| atc_tiny-nab-single | NABirds | Single | [link](https://huggingface.co/joakimbh/atc_tiny-nab-single) |
| atc_tiny-nab-complete | NABirds | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-nab-complete) |
<br>
| atc_base-coco-average | COCO 2014 | Average | [link](https://huggingface.co/joakimbh/atc_base-coco-average) |
| atc_base-coco-single | COCO 2014 | Single | [link](https://huggingface.co/joakimbh/atc_base-coco-single) |
| atc_base-coco-complete | COCO 2014 | Complete | [link](https://huggingface.co/joakimbh/atc_base-coco-complete) |
| atc_small-coco-average | COCO 2014 | Average | [link](https://huggingface.co/joakimbh/atc_small-coco-average) |
| atc_small-coco-single | COCO 2014 | Single | [link](https://huggingface.co/joakimbh/atc_small-coco-single) |
| atc_small-coco-complete | COCO 2014 | Complete | [link](https://huggingface.co/joakimbh/atc_small-coco-complete) |
| atc_tiny-coco-average | COCO 2014 | Average | [link](https://huggingface.co/joakimbh/atc_tiny-coco-average) |
| atc_tiny-coco-single | COCO 2014 | Single | [link](https://huggingface.co/joakimbh/atc_tiny-coco-single) |
| atc_tiny-coco-complete | COCO 2014 | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-coco-complete) |
<br>
| atc_base-nus-average | NUS-WIDE | Average | [link](https://huggingface.co/joakimbh/atc_base-nus-average) |
| atc_base-nus-single | NUS-WIDE | Single | [link](https://huggingface.co/joakimbh/atc_base-nus-single) |
| atc_base-nus-complete | NUS-WIDE | Complete | [link](https://huggingface.co/joakimbh/atc_base-nus-complete) |
| atc_small-nus-average | NUS-WIDE | Average | [link](https://huggingface.co/joakimbh/atc_small-nus-average) |
| atc_small-nus-single | NUS-WIDE | Single | [link](https://huggingface.co/joakimbh/atc_small-nus-single) |
| atc_small-nus-complete | NUS-WIDE | Complete | [link](https://huggingface.co/joakimbh/atc_small-nus-complete) |
| atc_tiny-nus-average| NUS-WIDE | Average | [link](https://huggingface.co/joakimbh/atc_tiny-nus-average) |
| atc_tiny-nus-single | NUS-WIDE | Single | [link](https://huggingface.co/joakimbh/atc_tiny-nus-single) |
| atc_tiny-nus-complete | NUS-WIDE | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-nus-complete) |
<br>
Object Detection and Segmentation Models:
| Model Name | Dataset | Keep Rate | Weights |
|:-----------|:--------:|:-----------:|:--------:|
| atc_small-coco17-average | COCO 2017 | Average | [link](https://huggingface.co/joakimbh/atc_small-coco17-average) |
| atc_small-coco17-single | COCO 2017 | Single | [link](https://huggingface.co/joakimbh/atc_small-coco17-single) |
| atc_small-coco17-complete | COCO 2017 | Complete | [link](https://huggingface.co/joakimbh/atc_small-coco17-complete) |
| tome_small-coco17 | COCO 2017 | - | [link](https://huggingface.co/joakimbh/tome_small-coco17) |
<br>
| atc_tiny-coco17-average | COCO 2017 | Average | [link](https://huggingface.co/joakimbh/atc_tiny-coco17-average) |
| atc_tiny-coco17-single | COCO 2017 | Single | [link](https://huggingface.co/joakimbh/atc_tiny-coco17-single) |
| atc_tiny-coco17-complete | COCO 2017 | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-coco17-complete) |
| tome_tiny-coco17 | COCO 2017 | - | [link](https://huggingface.co/joakimbh/tome_tiny-coco17) |
<br> |