Hugging Face's logo

joakimbh
/
TokenReduction

English
Model card Files
xet
 Community
TokenReduction / README.md
joakimbh's picture
joakimbh
Fix DeiT model table
4ad700a verified
preview code
|
raw
history blame contribute delete
9.93 kB
metadata 
license: mit
language:
  - en

Model Details

Since the introduction of the Vision Transformer (ViT), researchers have sought to make ViTs more efficient by removing redundant information in the processed tokens. While different methods have been explored to achieve this goal, we still lack understanding of the resulting reduction patterns and how those patterns differ across token reduction methods and datasets. To close this gap, we set out to understand the reduction patterns of 10 different token reduction methods using four image classification datasets: ImageNet, NABirds, COCO, and NUS-WIDE.

We provide DeiT checkpoints (Tiny, Small, and Base) at four keep rates (0.9, 0.7, 0.5, and 0.25) for four classification datasets: ImageNet-1K, NABirds, COCO 2014, and NUS-WIDE.

Model Description

  • Developed by: Joakim Bruslund Haurum, Sergio Escalera, Graham W. Taylor, and Thomas B. Moeslund
  • Model type: Vision Transformers
  • License: MIT License

More Resources

Model Zoo

Note: This repository does not host any checkpoints but contains links to all the model repositories. Each token reduction method repository contains the checkpoints for the four considered keep rates.

Baseline DeiT

Model Name Dataset Weights
deit_base-nab NABirds link
deit_small-nab NABirds link
deit_tiny-nab NABirds link
deit_base-coco COCO 2014 link
deit_small-coco COCO 2014 link
deit_tiny-coco COCO 2014 link
deit_base-nus NUS-WIDE link
deit_small-nus NUS-WIDE link
deit_tiny-nus NUS-WIDE link

Top-K:

Model Name Dataset Weights
topk_base-im1k ImageNet-1K link
topk_small-im1k ImageNet-1K link
topk_tiny-im1k ImageNet-1K link
topk_base-nab NABirds link
topk_small-nab NABirds link
topk_tiny-nab NABirds link
topk_base-coco COCO 2014 link
topk_small-coco COCO 2014 link
topk_tiny-coco COCO 2014 link
topk_base-nus NUS-WIDE link
topk_small-nus NUS-WIDE link
topk_tiny-nus NUS-WIDE link

EViT

Model Name Dataset Weights
evit_base-im1k ImageNet-1K link
evit_small-im1k ImageNet-1K link
evit_tiny-im1k ImageNet-1K link
evit_base-nab NABirds link
evit_small-nab NABirds link
evit_tiny-nab NABirds link
evit_base-coco COCO 2014 link
evit_small-coco COCO 2014 link
evit_tiny-coco COCO 2014 link
evit_base-nus NUS-WIDE link
evit_small-nus NUS-WIDE link
evit_tiny-nus NUS-WIDE link

DynamicViT

Model Name Dataset Weights
dyvit_base-im1k ImageNet-1K link
dyvit_small-im1k ImageNet-1K link
dyvit_tiny-im1k ImageNet-1K link
dyvit_base-nab NABirds link
dyvit_small-nab NABirds link
dyvit_tiny-nab NABirds link
dyvit_base-coco COCO 2014 link
dyvit_small-coco COCO 2014 link
dyvit_tiny-coco COCO 2014 link
dyvit_base-nus NUS-WIDE link
dyvit_small-nus NUS-WIDE link
dyvit_tiny-nus NUS-WIDE link

ATS

Model Name Dataset Weights
ats_base-im1k ImageNet-1K link
ats_small-im1k ImageNet-1K link
ats_tiny-im1k ImageNet-1K link
ats_base-nab NABirds link
ats_small-nab NABirds link
ats_tiny-nab NABirds link
ats_base-coco COCO 2014 link
ats_small-coco COCO 2014 link
ats_tiny-coco COCO 2014 link
ats_base-nus NUS-WIDE link
ats_small-nus NUS-WIDE link
ats_tiny-nus NUS-WIDE link

L1

Model Name Dataset Weights
l1_base-im1k ImageNet-1K link
l1_small-im1k ImageNet-1K link
l1_tiny-im1k ImageNet-1K link
l1_base-nab NABirds link
l1_small-nab NABirds link
l1_tiny-nab NABirds link
l1_base-coco COCO 2014 link
l1_small-coco COCO 2014 link
l1_tiny-coco COCO 2014 link
l1_base-nus NUS-WIDE link
l1_small-nus NUS-WIDE link
l1_tiny-nus NUS-WIDE link

L2

Model Name Dataset Weights
l2_base-im1k ImageNet-1K link
l2_small-im1k ImageNet-1K link
l2_tiny-im1k ImageNet-1K link
l2_base-nab NABirds link
l2_small-nab NABirds link
l2_tiny-nab NABirds link
l2_base-coco COCO 2014 link
l2_small-coco COCO 2014 link
l2_tiny-coco COCO 2014 link
l2_base-nus NUS-WIDE link
l2_small-nus NUS-WIDE link
l2_tiny-nus NUS-WIDE link

L-Infinity

Model Name Dataset Weights
linf_base-im1k ImageNet-1K link
linf_small-im1k ImageNet-1K link
linf_tiny-im1k ImageNet-1K link
linf_base-nab NABirds link
linf_small-nab NABirds link
linf_tiny-nab NABirds link
linf_base-coco COCO 2014 link
linf_small-coco COCO 2014 link
linf_tiny-coco COCO 2014 link
linf_base-nus NUS-WIDE link
linf_small-nus NUS-WIDE link
linf_tiny-nus NUS-WIDE link

ToMe

Model Name Dataset Weights
tome_base-im1k ImageNet-1K link
tome_small-im1k ImageNet-1K link
tome_tiny-im1k ImageNet-1K link
tome_base-nab NABirds link
tome_small-nab NABirds link
tome_tiny-nab NABirds link
tome_base-coco COCO 2014 link
tome_small-coco COCO 2014 link
tome_tiny-coco COCO 2014 link
tome_base-nus NUS-WIDE link
tome_small-nus NUS-WIDE link
tome_tiny-nus NUS-WIDE link

K-Medoids

Model Name Dataset Weights
kmedoids_base-im1k ImageNet-1K link
kmedoids-small_im1k ImageNet-1K link
kmedoids_tiny-im1k ImageNet-1K link
kmedoids_base-nab NABirds link
kmedoids-small_nab NABirds link
kmedoids_tiny-nab NABirds link
kmedoids_base-coco COCO 2014 link
kmedoids-small_coco COCO 2014 link
kmedoids_tiny-coco COCO 2014 link
kmedoids_base-nus NUS-WIDE link
kmedoids-small_nus NUS-WIDE link
kmedoids_tiny-nus NUS-WIDE link

DPC-KNN

Model Name Dataset Weights
dpcknn_base-im1k ImageNet-1K link
dpcknn_small_im1k ImageNet-1K link
dpcknn_tiny-im1k ImageNet-1K link
dpcknn_base-nab NABirds link
dpcknn_small_nab NABirds link
dpcknn_tiny-nab NABirds link
dpcknn_base-coco COCO 2014 link
dpcknn_small_coco COCO 2014 link
dpcknn_tiny-coco COCO 2014 link
dpcknn_base-nus NUS-WIDE link
dpcknn_small_nus NUS-WIDE link
dpcknn_tiny-nus NUS-WIDE link

SiT

Model Name Dataset Weights
sit_base-im1k ImageNet-1K link
sit_small_im1k ImageNet-1K link
sit_tiny-im1k ImageNet-1K link
sit_base-nab NABirds link
sit_small_nab NABirds link
sit_tiny-nab NABirds link
sit_base-coco COCO 2014 link
sit_small_coco COCO 2014 link
sit_tiny-coco COCO 2014 link
sit_base-nus NUS-WIDE link
sit_small_nus NUS-WIDE link
sit_tiny-nus NUS-WIDE link

PatchMerger

Model Name Dataset Weights
patchmerger_base-im1k ImageNet-1K link
patchmerger_small_im1k ImageNet-1K link
patchmerger_tiny-im1k ImageNet-1K link
patchmerger_base-nab NABirds link
patchmerger_small_nab NABirds link
patchmerger_tiny-nab NABirds link
patchmerger_base-coco COCO 2014 link
patchmerger_small_coco COCO 2014 link
patchmerger_tiny-coco COCO 2014 link
patchmerger_base-nus NUS-WIDE link
patchmerger_small_nus NUS-WIDE link
patchmerger_tiny-nus NUS-WIDE link

Sinkhorn

Model Name Dataset Weights
sinkhorn_base-im1k ImageNet-1K link
sinkhorn_small_im1k ImageNet-1K link
sinkhorn_tiny-im1k ImageNet-1K link
sinkhorn_base-nab NABirds link
sinkhorn_small_nab NABirds link
sinkhorn_tiny-nab NABirds link
sinkhorn_base-coco COCO 2014 link
sinkhorn_small_coco COCO 2014 link
sinkhorn_tiny-coco COCO 2014 link
sinkhorn_base-nus NUS-WIDE link
sinkhorn_small_nus NUS-WIDE link
sinkhorn_tiny-nus NUS-WIDE link