| | --- |
| | library_name: keras-hub |
| | --- |
| | ### Model Overview |
| | Data-efficient Image Transformer (DeiT). |
| |
|
| | **Reference** |
| |
|
| | - [Training data-efficient image transformers & distillation through attention](https://arxiv.org/abs/2012.12877) |
| |
|
| | ViT models required training on expensive infrastructure for multiple weeks, using external data. DeiT (data-efficient image transformers) are more efficiently trained transformers for image classification, requiring far less data and far less computing resources compared to the original ViT models. |
| |
|
| | ## Links |
| |
|
| | * [DeiT Quickstart Notebook] - coming soon |
| | * [DeiT API Documentation] - coming soon |
| | * [DeiT Beginner Guide] - coming soon |
| | * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/) |
| |
|
| |
|
| | ## Installation |
| |
|
| | Keras and KerasHub can be installed with: |
| |
|
| | ``` |
| | pip install -U -q keras-hub |
| | pip install -U -q keras |
| | ``` |
| |
|
| | Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page. |
| |
|
| | ## Presets |
| |
|
| | The following model checkpoints are provided by the Keras team. Weights have been ported from: https://huggingface.co. Full code examples for each are available below. |
| |
|
| | | Preset name | Parameters | Description | |
| | |------------------------------------|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
| | | deit_tiny_distilled_patch16_224_imagenet | 5.52M | DeiT-T16 model pre-trained on the ImageNet 1k dataset with image resolution of 224x224 | |
| | | deit_small_distilled_patch16_224_imagenet | 21.66M | DeiT-S16 model pre-trained on the ImageNet 1k dataset with image resolution of 224x224 | |
| | | deit_base_distilled_patch16_224_imagenet | 85.80M | DeiT-B16 model pre-trained on the ImageNet 1k dataset with image resolution of 224x224 . | |
| | | deit_base_distilled_patch16_384_imagenet | 86.09M | DeiT-B16 model pre-trained on the ImageNet 1k dataset with image resolution of 384x384 | |
| |
|
| |
|
