transformers/main/en/main_classes/backbones.md

Backbone

A backbone is a model used for feature extraction for higher level computer vision tasks such as object detection and image classification. Transformers provides an AutoBackbone class for initializing a Transformers backbone from pretrained model weights, and two utility classes:

• BackboneMixin enables initializing a backbone from Transformers or timm and includes functions for returning the output features and indices.
• BackboneConfigMixin sets the output features and indices of the backbone configuration.

timm models are loaded with the TimmBackbone and TimmBackboneConfig classes.

Backbones are supported for the following models:

• BEiT
• BiT
• ConvNext
• ConvNextV2
• DiNAT
• DINOV2
• FocalNet
• MaskFormer
• NAT
• ResNet
• Swin Transformer
• Swin Transformer v2
• ViTDet

AutoBackbone[[transformers.AutoBackbone]]

transformers.AutoBackbone[[transformers.AutoBackbone]]

Source

BackboneMixin[[transformers.BackboneMixin]]

transformers.BackboneMixin[[transformers.BackboneMixin]]

Source

post_inittransformers.BackboneMixin.post_inithttps://github.com/huggingface/transformers/blob/main/src/transformers/backbone_utils.py#L207[]

Override post_init to always install capturing hooks, as backbone will ALWAYS capture outputs. We need to do it in post_init, as modules need to be already instantiated. It avoids some mixups with torch.compile, as the first hook installation will need/create a graph break, which can clash with external user call such as model = torch.compile(model...).

BackboneConfigMixin[[transformers.BackboneConfigMixin]]

transformers.BackboneConfigMixin[[transformers.BackboneConfigMixin]]

Source

A Mixin to support handling the out_features and out_indices attributes for the backbone configurations.

set_output_features_output_indicestransformers.BackboneConfigMixin.set_output_features_output_indiceshttps://github.com/huggingface/transformers/blob/main/src/transformers/backbone_utils.py#L40[{"name": "out_features", "val": ": list | None"}, {"name": "out_indices", "val": ": list | None"}]- out_features (list[str], optional) -- The names of the features for the backbone to output. Defaults to config._out_features if not provided.

• out_indices (list[int] or tuple[int], optional) -- The indices of the features for the backbone to output. Defaults to config._out_indices if not provided.0

Sets output indices and features to new values and aligns them with the given stage_names. If one of the inputs is not given, find the corresponding out_features or out_indices for the given stage_names.

Parameters:

out_features (list[str], optional) : The names of the features for the backbone to output. Defaults to config._out_features if not provided.

out_indices (list[int] or tuple[int], optional) : The indices of the features for the backbone to output. Defaults to config._out_indices if not provided.

to_dict[[transformers.BackboneConfigMixin.to_dict]]

Source

Serializes this instance to a Python dictionary. Override the default to_dict() from PreTrainedConfig to include the out_features and out_indices attributes.

verify_out_features_out_indices[[transformers.BackboneConfigMixin.verify_out_features_out_indices]]

Source

Verify that out_indices and out_features are valid for the given stage_names.

TimmBackbone[[transformers.TimmBackbone]]

transformers.TimmBackbone[[transformers.TimmBackbone]]

Source

Wrapper class for timm models to be used as backbones. This enables using the timm models interchangeably with the other models in the library keeping the same API.

TimmBackboneConfig[[transformers.TimmBackboneConfig]]

transformers.TimmBackboneConfig[[transformers.TimmBackboneConfig]]

Source

This is the configuration class to store the configuration of a TimmBackbone. It is used to instantiate a Timm Backbone model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the

Configuration objects inherit from PreTrainedConfig and can be used to control the model outputs. Read the documentation from PreTrainedConfig for more information.

Example:

>>> from transformers import TimmBackboneConfig, TimmBackbone

>>> # Initializing a timm backbone
>>> configuration = TimmBackboneConfig("resnet50")

>>> # Initializing a model from the configuration
>>> model = TimmBackbone(configuration)

>>> # Accessing the model configuration
>>> configuration = model.config

Parameters:

backbone (str, optional) : The timm checkpoint to load.

num_channels (int, optional, defaults to 3) : The number of input channels.

features_only (bool, optional, defaults to True) : Whether to output only the features or also the logits.

freeze_batch_norm_2d (bool, optional, defaults to False) : Converts all BatchNorm2d and SyncBatchNorm layers of provided module into FrozenBatchNorm2d.

output_stride (int, optional) : The ratio between the spatial resolution of the input and output feature maps.