transformers/pr_48167/en/main_classes/backbones.md

Backbone

A backbone is a model used for feature extraction for higher level computer vision tasks such as object detection and image classification. Transformers provides an AutoBackbone class for initializing a Transformers backbone from pretrained model weights, and two utility classes:

• BackboneMixin enables initializing a backbone from Transformers or timm and includes functions for returning the output features and indices.
• BackboneConfigMixin sets the output features and indices of the backbone configuration.

timm models are loaded with the TimmBackbone and TimmBackboneConfig classes.

Backbones are supported for the following models:

• BEiT
• BiT
• ConvNext
• ConvNextV2
• DiNAT
• DINOV2
• FocalNet
• MaskFormer
• NAT
• ResNet
• Swin Transformer
• Swin Transformer v2
• ViTDet

AutoBackbone[[transformers.AutoBackbone]]

transformers.AutoBackbone[[transformers.AutoBackbone]]

transformers.AutoBackbone(*args, **kwargs)

Source

BackboneMixin[[transformers.BackboneMixin]]

transformers.BackboneMixin[[transformers.BackboneMixin]]

transformers.BackboneMixin(*args, **kwargs)

Source

post_init[[transformers.BackboneMixin.post_init]]

post_init()

Source

Override post_init to always install capturing hooks, as backbone will ALWAYS capture outputs. We need to do it in post_init, as modules need to be already instantiated. It avoids some mixups with torch.compile, as the first hook installation will need/create a graph break, which can clash with external user call such as model = torch.compile(model...).

BackboneConfigMixin[[transformers.BackboneConfigMixin]]

transformers.BackboneConfigMixin[[transformers.BackboneConfigMixin]]

transformers.BackboneConfigMixin()

Source

A Mixin to support handling the out_features and out_indices attributes for the backbone configurations.

set_output_features_output_indices[[transformers.BackboneConfigMixin.set_output_features_output_indices]]

set_output_features_output_indices(out_features: list | None, out_indices: list | None)

Source

Parameters:

out_features (list[str], optional) : The names of the features for the backbone to output. Defaults to config._out_features if not provided.

out_indices (list[int] or tuple[int], optional) : The indices of the features for the backbone to output. Defaults to config._out_indices if not provided.

Sets output indices and features to new values and aligns them with the given stage_names. If one of the inputs is not given, find the corresponding out_features or out_indices for the given stage_names.

to_dict[[transformers.BackboneConfigMixin.to_dict]]

to_dict()

Source

Serializes this instance to a Python dictionary. Override the default to_dict() from PreTrainedConfig to include the out_features and out_indices attributes.

verify_out_features_out_indices[[transformers.BackboneConfigMixin.verify_out_features_out_indices]]

verify_out_features_out_indices()

Source

Verify that out_indices and out_features are valid for the given stage_names.

TimmBackbone[[transformers.TimmBackbone]]

transformers.TimmBackbone[[transformers.TimmBackbone]]

transformers.TimmBackbone(config, **kwargs)

Source

Wrapper class for timm models to be used as backbones. This enables using the timm models interchangeably with the other models in the library keeping the same API.

TimmBackboneConfig[[transformers.TimmBackboneConfig]]

transformers.TimmBackboneConfig[[transformers.TimmBackboneConfig]]

transformers.TimmBackboneConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: typing.Union[str, ForwardRef('torch.dtype'), NoneType] = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: typing.Optional[typing.Literal['regression', 'single_label_classification', 'multi_label_classification']] = None, backbone: str | None = None, num_channels: int = 3, features_only: bool = True, _out_indices: list[int] | None = None, freeze_batch_norm_2d: bool = False, output_stride: int | None = None)

Source

Parameters:

backbone (str, optional) : The timm checkpoint to load.

num_channels (int, optional, defaults to 3) : The number of input channels.

features_only (bool, optional, defaults to True) : Whether to output only the features or also the logits.

freeze_batch_norm_2d (bool, optional, defaults to False) : Converts all BatchNorm2d and SyncBatchNorm layers of provided module into FrozenBatchNorm2d.

output_stride (int, optional) : The ratio between the spatial resolution of the input and output feature maps.

This is the configuration class to store the configuration of a TimmBackbone. It is used to instantiate a Timm Backbone model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the

Configuration objects inherit from PreTrainedConfig and can be used to control the model outputs. Read the documentation from PreTrainedConfig for more information.

Example:

>>> from transformers import TimmBackboneConfig, TimmBackbone

>>> # Initializing a timm backbone
>>> configuration = TimmBackboneConfig("resnet50")

>>> # Initializing a model from the configuration
>>> model = TimmBackbone(configuration)

>>> # Accessing the model configuration
>>> configuration = model.config