transformers/pr_33892/en/internal/file_utils.md

General Utilities

This page lists all of Transformers general utility functions that are found in the file utils.py.

Most of those are only useful if you are studying the general code in the library.

Enums and namedtuples[[transformers.utils.ExplicitEnum]]

class transformers.utils.ExplicitEnumtransformers.utils.ExplicitEnumhttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/generic.py#L389[{"name": "value", "val": ""}, {"name": "names", "val": " = None"}, {"name": "module", "val": " = None"}, {"name": "qualname", "val": " = None"}, {"name": "type", "val": " = None"}, {"name": "start", "val": " = 1"}]

Enum with more explicit error message for missing values.

class transformers.utils.PaddingStrategytransformers.utils.PaddingStrategyhttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/generic.py#L401[{"name": "value", "val": ""}, {"name": "names", "val": " = None"}, {"name": "module", "val": " = None"}, {"name": "qualname", "val": " = None"}, {"name": "type", "val": " = None"}, {"name": "start", "val": " = 1"}]

Possible values for the padding argument in PreTrainedTokenizerBase.call(). Useful for tab-completion in an IDE.

class transformers.TensorTypetransformers.TensorTypehttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/generic.py#L412[{"name": "value", "val": ""}, {"name": "names", "val": " = None"}, {"name": "module", "val": " = None"}, {"name": "qualname", "val": " = None"}, {"name": "type", "val": " = None"}, {"name": "start", "val": " = 1"}]

Possible values for the return_tensors argument in PreTrainedTokenizerBase.call(). Useful for tab-completion in an IDE.

Special Decorators[[transformers.add_start_docstrings]]

transformers.add_start_docstringstransformers.add_start_docstringshttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/doc.py#L37[{"name": "*docstr", "val": ""}]

transformers.utils.add_start_docstrings_to_model_forwardtransformers.utils.add_start_docstrings_to_model_forwardhttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/doc.py#L45[{"name": "*docstr", "val": ""}]

transformers.add_end_docstringstransformers.add_end_docstringshttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/doc.py#L81[{"name": "*docstr", "val": ""}]

transformers.utils.add_code_sample_docstringstransformers.utils.add_code_sample_docstringshttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/doc.py#L1006[{"name": "*docstr", "val": ""}, {"name": "processor_class", "val": " = None"}, {"name": "checkpoint", "val": " = None"}, {"name": "output_type", "val": " = None"}, {"name": "config_class", "val": " = None"}, {"name": "mask", "val": " = '[MASK]'"}, {"name": "qa_target_start_index", "val": " = 14"}, {"name": "qa_target_end_index", "val": " = 15"}, {"name": "model_cls", "val": " = None"}, {"name": "modality", "val": " = None"}, {"name": "expected_output", "val": " = None"}, {"name": "expected_loss", "val": " = None"}, {"name": "real_checkpoint", "val": " = None"}, {"name": "revision", "val": " = None"}]

transformers.utils.replace_return_docstringstransformers.utils.replace_return_docstringshttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/doc.py#L1100[{"name": "output_type", "val": " = None"}, {"name": "config_class", "val": " = None"}]

Other Utilities[[transformers.utils._LazyModule]]

class transformers.utils._LazyModuletransformers.utils._LazyModulehttps://github.com/huggingface/transformers/blob/vr_33892/src/transformers/utils/import_utils.py#L1781[{"name": "name", "val": ": str"}, {"name": "module_file", "val": ": str"}, {"name": "import_structure", "val": ": dict"}, {"name": "module_spec", "val": ": _frozen_importlib.ModuleSpec | None = None"}, {"name": "extra_objects", "val": ": dict[str, object] | None = None"}, {"name": "explicit_import_shortcut", "val": ": dict[str, list[str]] | None = None"}]

Module class that surfaces all objects but only performs associated imports when the objects are requested.