Buckets:
| # LayoutLM | |
| [LayoutLM](https://huggingface.co/papers/1912.13318) jointly learns text and the document layout rather than focusing only on text. It incorporates positional layout information and visual features of words from the document images. | |
| You can find all the original LayoutLM checkpoints under the [LayoutLM](https://huggingface.co/collections/microsoft/layoutlm-6564539601de72cb631d0902) collection. | |
| > [!TIP] | |
| > Click on the LayoutLM models in the right sidebar for more examples of how to apply LayoutLM to different vision and language tasks. | |
| The example below demonstrates question answering with the [AutoModel](/docs/transformers/pr_33962/en/model_doc/auto#transformers.AutoModel) class. | |
| <hfoptions id="usage"> | |
| <hfoption id="AutoModel"> | |
| ```py | |
| import torch | |
| from datasets import load_dataset | |
| from transformers import AutoTokenizer, LayoutLMForQuestionAnswering | |
| tokenizer = AutoTokenizer.from_pretrained("impira/layoutlm-document-qa", add_prefix_space=True) | |
| model = LayoutLMForQuestionAnswering.from_pretrained("impira/layoutlm-document-qa", dtype=torch.float16) | |
| dataset = load_dataset("nielsr/funsd", split="train") | |
| example = dataset[0] | |
| question = "what's his name?" | |
| words = example["words"] | |
| boxes = example["bboxes"] | |
| encoding = tokenizer( | |
| question.split(), | |
| words, | |
| is_split_into_words=True, | |
| return_token_type_ids=True, | |
| return_tensors="pt" | |
| ) | |
| bbox = [] | |
| for i, s, w in zip(encoding.input_ids[0], encoding.sequence_ids(0), encoding.word_ids(0)): | |
| if s == 1: | |
| bbox.append(boxes[w]) | |
| elif i == tokenizer.sep_token_id: | |
| bbox.append([1000] * 4) | |
| else: | |
| bbox.append([0] * 4) | |
| encoding["bbox"] = torch.tensor([bbox]) | |
| word_ids = encoding.word_ids(0) | |
| outputs = model(**encoding) | |
| loss = outputs.loss | |
| start_scores = outputs.start_logits | |
| end_scores = outputs.end_logits | |
| start, end = word_ids[start_scores.argmax(-1)], word_ids[end_scores.argmax(-1)] | |
| print(" ".join(words[start : end + 1])) | |
| ``` | |
| </hfoption> | |
| </hfoptions> | |
| ## Notes | |
| - The original LayoutLM was not designed with a unified processing workflow. Instead, it expects preprocessed text (`words`) and bounding boxes (`boxes`) from an external OCR engine (like [Pytesseract](https://pypi.org/project/pytesseract/)) and provide them as additional inputs to the tokenizer. | |
| - The [forward()](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMModel.forward) method expects the input `bbox` (bounding boxes of the input tokens). Each bounding box should be in the format `(x0, y0, x1, y1)`. `(x0, y0)` corresponds to the upper left corner of the bounding box and `{x1, y1)` corresponds to the lower right corner. The bounding boxes need to be normalized on a 0-1000 scale as shown below. | |
| ```python | |
| def normalize_bbox(bbox, width, height): | |
| return [ | |
| int(1000 * (bbox[0] / width)), | |
| int(1000 * (bbox[1] / height)), | |
| int(1000 * (bbox[2] / width)), | |
| int(1000 * (bbox[3] / height)), | |
| ] | |
| ``` | |
| - `width` and `height` correspond to the width and height of the original document in which the token occurs. These values can be obtained as shown below. | |
| ```python | |
| from PIL import Image | |
| # Document can be a png, jpg, etc. PDFs must be converted to images. | |
| image = Image.open(name_of_your_document).convert("RGB") | |
| width, height = image.size | |
| ``` | |
| ## Resources | |
| A list of official Hugging Face and community (indicated by 🌎) resources to help you get started with LayoutLM. If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource. | |
| - Read [fine-tuning LayoutLM for document-understanding using Keras & Hugging Face Transformers](https://www.philschmid.de/fine-tuning-layoutlm-keras) to learn more. | |
| - Read [fine-tune LayoutLM for document-understanding using only Hugging Face Transformers](https://www.philschmid.de/fine-tuning-layoutlm) for more information. | |
| - Refer to this [notebook](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/LayoutLM/Add_image_embeddings_to_LayoutLM.ipynb) for a practical example of how to fine-tune LayoutLM. | |
| - Refer to this [notebook](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/LayoutLM/Fine_tuning_LayoutLMForSequenceClassification_on_RVL_CDIP.ipynb) for an example of how to fine-tune LayoutLM for sequence classification. | |
| - Refer to this [notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LayoutLM/Fine_tuning_LayoutLMForTokenClassification_on_FUNSD.ipynb) for an example of how to fine-tune LayoutLM for token classification. | |
| - Read [Deploy LayoutLM with Hugging Face Inference Endpoints](https://www.philschmid.de/inference-endpoints-layoutlm) to learn how to deploy LayoutLM. | |
| ## LayoutLMConfig[[transformers.LayoutLMConfig]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMConfig</name><anchor>transformers.LayoutLMConfig</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/configuration_layoutlm.py#L29</source><parameters>[{"name": "vocab_size", "val": " = 30522"}, {"name": "hidden_size", "val": " = 768"}, {"name": "num_hidden_layers", "val": " = 12"}, {"name": "num_attention_heads", "val": " = 12"}, {"name": "intermediate_size", "val": " = 3072"}, {"name": "hidden_act", "val": " = 'gelu'"}, {"name": "hidden_dropout_prob", "val": " = 0.1"}, {"name": "attention_probs_dropout_prob", "val": " = 0.1"}, {"name": "max_position_embeddings", "val": " = 512"}, {"name": "type_vocab_size", "val": " = 2"}, {"name": "initializer_range", "val": " = 0.02"}, {"name": "layer_norm_eps", "val": " = 1e-12"}, {"name": "pad_token_id", "val": " = 0"}, {"name": "use_cache", "val": " = True"}, {"name": "max_2d_position_embeddings", "val": " = 1024"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **vocab_size** (`int`, *optional*, defaults to 30522) -- | |
| Vocabulary size of the LayoutLM model. Defines the different tokens that can be represented by the | |
| *inputs_ids* passed to the forward method of [LayoutLMModel](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMModel). | |
| - **hidden_size** (`int`, *optional*, defaults to 768) -- | |
| Dimensionality of the encoder layers and the pooler layer. | |
| - **num_hidden_layers** (`int`, *optional*, defaults to 12) -- | |
| Number of hidden layers in the Transformer encoder. | |
| - **num_attention_heads** (`int`, *optional*, defaults to 12) -- | |
| Number of attention heads for each attention layer in the Transformer encoder. | |
| - **intermediate_size** (`int`, *optional*, defaults to 3072) -- | |
| Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. | |
| - **hidden_act** (`str` or `function`, *optional*, defaults to `"gelu"`) -- | |
| The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, | |
| `"relu"`, `"silu"` and `"gelu_new"` are supported. | |
| - **hidden_dropout_prob** (`float`, *optional*, defaults to 0.1) -- | |
| The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. | |
| - **attention_probs_dropout_prob** (`float`, *optional*, defaults to 0.1) -- | |
| The dropout ratio for the attention probabilities. | |
| - **max_position_embeddings** (`int`, *optional*, defaults to 512) -- | |
| The maximum sequence length that this model might ever be used with. Typically set this to something large | |
| just in case (e.g., 512 or 1024 or 2048). | |
| - **type_vocab_size** (`int`, *optional*, defaults to 2) -- | |
| The vocabulary size of the `token_type_ids` passed into [LayoutLMModel](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMModel). | |
| - **initializer_range** (`float`, *optional*, defaults to 0.02) -- | |
| The standard deviation of the truncated_normal_initializer for initializing all weight matrices. | |
| - **layer_norm_eps** (`float`, *optional*, defaults to 1e-12) -- | |
| The epsilon used by the layer normalization layers. | |
| - **pad_token_id** (`int`, *optional*, defaults to 0) -- | |
| The value used to pad input_ids. | |
| - **use_cache** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not the model should return the last key/values attentions (not used by all models). Only | |
| relevant if `config.is_decoder=True`. | |
| - **max_2d_position_embeddings** (`int`, *optional*, defaults to 1024) -- | |
| The maximum value that the 2D position embedding might ever used. Typically set this to something large | |
| just in case (e.g., 1024).</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| This is the configuration class to store the configuration of a [LayoutLMModel](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMModel). It is used to instantiate a | |
| LayoutLM model according to the specified arguments, defining the model architecture. Instantiating a configuration | |
| with the defaults will yield a similar configuration to that of the LayoutLM | |
| [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) architecture. | |
| Configuration objects inherit from [BertConfig](/docs/transformers/pr_33962/en/model_doc/bert#transformers.BertConfig) and can be used to control the model outputs. Read the | |
| documentation from [BertConfig](/docs/transformers/pr_33962/en/model_doc/bert#transformers.BertConfig) for more information. | |
| <ExampleCodeBlock anchor="transformers.LayoutLMConfig.example"> | |
| Examples: | |
| ```python | |
| >>> from transformers import LayoutLMConfig, LayoutLMModel | |
| >>> # Initializing a LayoutLM configuration | |
| >>> configuration = LayoutLMConfig() | |
| >>> # Initializing a model (with random weights) from the configuration | |
| >>> model = LayoutLMModel(configuration) | |
| >>> # Accessing the model configuration | |
| >>> configuration = model.config | |
| ``` | |
| </ExampleCodeBlock> | |
| </div> | |
| ## LayoutLMTokenizer[[transformers.LayoutLMTokenizer]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMTokenizer</name><anchor>transformers.LayoutLMTokenizer</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/tokenization_layoutlm.py#L54</source><parameters>[{"name": "vocab_file", "val": ""}, {"name": "do_lower_case", "val": " = True"}, {"name": "do_basic_tokenize", "val": " = True"}, {"name": "never_split", "val": " = None"}, {"name": "unk_token", "val": " = '[UNK]'"}, {"name": "sep_token", "val": " = '[SEP]'"}, {"name": "pad_token", "val": " = '[PAD]'"}, {"name": "cls_token", "val": " = '[CLS]'"}, {"name": "mask_token", "val": " = '[MASK]'"}, {"name": "tokenize_chinese_chars", "val": " = True"}, {"name": "strip_accents", "val": " = None"}, {"name": "clean_up_tokenization_spaces", "val": " = True"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **vocab_file** (`str`) -- | |
| File containing the vocabulary. | |
| - **do_lower_case** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to lowercase the input when tokenizing. | |
| - **do_basic_tokenize** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to do basic tokenization before WordPiece. | |
| - **never_split** (`Iterable`, *optional*) -- | |
| Collection of tokens which will never be split during tokenization. Only has an effect when | |
| `do_basic_tokenize=True` | |
| - **unk_token** (`str`, *optional*, defaults to `"[UNK]"`) -- | |
| The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this | |
| token instead. | |
| - **sep_token** (`str`, *optional*, defaults to `"[SEP]"`) -- | |
| The separator token, which is used when building a sequence from multiple sequences, e.g. two sequences for | |
| sequence classification or for a text and a question for question answering. It is also used as the last | |
| token of a sequence built with special tokens. | |
| - **pad_token** (`str`, *optional*, defaults to `"[PAD]"`) -- | |
| The token used for padding, for example when batching sequences of different lengths. | |
| - **cls_token** (`str`, *optional*, defaults to `"[CLS]"`) -- | |
| The classifier token which is used when doing sequence classification (classification of the whole sequence | |
| instead of per-token classification). It is the first token of the sequence when built with special tokens. | |
| - **mask_token** (`str`, *optional*, defaults to `"[MASK]"`) -- | |
| The token used for masking values. This is the token used when training this model with masked language | |
| modeling. This is the token which the model will try to predict. | |
| - **tokenize_chinese_chars** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to tokenize Chinese characters. | |
| This should likely be deactivated for Japanese (see this | |
| [issue](https://github.com/huggingface/transformers/issues/328)). | |
| - **strip_accents** (`bool`, *optional*) -- | |
| Whether or not to strip all accents. If this option is not specified, then it will be determined by the | |
| value for `lowercase` (as in the original LayoutLM). | |
| - **clean_up_tokenization_spaces** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to cleanup spaces after decoding, cleanup consists in removing potential artifacts like | |
| extra spaces.</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| Construct a LayoutLM tokenizer. Based on WordPiece. | |
| This tokenizer inherits from [PreTrainedTokenizer](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.PreTrainedTokenizer) which contains most of the main methods. Users should refer to | |
| this superclass for more information regarding those methods. | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>__call__</name><anchor>transformers.LayoutLMTokenizer.__call__</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/tokenization_utils_base.py#L2874</source><parameters>[{"name": "text", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "text_pair", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "text_target", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "text_pair_target", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "add_special_tokens", "val": ": bool = True"}, {"name": "padding", "val": ": typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = False"}, {"name": "truncation", "val": ": typing.Union[bool, str, transformers.tokenization_utils_base.TruncationStrategy, NoneType] = None"}, {"name": "max_length", "val": ": typing.Optional[int] = None"}, {"name": "stride", "val": ": int = 0"}, {"name": "is_split_into_words", "val": ": bool = False"}, {"name": "pad_to_multiple_of", "val": ": typing.Optional[int] = None"}, {"name": "padding_side", "val": ": typing.Optional[str] = None"}, {"name": "return_tensors", "val": ": typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None"}, {"name": "return_token_type_ids", "val": ": typing.Optional[bool] = None"}, {"name": "return_attention_mask", "val": ": typing.Optional[bool] = None"}, {"name": "return_overflowing_tokens", "val": ": bool = False"}, {"name": "return_special_tokens_mask", "val": ": bool = False"}, {"name": "return_offsets_mapping", "val": ": bool = False"}, {"name": "return_length", "val": ": bool = False"}, {"name": "verbose", "val": ": bool = True"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **text** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings | |
| (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set | |
| `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **text_pair** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings | |
| (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set | |
| `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **text_target** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded as target texts. Each sequence can be a string or a | |
| list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), | |
| you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **text_pair_target** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded as target texts. Each sequence can be a string or a | |
| list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), | |
| you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **add_special_tokens** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to add special tokens when encoding the sequences. This will use the underlying | |
| `PretrainedTokenizerBase.build_inputs_with_special_tokens` function, which defines which tokens are | |
| automatically added to the input ids. This is useful if you want to add `bos` or `eos` tokens | |
| automatically. | |
| - **padding** (`bool`, `str` or [PaddingStrategy](/docs/transformers/pr_33962/en/internal/file_utils#transformers.utils.PaddingStrategy), *optional*, defaults to `False`) -- | |
| Activates and controls padding. Accepts the following values: | |
| - `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single | |
| sequence is provided). | |
| - `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum | |
| acceptable input length for the model if that argument is not provided. | |
| - `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different | |
| lengths). | |
| - **truncation** (`bool`, `str` or [TruncationStrategy](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.tokenization_utils_base.TruncationStrategy), *optional*, defaults to `False`) -- | |
| Activates and controls truncation. Accepts the following values: | |
| - `True` or `'longest_first'`: Truncate to a maximum length specified with the argument `max_length` or | |
| to the maximum acceptable input length for the model if that argument is not provided. This will | |
| truncate token by token, removing a token from the longest sequence in the pair if a pair of | |
| sequences (or a batch of pairs) is provided. | |
| - `'only_first'`: Truncate to a maximum length specified with the argument `max_length` or to the | |
| maximum acceptable input length for the model if that argument is not provided. This will only | |
| truncate the first sequence of a pair if a pair of sequences (or a batch of pairs) is provided. | |
| - `'only_second'`: Truncate to a maximum length specified with the argument `max_length` or to the | |
| maximum acceptable input length for the model if that argument is not provided. This will only | |
| truncate the second sequence of a pair if a pair of sequences (or a batch of pairs) is provided. | |
| - `False` or `'do_not_truncate'` (default): No truncation (i.e., can output batch with sequence lengths | |
| greater than the model maximum admissible input size). | |
| - **max_length** (`int`, *optional*) -- | |
| Controls the maximum length to use by one of the truncation/padding parameters. | |
| If left unset or set to `None`, this will use the predefined model maximum length if a maximum length | |
| is required by one of the truncation/padding parameters. If the model has no specific maximum input | |
| length (like XLNet) truncation/padding to a maximum length will be deactivated. | |
| - **stride** (`int`, *optional*, defaults to 0) -- | |
| If set to a number along with `max_length`, the overflowing tokens returned when | |
| `return_overflowing_tokens=True` will contain some tokens from the end of the truncated sequence | |
| returned to provide some overlap between truncated and overflowing sequences. The value of this | |
| argument defines the number of overlapping tokens. | |
| - **is_split_into_words** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not the input is already pre-tokenized (e.g., split into words). If set to `True`, the | |
| tokenizer assumes the input is already split into words (for instance, by splitting it on whitespace) | |
| which it will tokenize. This is useful for NER or token classification. | |
| - **pad_to_multiple_of** (`int`, *optional*) -- | |
| If set will pad the sequence to a multiple of the provided value. Requires `padding` to be activated. | |
| This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability | |
| `>= 7.5` (Volta). | |
| - **padding_side** (`str`, *optional*) -- | |
| The side on which the model should have padding applied. Should be selected between ['right', 'left']. | |
| Default value is picked from the class attribute of the same name. | |
| - **return_tensors** (`str` or [TensorType](/docs/transformers/pr_33962/en/internal/file_utils#transformers.TensorType), *optional*) -- | |
| If set, will return tensors instead of list of python integers. Acceptable values are: | |
| - `'pt'`: Return PyTorch `torch.Tensor` objects. | |
| - `'np'`: Return Numpy `np.ndarray` objects. | |
| - **return_token_type_ids** (`bool`, *optional*) -- | |
| Whether to return token type IDs. If left to the default, will return the token type IDs according to | |
| the specific tokenizer's default, defined by the `return_outputs` attribute. | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **return_attention_mask** (`bool`, *optional*) -- | |
| Whether to return the attention mask. If left to the default, will return the attention mask according | |
| to the specific tokenizer's default, defined by the `return_outputs` attribute. | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **return_overflowing_tokens** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return overflowing token sequences. If a pair of sequences of input ids (or a batch | |
| of pairs) is provided with `truncation_strategy = longest_first` or `True`, an error is raised instead | |
| of returning overflowing tokens. | |
| - **return_special_tokens_mask** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return special tokens mask information. | |
| - **return_offsets_mapping** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return `(char_start, char_end)` for each token. | |
| This is only available on fast tokenizers inheriting from [PreTrainedTokenizerFast](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.PreTrainedTokenizerFast), if using | |
| Python's tokenizer, this method will raise `NotImplementedError`. | |
| - **return_length** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return the lengths of the encoded inputs. | |
| - **verbose** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to print more information and warnings. | |
| - ****kwargs** -- passed to the `self.tokenize()` method</paramsdesc><paramgroups>0</paramgroups><rettype>[BatchEncoding](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.BatchEncoding)</rettype><retdesc>A [BatchEncoding](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.BatchEncoding) with the following fields: | |
| - **input_ids** -- List of token ids to be fed to a model. | |
| [What are input IDs?](../glossary#input-ids) | |
| - **token_type_ids** -- List of token type ids to be fed to a model (when `return_token_type_ids=True` or | |
| if *"token_type_ids"* is in `self.model_input_names`). | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when | |
| `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names`). | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **overflowing_tokens** -- List of overflowing tokens sequences (when a `max_length` is specified and | |
| `return_overflowing_tokens=True`). | |
| - **num_truncated_tokens** -- Number of tokens truncated (when a `max_length` is specified and | |
| `return_overflowing_tokens=True`). | |
| - **special_tokens_mask** -- List of 0s and 1s, with 1 specifying added special tokens and 0 specifying | |
| regular sequence tokens (when `add_special_tokens=True` and `return_special_tokens_mask=True`). | |
| - **length** -- The length of the inputs (when `return_length=True`)</retdesc></docstring> | |
| Main method to tokenize and prepare for the model one or several sequence(s) or one or several pair(s) of | |
| sequences. | |
| </div></div> | |
| ## LayoutLMTokenizerFast[[transformers.LayoutLMTokenizerFast]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMTokenizerFast</name><anchor>transformers.LayoutLMTokenizerFast</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/tokenization_layoutlm_fast.py#L33</source><parameters>[{"name": "vocab_file", "val": " = None"}, {"name": "tokenizer_file", "val": " = None"}, {"name": "do_lower_case", "val": " = True"}, {"name": "unk_token", "val": " = '[UNK]'"}, {"name": "sep_token", "val": " = '[SEP]'"}, {"name": "pad_token", "val": " = '[PAD]'"}, {"name": "cls_token", "val": " = '[CLS]'"}, {"name": "mask_token", "val": " = '[MASK]'"}, {"name": "tokenize_chinese_chars", "val": " = True"}, {"name": "strip_accents", "val": " = None"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **vocab_file** (`str`) -- | |
| File containing the vocabulary. | |
| - **do_lower_case** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to lowercase the input when tokenizing. | |
| - **unk_token** (`str`, *optional*, defaults to `"[UNK]"`) -- | |
| The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this | |
| token instead. | |
| - **sep_token** (`str`, *optional*, defaults to `"[SEP]"`) -- | |
| The separator token, which is used when building a sequence from multiple sequences, e.g. two sequences for | |
| sequence classification or for a text and a question for question answering. It is also used as the last | |
| token of a sequence built with special tokens. | |
| - **pad_token** (`str`, *optional*, defaults to `"[PAD]"`) -- | |
| The token used for padding, for example when batching sequences of different lengths. | |
| - **cls_token** (`str`, *optional*, defaults to `"[CLS]"`) -- | |
| The classifier token which is used when doing sequence classification (classification of the whole sequence | |
| instead of per-token classification). It is the first token of the sequence when built with special tokens. | |
| - **mask_token** (`str`, *optional*, defaults to `"[MASK]"`) -- | |
| The token used for masking values. This is the token used when training this model with masked language | |
| modeling. This is the token which the model will try to predict. | |
| - **clean_text** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to clean the text before tokenization by removing any control characters and replacing all | |
| whitespaces by the classic one. | |
| - **tokenize_chinese_chars** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to tokenize Chinese characters. This should likely be deactivated for Japanese (see [this | |
| issue](https://github.com/huggingface/transformers/issues/328)). | |
| - **strip_accents** (`bool`, *optional*) -- | |
| Whether or not to strip all accents. If this option is not specified, then it will be determined by the | |
| value for `lowercase` (as in the original LayoutLM). | |
| - **wordpieces_prefix** (`str`, *optional*, defaults to `"##"`) -- | |
| The prefix for subwords.</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| Construct a "fast" LayoutLM tokenizer (backed by HuggingFace's *tokenizers* library). Based on WordPiece. | |
| This tokenizer inherits from [PreTrainedTokenizerFast](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.PreTrainedTokenizerFast) which contains most of the main methods. Users should | |
| refer to this superclass for more information regarding those methods. | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>__call__</name><anchor>transformers.LayoutLMTokenizerFast.__call__</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/tokenization_utils_base.py#L2874</source><parameters>[{"name": "text", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "text_pair", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "text_target", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "text_pair_target", "val": ": typing.Union[str, list[str], list[list[str]], NoneType] = None"}, {"name": "add_special_tokens", "val": ": bool = True"}, {"name": "padding", "val": ": typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = False"}, {"name": "truncation", "val": ": typing.Union[bool, str, transformers.tokenization_utils_base.TruncationStrategy, NoneType] = None"}, {"name": "max_length", "val": ": typing.Optional[int] = None"}, {"name": "stride", "val": ": int = 0"}, {"name": "is_split_into_words", "val": ": bool = False"}, {"name": "pad_to_multiple_of", "val": ": typing.Optional[int] = None"}, {"name": "padding_side", "val": ": typing.Optional[str] = None"}, {"name": "return_tensors", "val": ": typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None"}, {"name": "return_token_type_ids", "val": ": typing.Optional[bool] = None"}, {"name": "return_attention_mask", "val": ": typing.Optional[bool] = None"}, {"name": "return_overflowing_tokens", "val": ": bool = False"}, {"name": "return_special_tokens_mask", "val": ": bool = False"}, {"name": "return_offsets_mapping", "val": ": bool = False"}, {"name": "return_length", "val": ": bool = False"}, {"name": "verbose", "val": ": bool = True"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **text** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings | |
| (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set | |
| `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **text_pair** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings | |
| (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set | |
| `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **text_target** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded as target texts. Each sequence can be a string or a | |
| list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), | |
| you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **text_pair_target** (`str`, `list[str]`, `list[list[str]]`, *optional*) -- | |
| The sequence or batch of sequences to be encoded as target texts. Each sequence can be a string or a | |
| list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), | |
| you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). | |
| - **add_special_tokens** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to add special tokens when encoding the sequences. This will use the underlying | |
| `PretrainedTokenizerBase.build_inputs_with_special_tokens` function, which defines which tokens are | |
| automatically added to the input ids. This is useful if you want to add `bos` or `eos` tokens | |
| automatically. | |
| - **padding** (`bool`, `str` or [PaddingStrategy](/docs/transformers/pr_33962/en/internal/file_utils#transformers.utils.PaddingStrategy), *optional*, defaults to `False`) -- | |
| Activates and controls padding. Accepts the following values: | |
| - `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single | |
| sequence is provided). | |
| - `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum | |
| acceptable input length for the model if that argument is not provided. | |
| - `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different | |
| lengths). | |
| - **truncation** (`bool`, `str` or [TruncationStrategy](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.tokenization_utils_base.TruncationStrategy), *optional*, defaults to `False`) -- | |
| Activates and controls truncation. Accepts the following values: | |
| - `True` or `'longest_first'`: Truncate to a maximum length specified with the argument `max_length` or | |
| to the maximum acceptable input length for the model if that argument is not provided. This will | |
| truncate token by token, removing a token from the longest sequence in the pair if a pair of | |
| sequences (or a batch of pairs) is provided. | |
| - `'only_first'`: Truncate to a maximum length specified with the argument `max_length` or to the | |
| maximum acceptable input length for the model if that argument is not provided. This will only | |
| truncate the first sequence of a pair if a pair of sequences (or a batch of pairs) is provided. | |
| - `'only_second'`: Truncate to a maximum length specified with the argument `max_length` or to the | |
| maximum acceptable input length for the model if that argument is not provided. This will only | |
| truncate the second sequence of a pair if a pair of sequences (or a batch of pairs) is provided. | |
| - `False` or `'do_not_truncate'` (default): No truncation (i.e., can output batch with sequence lengths | |
| greater than the model maximum admissible input size). | |
| - **max_length** (`int`, *optional*) -- | |
| Controls the maximum length to use by one of the truncation/padding parameters. | |
| If left unset or set to `None`, this will use the predefined model maximum length if a maximum length | |
| is required by one of the truncation/padding parameters. If the model has no specific maximum input | |
| length (like XLNet) truncation/padding to a maximum length will be deactivated. | |
| - **stride** (`int`, *optional*, defaults to 0) -- | |
| If set to a number along with `max_length`, the overflowing tokens returned when | |
| `return_overflowing_tokens=True` will contain some tokens from the end of the truncated sequence | |
| returned to provide some overlap between truncated and overflowing sequences. The value of this | |
| argument defines the number of overlapping tokens. | |
| - **is_split_into_words** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not the input is already pre-tokenized (e.g., split into words). If set to `True`, the | |
| tokenizer assumes the input is already split into words (for instance, by splitting it on whitespace) | |
| which it will tokenize. This is useful for NER or token classification. | |
| - **pad_to_multiple_of** (`int`, *optional*) -- | |
| If set will pad the sequence to a multiple of the provided value. Requires `padding` to be activated. | |
| This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability | |
| `>= 7.5` (Volta). | |
| - **padding_side** (`str`, *optional*) -- | |
| The side on which the model should have padding applied. Should be selected between ['right', 'left']. | |
| Default value is picked from the class attribute of the same name. | |
| - **return_tensors** (`str` or [TensorType](/docs/transformers/pr_33962/en/internal/file_utils#transformers.TensorType), *optional*) -- | |
| If set, will return tensors instead of list of python integers. Acceptable values are: | |
| - `'pt'`: Return PyTorch `torch.Tensor` objects. | |
| - `'np'`: Return Numpy `np.ndarray` objects. | |
| - **return_token_type_ids** (`bool`, *optional*) -- | |
| Whether to return token type IDs. If left to the default, will return the token type IDs according to | |
| the specific tokenizer's default, defined by the `return_outputs` attribute. | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **return_attention_mask** (`bool`, *optional*) -- | |
| Whether to return the attention mask. If left to the default, will return the attention mask according | |
| to the specific tokenizer's default, defined by the `return_outputs` attribute. | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **return_overflowing_tokens** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return overflowing token sequences. If a pair of sequences of input ids (or a batch | |
| of pairs) is provided with `truncation_strategy = longest_first` or `True`, an error is raised instead | |
| of returning overflowing tokens. | |
| - **return_special_tokens_mask** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return special tokens mask information. | |
| - **return_offsets_mapping** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return `(char_start, char_end)` for each token. | |
| This is only available on fast tokenizers inheriting from [PreTrainedTokenizerFast](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.PreTrainedTokenizerFast), if using | |
| Python's tokenizer, this method will raise `NotImplementedError`. | |
| - **return_length** (`bool`, *optional*, defaults to `False`) -- | |
| Whether or not to return the lengths of the encoded inputs. | |
| - **verbose** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to print more information and warnings. | |
| - ****kwargs** -- passed to the `self.tokenize()` method</paramsdesc><paramgroups>0</paramgroups><rettype>[BatchEncoding](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.BatchEncoding)</rettype><retdesc>A [BatchEncoding](/docs/transformers/pr_33962/en/main_classes/tokenizer#transformers.BatchEncoding) with the following fields: | |
| - **input_ids** -- List of token ids to be fed to a model. | |
| [What are input IDs?](../glossary#input-ids) | |
| - **token_type_ids** -- List of token type ids to be fed to a model (when `return_token_type_ids=True` or | |
| if *"token_type_ids"* is in `self.model_input_names`). | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when | |
| `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names`). | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **overflowing_tokens** -- List of overflowing tokens sequences (when a `max_length` is specified and | |
| `return_overflowing_tokens=True`). | |
| - **num_truncated_tokens** -- Number of tokens truncated (when a `max_length` is specified and | |
| `return_overflowing_tokens=True`). | |
| - **special_tokens_mask** -- List of 0s and 1s, with 1 specifying added special tokens and 0 specifying | |
| regular sequence tokens (when `add_special_tokens=True` and `return_special_tokens_mask=True`). | |
| - **length** -- The length of the inputs (when `return_length=True`)</retdesc></docstring> | |
| Main method to tokenize and prepare for the model one or several sequence(s) or one or several pair(s) of | |
| sequences. | |
| </div></div> | |
| ## LayoutLMModel[[transformers.LayoutLMModel]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMModel</name><anchor>transformers.LayoutLMModel</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L452</source><parameters>[{"name": "config", "val": ""}]</parameters><paramsdesc>- **config** ([LayoutLMModel](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMModel)) -- | |
| Model configuration class with all the parameters of the model. Initializing with a config file does not | |
| load the weights associated with the model, only the configuration. Check out the | |
| [from_pretrained()](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights.</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| The bare Layoutlm Model outputting raw hidden-states without any specific head on top. | |
| This model inherits from [PreTrainedModel](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the | |
| library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads | |
| etc.) | |
| This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. | |
| Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage | |
| and behavior. | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>forward</name><anchor>transformers.LayoutLMModel.forward</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L470</source><parameters>[{"name": "input_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "bbox", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "attention_mask", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "token_type_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "position_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "inputs_embeds", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "output_attentions", "val": ": typing.Optional[bool] = None"}, {"name": "output_hidden_states", "val": ": typing.Optional[bool] = None"}, {"name": "return_dict", "val": ": typing.Optional[bool] = None"}]</parameters><paramsdesc>- **input_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. | |
| Indices can be obtained using [AutoTokenizer](/docs/transformers/pr_33962/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and | |
| [PreTrainedTokenizer.__call__()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. | |
| [What are input IDs?](../glossary#input-ids) | |
| - **bbox** (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*) -- | |
| Bounding boxes of each input sequence tokens. Selected in the range `[0, | |
| config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) | |
| format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, | |
| y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. | |
| - **attention_mask** (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: | |
| - 1 for tokens that are **not masked**, | |
| - 0 for tokens that are **masked**. | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **token_type_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: | |
| - 0 corresponds to a *sentence A* token, | |
| - 1 corresponds to a *sentence B* token. | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **position_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. | |
| [What are position IDs?](../glossary#position-ids) | |
| - **inputs_embeds** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) -- | |
| Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This | |
| is useful if you want more control over how to convert `input_ids` indices into associated vectors than the | |
| model's internal embedding lookup matrix. | |
| - **output_attentions** (`bool`, *optional*) -- | |
| Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned | |
| tensors for more detail. | |
| - **output_hidden_states** (`bool`, *optional*) -- | |
| Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for | |
| more detail. | |
| - **return_dict** (`bool`, *optional*) -- | |
| Whether or not to return a [ModelOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.utils.ModelOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[transformers.modeling_outputs.BaseModelOutputWithPooling](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPooling) or `tuple(torch.FloatTensor)`</rettype><retdesc>A [transformers.modeling_outputs.BaseModelOutputWithPooling](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.BaseModelOutputWithPooling) or a tuple of | |
| `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various | |
| elements depending on the configuration ([LayoutLMConfig](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMConfig)) and inputs. | |
| - **last_hidden_state** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`) -- Sequence of hidden-states at the output of the last layer of the model. | |
| - **pooler_output** (`torch.FloatTensor` of shape `(batch_size, hidden_size)`) -- Last layer hidden-state of the first token of the sequence (classification token) after further processing | |
| through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns | |
| the classification token after processing through a linear layer and a tanh activation function. The linear | |
| layer weights are trained from the next sentence prediction (classification) objective during pretraining. | |
| - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + | |
| one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. | |
| Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. | |
| - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, | |
| sequence_length)`. | |
| Attentions weights after the attention softmax, used to compute the weighted average in the self-attention | |
| heads.</retdesc></docstring> | |
| The [LayoutLMModel](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMModel) forward method, overrides the `__call__` special method. | |
| <Tip> | |
| Although the recipe for forward pass needs to be defined within this function, one should call the `Module` | |
| instance afterwards instead of this since the former takes care of running the pre and post processing steps while | |
| the latter silently ignores them. | |
| </Tip> | |
| <ExampleCodeBlock anchor="transformers.LayoutLMModel.forward.example"> | |
| Examples: | |
| ```python | |
| >>> from transformers import AutoTokenizer, LayoutLMModel | |
| >>> import torch | |
| >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> model = LayoutLMModel.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> words = ["Hello", "world"] | |
| >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] | |
| >>> token_boxes = [] | |
| >>> for word, box in zip(words, normalized_word_boxes): | |
| ... word_tokens = tokenizer.tokenize(word) | |
| ... token_boxes.extend([box] * len(word_tokens)) | |
| >>> # add bounding boxes of cls + sep tokens | |
| >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] | |
| >>> encoding = tokenizer(" ".join(words), return_tensors="pt") | |
| >>> input_ids = encoding["input_ids"] | |
| >>> attention_mask = encoding["attention_mask"] | |
| >>> token_type_ids = encoding["token_type_ids"] | |
| >>> bbox = torch.tensor([token_boxes]) | |
| >>> outputs = model( | |
| ... input_ids=input_ids, bbox=bbox, attention_mask=attention_mask, token_type_ids=token_type_ids | |
| ... ) | |
| >>> last_hidden_states = outputs.last_hidden_state | |
| ``` | |
| </ExampleCodeBlock> | |
| </div></div> | |
| ## LayoutLMForMaskedLM[[transformers.LayoutLMForMaskedLM]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMForMaskedLM</name><anchor>transformers.LayoutLMForMaskedLM</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L579</source><parameters>[{"name": "config", "val": ""}]</parameters><paramsdesc>- **config** ([LayoutLMForMaskedLM](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForMaskedLM)) -- | |
| Model configuration class with all the parameters of the model. Initializing with a config file does not | |
| load the weights associated with the model, only the configuration. Check out the | |
| [from_pretrained()](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights.</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| The Layoutlm Model with a `language modeling` head on top." | |
| This model inherits from [PreTrainedModel](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the | |
| library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads | |
| etc.) | |
| This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. | |
| Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage | |
| and behavior. | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>forward</name><anchor>transformers.LayoutLMForMaskedLM.forward</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L601</source><parameters>[{"name": "input_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "bbox", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "attention_mask", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "token_type_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "position_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "inputs_embeds", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "labels", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "output_attentions", "val": ": typing.Optional[bool] = None"}, {"name": "output_hidden_states", "val": ": typing.Optional[bool] = None"}, {"name": "return_dict", "val": ": typing.Optional[bool] = None"}]</parameters><paramsdesc>- **input_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. | |
| Indices can be obtained using [AutoTokenizer](/docs/transformers/pr_33962/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and | |
| [PreTrainedTokenizer.__call__()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. | |
| [What are input IDs?](../glossary#input-ids) | |
| - **bbox** (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*) -- | |
| Bounding boxes of each input sequence tokens. Selected in the range `[0, | |
| config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) | |
| format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, | |
| y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. | |
| - **attention_mask** (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: | |
| - 1 for tokens that are **not masked**, | |
| - 0 for tokens that are **masked**. | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **token_type_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: | |
| - 0 corresponds to a *sentence A* token, | |
| - 1 corresponds to a *sentence B* token. | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **position_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. | |
| [What are position IDs?](../glossary#position-ids) | |
| - **inputs_embeds** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) -- | |
| Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This | |
| is useful if you want more control over how to convert `input_ids` indices into associated vectors than the | |
| model's internal embedding lookup matrix. | |
| - **labels** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., | |
| config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the | |
| loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` | |
| - **output_attentions** (`bool`, *optional*) -- | |
| Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned | |
| tensors for more detail. | |
| - **output_hidden_states** (`bool`, *optional*) -- | |
| Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for | |
| more detail. | |
| - **return_dict** (`bool`, *optional*) -- | |
| Whether or not to return a [ModelOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.utils.ModelOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[transformers.modeling_outputs.MaskedLMOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.MaskedLMOutput) or `tuple(torch.FloatTensor)`</rettype><retdesc>A [transformers.modeling_outputs.MaskedLMOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.MaskedLMOutput) or a tuple of | |
| `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various | |
| elements depending on the configuration ([LayoutLMConfig](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMConfig)) and inputs. | |
| - **loss** (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) -- Masked language modeling (MLM) loss. | |
| - **logits** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) -- Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). | |
| - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + | |
| one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. | |
| Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. | |
| - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, | |
| sequence_length)`. | |
| Attentions weights after the attention softmax, used to compute the weighted average in the self-attention | |
| heads.</retdesc></docstring> | |
| The [LayoutLMForMaskedLM](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForMaskedLM) forward method, overrides the `__call__` special method. | |
| <Tip> | |
| Although the recipe for forward pass needs to be defined within this function, one should call the `Module` | |
| instance afterwards instead of this since the former takes care of running the pre and post processing steps while | |
| the latter silently ignores them. | |
| </Tip> | |
| <ExampleCodeBlock anchor="transformers.LayoutLMForMaskedLM.forward.example"> | |
| Examples: | |
| ```python | |
| >>> from transformers import AutoTokenizer, LayoutLMForMaskedLM | |
| >>> import torch | |
| >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> model = LayoutLMForMaskedLM.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> words = ["Hello", "[MASK]"] | |
| >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] | |
| >>> token_boxes = [] | |
| >>> for word, box in zip(words, normalized_word_boxes): | |
| ... word_tokens = tokenizer.tokenize(word) | |
| ... token_boxes.extend([box] * len(word_tokens)) | |
| >>> # add bounding boxes of cls + sep tokens | |
| >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] | |
| >>> encoding = tokenizer(" ".join(words), return_tensors="pt") | |
| >>> input_ids = encoding["input_ids"] | |
| >>> attention_mask = encoding["attention_mask"] | |
| >>> token_type_ids = encoding["token_type_ids"] | |
| >>> bbox = torch.tensor([token_boxes]) | |
| >>> labels = tokenizer("Hello world", return_tensors="pt")["input_ids"] | |
| >>> outputs = model( | |
| ... input_ids=input_ids, | |
| ... bbox=bbox, | |
| ... attention_mask=attention_mask, | |
| ... token_type_ids=token_type_ids, | |
| ... labels=labels, | |
| ... ) | |
| >>> loss = outputs.loss | |
| ``` | |
| </ExampleCodeBlock> | |
| </div></div> | |
| ## LayoutLMForSequenceClassification[[transformers.LayoutLMForSequenceClassification]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMForSequenceClassification</name><anchor>transformers.LayoutLMForSequenceClassification</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L703</source><parameters>[{"name": "config", "val": ""}]</parameters><paramsdesc>- **config** ([LayoutLMForSequenceClassification](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForSequenceClassification)) -- | |
| Model configuration class with all the parameters of the model. Initializing with a config file does not | |
| load the weights associated with the model, only the configuration. Check out the | |
| [from_pretrained()](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights.</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| LayoutLM Model with a sequence classification head on top (a linear layer on top of the pooled output) e.g. for | |
| document image classification tasks such as the [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/) dataset. | |
| This model inherits from [PreTrainedModel](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the | |
| library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads | |
| etc.) | |
| This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. | |
| Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage | |
| and behavior. | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>forward</name><anchor>transformers.LayoutLMForSequenceClassification.forward</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L717</source><parameters>[{"name": "input_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "bbox", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "attention_mask", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "token_type_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "position_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "inputs_embeds", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "labels", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "output_attentions", "val": ": typing.Optional[bool] = None"}, {"name": "output_hidden_states", "val": ": typing.Optional[bool] = None"}, {"name": "return_dict", "val": ": typing.Optional[bool] = None"}]</parameters><paramsdesc>- **input_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. | |
| Indices can be obtained using [AutoTokenizer](/docs/transformers/pr_33962/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and | |
| [PreTrainedTokenizer.__call__()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. | |
| [What are input IDs?](../glossary#input-ids) | |
| - **bbox** (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*) -- | |
| Bounding boxes of each input sequence tokens. Selected in the range `[0, | |
| config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) | |
| format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, | |
| y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. | |
| - **attention_mask** (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: | |
| - 1 for tokens that are **not masked**, | |
| - 0 for tokens that are **masked**. | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **token_type_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: | |
| - 0 corresponds to a *sentence A* token, | |
| - 1 corresponds to a *sentence B* token. | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **position_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. | |
| [What are position IDs?](../glossary#position-ids) | |
| - **inputs_embeds** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) -- | |
| Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This | |
| is useful if you want more control over how to convert `input_ids` indices into associated vectors than the | |
| model's internal embedding lookup matrix. | |
| - **labels** (`torch.LongTensor` of shape `(batch_size,)`, *optional*) -- | |
| Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., | |
| config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If | |
| `config.num_labels > 1` a classification loss is computed (Cross-Entropy). | |
| - **output_attentions** (`bool`, *optional*) -- | |
| Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned | |
| tensors for more detail. | |
| - **output_hidden_states** (`bool`, *optional*) -- | |
| Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for | |
| more detail. | |
| - **return_dict** (`bool`, *optional*) -- | |
| Whether or not to return a [ModelOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.utils.ModelOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[transformers.modeling_outputs.SequenceClassifierOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.SequenceClassifierOutput) or `tuple(torch.FloatTensor)`</rettype><retdesc>A [transformers.modeling_outputs.SequenceClassifierOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.SequenceClassifierOutput) or a tuple of | |
| `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various | |
| elements depending on the configuration ([LayoutLMConfig](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMConfig)) and inputs. | |
| - **loss** (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) -- Classification (or regression if config.num_labels==1) loss. | |
| - **logits** (`torch.FloatTensor` of shape `(batch_size, config.num_labels)`) -- Classification (or regression if config.num_labels==1) scores (before SoftMax). | |
| - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + | |
| one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. | |
| Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. | |
| - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, | |
| sequence_length)`. | |
| Attentions weights after the attention softmax, used to compute the weighted average in the self-attention | |
| heads.</retdesc></docstring> | |
| The [LayoutLMForSequenceClassification](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForSequenceClassification) forward method, overrides the `__call__` special method. | |
| <Tip> | |
| Although the recipe for forward pass needs to be defined within this function, one should call the `Module` | |
| instance afterwards instead of this since the former takes care of running the pre and post processing steps while | |
| the latter silently ignores them. | |
| </Tip> | |
| <ExampleCodeBlock anchor="transformers.LayoutLMForSequenceClassification.forward.example"> | |
| Examples: | |
| ```python | |
| >>> from transformers import AutoTokenizer, LayoutLMForSequenceClassification | |
| >>> import torch | |
| >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> model = LayoutLMForSequenceClassification.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> words = ["Hello", "world"] | |
| >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] | |
| >>> token_boxes = [] | |
| >>> for word, box in zip(words, normalized_word_boxes): | |
| ... word_tokens = tokenizer.tokenize(word) | |
| ... token_boxes.extend([box] * len(word_tokens)) | |
| >>> # add bounding boxes of cls + sep tokens | |
| >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] | |
| >>> encoding = tokenizer(" ".join(words), return_tensors="pt") | |
| >>> input_ids = encoding["input_ids"] | |
| >>> attention_mask = encoding["attention_mask"] | |
| >>> token_type_ids = encoding["token_type_ids"] | |
| >>> bbox = torch.tensor([token_boxes]) | |
| >>> sequence_label = torch.tensor([1]) | |
| >>> outputs = model( | |
| ... input_ids=input_ids, | |
| ... bbox=bbox, | |
| ... attention_mask=attention_mask, | |
| ... token_type_ids=token_type_ids, | |
| ... labels=sequence_label, | |
| ... ) | |
| >>> loss = outputs.loss | |
| >>> logits = outputs.logits | |
| ``` | |
| </ExampleCodeBlock> | |
| </div></div> | |
| ## LayoutLMForTokenClassification[[transformers.LayoutLMForTokenClassification]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMForTokenClassification</name><anchor>transformers.LayoutLMForTokenClassification</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L837</source><parameters>[{"name": "config", "val": ""}]</parameters><paramsdesc>- **config** ([LayoutLMForTokenClassification](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForTokenClassification)) -- | |
| Model configuration class with all the parameters of the model. Initializing with a config file does not | |
| load the weights associated with the model, only the configuration. Check out the | |
| [from_pretrained()](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights.</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| LayoutLM Model with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for | |
| sequence labeling (information extraction) tasks such as the [FUNSD](https://guillaumejaume.github.io/FUNSD/) | |
| dataset and the [SROIE](https://rrc.cvc.uab.es/?ch=13) dataset. | |
| This model inherits from [PreTrainedModel](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the | |
| library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads | |
| etc.) | |
| This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. | |
| Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage | |
| and behavior. | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>forward</name><anchor>transformers.LayoutLMForTokenClassification.forward</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L851</source><parameters>[{"name": "input_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "bbox", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "attention_mask", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "token_type_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "position_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "inputs_embeds", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "labels", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "output_attentions", "val": ": typing.Optional[bool] = None"}, {"name": "output_hidden_states", "val": ": typing.Optional[bool] = None"}, {"name": "return_dict", "val": ": typing.Optional[bool] = None"}]</parameters><paramsdesc>- **input_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. | |
| Indices can be obtained using [AutoTokenizer](/docs/transformers/pr_33962/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and | |
| [PreTrainedTokenizer.__call__()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. | |
| [What are input IDs?](../glossary#input-ids) | |
| - **bbox** (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*) -- | |
| Bounding boxes of each input sequence tokens. Selected in the range `[0, | |
| config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) | |
| format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, | |
| y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. | |
| - **attention_mask** (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: | |
| - 1 for tokens that are **not masked**, | |
| - 0 for tokens that are **masked**. | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **token_type_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: | |
| - 0 corresponds to a *sentence A* token, | |
| - 1 corresponds to a *sentence B* token. | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **position_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. | |
| [What are position IDs?](../glossary#position-ids) | |
| - **inputs_embeds** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) -- | |
| Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This | |
| is useful if you want more control over how to convert `input_ids` indices into associated vectors than the | |
| model's internal embedding lookup matrix. | |
| - **labels** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. | |
| - **output_attentions** (`bool`, *optional*) -- | |
| Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned | |
| tensors for more detail. | |
| - **output_hidden_states** (`bool`, *optional*) -- | |
| Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for | |
| more detail. | |
| - **return_dict** (`bool`, *optional*) -- | |
| Whether or not to return a [ModelOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.utils.ModelOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[transformers.modeling_outputs.TokenClassifierOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.TokenClassifierOutput) or `tuple(torch.FloatTensor)`</rettype><retdesc>A [transformers.modeling_outputs.TokenClassifierOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.TokenClassifierOutput) or a tuple of | |
| `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various | |
| elements depending on the configuration ([LayoutLMConfig](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMConfig)) and inputs. | |
| - **loss** (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) -- Classification loss. | |
| - **logits** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.num_labels)`) -- Classification scores (before SoftMax). | |
| - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + | |
| one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. | |
| Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. | |
| - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, | |
| sequence_length)`. | |
| Attentions weights after the attention softmax, used to compute the weighted average in the self-attention | |
| heads.</retdesc></docstring> | |
| The [LayoutLMForTokenClassification](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForTokenClassification) forward method, overrides the `__call__` special method. | |
| <Tip> | |
| Although the recipe for forward pass needs to be defined within this function, one should call the `Module` | |
| instance afterwards instead of this since the former takes care of running the pre and post processing steps while | |
| the latter silently ignores them. | |
| </Tip> | |
| <ExampleCodeBlock anchor="transformers.LayoutLMForTokenClassification.forward.example"> | |
| Examples: | |
| ```python | |
| >>> from transformers import AutoTokenizer, LayoutLMForTokenClassification | |
| >>> import torch | |
| >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> model = LayoutLMForTokenClassification.from_pretrained("microsoft/layoutlm-base-uncased") | |
| >>> words = ["Hello", "world"] | |
| >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] | |
| >>> token_boxes = [] | |
| >>> for word, box in zip(words, normalized_word_boxes): | |
| ... word_tokens = tokenizer.tokenize(word) | |
| ... token_boxes.extend([box] * len(word_tokens)) | |
| >>> # add bounding boxes of cls + sep tokens | |
| >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] | |
| >>> encoding = tokenizer(" ".join(words), return_tensors="pt") | |
| >>> input_ids = encoding["input_ids"] | |
| >>> attention_mask = encoding["attention_mask"] | |
| >>> token_type_ids = encoding["token_type_ids"] | |
| >>> bbox = torch.tensor([token_boxes]) | |
| >>> token_labels = torch.tensor([1, 1, 0, 0]).unsqueeze(0) # batch size of 1 | |
| >>> outputs = model( | |
| ... input_ids=input_ids, | |
| ... bbox=bbox, | |
| ... attention_mask=attention_mask, | |
| ... token_type_ids=token_type_ids, | |
| ... labels=token_labels, | |
| ... ) | |
| >>> loss = outputs.loss | |
| >>> logits = outputs.logits | |
| ``` | |
| </ExampleCodeBlock> | |
| </div></div> | |
| ## LayoutLMForQuestionAnswering[[transformers.LayoutLMForQuestionAnswering]] | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>class transformers.LayoutLMForQuestionAnswering</name><anchor>transformers.LayoutLMForQuestionAnswering</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L945</source><parameters>[{"name": "config", "val": ""}, {"name": "has_visual_segment_embedding", "val": " = True"}]</parameters><paramsdesc>- **config** ([LayoutLMForQuestionAnswering](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForQuestionAnswering)) -- | |
| Model configuration class with all the parameters of the model. Initializing with a config file does not | |
| load the weights associated with the model, only the configuration. Check out the | |
| [from_pretrained()](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method to load the model weights. | |
| - **has_visual_segment_embedding** (`bool`, *optional*, defaults to `True`) -- | |
| Whether or not to add visual segment embeddings.</paramsdesc><paramgroups>0</paramgroups></docstring> | |
| The Layoutlm transformer with a span classification head on top for extractive question-answering tasks like | |
| SQuAD (a linear layer on top of the hidden-states output to compute `span start logits` and `span end logits`). | |
| This model inherits from [PreTrainedModel](/docs/transformers/pr_33962/en/main_classes/model#transformers.PreTrainedModel). Check the superclass documentation for the generic methods the | |
| library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads | |
| etc.) | |
| This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. | |
| Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage | |
| and behavior. | |
| <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"> | |
| <docstring><name>forward</name><anchor>transformers.LayoutLMForQuestionAnswering.forward</anchor><source>https://github.com/huggingface/transformers/blob/vr_33962/src/transformers/models/layoutlm/modeling_layoutlm.py#L963</source><parameters>[{"name": "input_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "bbox", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "attention_mask", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "token_type_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "position_ids", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "inputs_embeds", "val": ": typing.Optional[torch.FloatTensor] = None"}, {"name": "start_positions", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "end_positions", "val": ": typing.Optional[torch.LongTensor] = None"}, {"name": "output_attentions", "val": ": typing.Optional[bool] = None"}, {"name": "output_hidden_states", "val": ": typing.Optional[bool] = None"}, {"name": "return_dict", "val": ": typing.Optional[bool] = None"}]</parameters><paramsdesc>- **input_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of input sequence tokens in the vocabulary. Padding will be ignored by default. | |
| Indices can be obtained using [AutoTokenizer](/docs/transformers/pr_33962/en/model_doc/auto#transformers.AutoTokenizer). See [PreTrainedTokenizer.encode()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.encode) and | |
| [PreTrainedTokenizer.__call__()](/docs/transformers/pr_33962/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.__call__) for details. | |
| [What are input IDs?](../glossary#input-ids) | |
| - **bbox** (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*) -- | |
| Bounding boxes of each input sequence tokens. Selected in the range `[0, | |
| config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) | |
| format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, | |
| y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. | |
| - **attention_mask** (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: | |
| - 1 for tokens that are **not masked**, | |
| - 0 for tokens that are **masked**. | |
| [What are attention masks?](../glossary#attention-mask) | |
| - **token_type_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: | |
| - 0 corresponds to a *sentence A* token, | |
| - 1 corresponds to a *sentence B* token. | |
| [What are token type IDs?](../glossary#token-type-ids) | |
| - **position_ids** (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*) -- | |
| Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. | |
| [What are position IDs?](../glossary#position-ids) | |
| - **inputs_embeds** (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) -- | |
| Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This | |
| is useful if you want more control over how to convert `input_ids` indices into associated vectors than the | |
| model's internal embedding lookup matrix. | |
| - **start_positions** (`torch.LongTensor` of shape `(batch_size,)`, *optional*) -- | |
| Labels for position (index) of the start of the labelled span for computing the token classification loss. | |
| Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence | |
| are not taken into account for computing the loss. | |
| - **end_positions** (`torch.LongTensor` of shape `(batch_size,)`, *optional*) -- | |
| Labels for position (index) of the end of the labelled span for computing the token classification loss. | |
| Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence | |
| are not taken into account for computing the loss. | |
| - **output_attentions** (`bool`, *optional*) -- | |
| Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned | |
| tensors for more detail. | |
| - **output_hidden_states** (`bool`, *optional*) -- | |
| Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for | |
| more detail. | |
| - **return_dict** (`bool`, *optional*) -- | |
| Whether or not to return a [ModelOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.utils.ModelOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[transformers.modeling_outputs.QuestionAnsweringModelOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.QuestionAnsweringModelOutput) or `tuple(torch.FloatTensor)`</rettype><retdesc>A [transformers.modeling_outputs.QuestionAnsweringModelOutput](/docs/transformers/pr_33962/en/main_classes/output#transformers.modeling_outputs.QuestionAnsweringModelOutput) or a tuple of | |
| `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various | |
| elements depending on the configuration ([LayoutLMConfig](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMConfig)) and inputs. | |
| - **loss** (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) -- Total span extraction loss is the sum of a Cross-Entropy for the start and end positions. | |
| - **start_logits** (`torch.FloatTensor` of shape `(batch_size, sequence_length)`) -- Span-start scores (before SoftMax). | |
| - **end_logits** (`torch.FloatTensor` of shape `(batch_size, sequence_length)`) -- Span-end scores (before SoftMax). | |
| - **hidden_states** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) -- Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + | |
| one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. | |
| Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. | |
| - **attentions** (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) -- Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, | |
| sequence_length)`. | |
| Attentions weights after the attention softmax, used to compute the weighted average in the self-attention | |
| heads.</retdesc></docstring> | |
| The [LayoutLMForQuestionAnswering](/docs/transformers/pr_33962/en/model_doc/layoutlm#transformers.LayoutLMForQuestionAnswering) forward method, overrides the `__call__` special method. | |
| <Tip> | |
| Although the recipe for forward pass needs to be defined within this function, one should call the `Module` | |
| instance afterwards instead of this since the former takes care of running the pre and post processing steps while | |
| the latter silently ignores them. | |
| </Tip> | |
| Example: | |
| In the example below, we prepare a question + context pair for the LayoutLM model. It will give us a prediction | |
| of what it thinks the answer is (the span of the answer within the texts parsed from the image). | |
| <ExampleCodeBlock anchor="transformers.LayoutLMForQuestionAnswering.forward.example"> | |
| ```python | |
| >>> from transformers import AutoTokenizer, LayoutLMForQuestionAnswering | |
| >>> from datasets import load_dataset | |
| >>> import torch | |
| >>> tokenizer = AutoTokenizer.from_pretrained("impira/layoutlm-document-qa", add_prefix_space=True) | |
| >>> model = LayoutLMForQuestionAnswering.from_pretrained("impira/layoutlm-document-qa", revision="1e3ebac") | |
| >>> dataset = load_dataset("nielsr/funsd", split="train") | |
| >>> example = dataset[0] | |
| >>> question = "what's his name?" | |
| >>> words = example["words"] | |
| >>> boxes = example["bboxes"] | |
| >>> encoding = tokenizer( | |
| ... question.split(), words, is_split_into_words=True, return_token_type_ids=True, return_tensors="pt" | |
| ... ) | |
| >>> bbox = [] | |
| >>> for i, s, w in zip(encoding.input_ids[0], encoding.sequence_ids(0), encoding.word_ids(0)): | |
| ... if s == 1: | |
| ... bbox.append(boxes[w]) | |
| ... elif i == tokenizer.sep_token_id: | |
| ... bbox.append([1000] * 4) | |
| ... else: | |
| ... bbox.append([0] * 4) | |
| >>> encoding["bbox"] = torch.tensor([bbox]) | |
| >>> word_ids = encoding.word_ids(0) | |
| >>> outputs = model(**encoding) | |
| >>> loss = outputs.loss | |
| >>> start_scores = outputs.start_logits | |
| >>> end_scores = outputs.end_logits | |
| >>> start, end = word_ids[start_scores.argmax(-1)], word_ids[end_scores.argmax(-1)] | |
| >>> print(" ".join(words[start : end + 1])) | |
| M. Hamann P. Harper, P. Martinez | |
| ``` | |
| </ExampleCodeBlock> | |
| </div></div> | |
| <EditOnGithub source="https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/layoutlm.md" /> |
Xet Storage Details
- Size:
- 90 kB
- Xet hash:
- 1856d1359ccb9bffb58d6de49ec0c243f94168d2b6a79cc154300dfe108abef8
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.