course/pr_1213/en/chapter2/9.md

End-of-chapter quiz[[end-of-chapter-quiz]]

1. What is the order of the language modeling pipeline?

2. How many dimensions does the tensor output by the base Transformer model have, and what are they?

3. Which of the following is an example of subword tokenization?

4. What is a model head?

5. What is an AutoModel?

AutoTrain product?" }, { text: "An object that returns the correct architecture based on the checkpoint", explain: "Exactly: the AutoModel only needs to know the checkpoint from which to initialize to return the correct architecture.", correct: true }, { text: "A model that automatically detects the language used for its inputs to load the correct weights", explain: "While some checkpoints and models are capable of handling multiple languages, there are no built-in tools for automatic checkpoint selection according to language. You should head over to the Model Hub to find the best checkpoint for your task!" } ]} />

6. What are the techniques to be aware of when batching sequences of different lengths together?

7. What is the point of applying a SoftMax function to the logits output by a sequence classification model?

8. What method is most of the tokenizer API centered around?

encode, as it can encode text into IDs and IDs into predictions", explain: "Wrong! While the encode method does exist on tokenizers, it does not exist on models." }, { text: "Calling the tokenizer object directly.", explain: "Exactly! The call method of the tokenizer is a very powerful method which can handle pretty much anything. It is also the method used to retrieve predictions from a model.", correct: true }, { text: "pad", explain: "Wrong! Padding is very useful, but it's just one part of the tokenizer API." }, { text: "tokenize", explain: "The tokenize method is arguably one of the most useful methods, but it isn't the core of the tokenizer API." } ]} />

9. What does the result variable contain in this code sample?

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
result = tokenizer.tokenize("Hello!")

call or convert_tokens_to_ids method is for!" }, { text: "A string containing all of the tokens", explain: "This would be suboptimal, as the goal is to split the string into multiple tokens." } ]} />

10. Is there something wrong with the following code?

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
model = AutoModel.from_pretrained("gpt2")

encoded = tokenizer("Hey!", return_tensors="pt")
result = model(**encoded)