allenai

/

Olmo-3.1-32B-Think

like 84

Follow

Ai2 5.46k

Hi,

Let's say I generated text using Olmo 3 Think. Then it might give me something like "Okay, let's think, ...., \boxed{1}.</think>\n\nThe final answer is \boxed{1}.". Notably, the opening <think>tag will NOT be there because the chat template for tokenizer.apply_chat_template([{"role": "user", "content": "What's 2-1?"}], add_generation_prompt=True), will already include the <think> token in the input prompt to the model.

However, let’s say I wanted to then get the embeddings of the full prompt including the model’s response, or continue the chat, or whatever: when running
tokenizer.apply_chat_template([{"role": "user", "content": "What's 2-1?"}, {"role": "assistant", "content":"Okay, let's think, ...., \boxed{1}.</think>\n\nThe final answer is \boxed{1}."}], add_generation_prompt=False), you would get:
'<|im_start|>system\nYou are OLMo, a helpful function-calling AI assistant built by Ai2. Your date cutoff is November 2024, and your model weights are available at https://huggingface.co/allenai. You do not currently have access to any functions. <functions></functions><|im_end|>\n<|im_start|>user\nhello there<|im_end|>\n<|im_start|>assistant\nOkay, let's think, ...., \boxed{1}.</think>\n\nThe final answer is \boxed{1}.<|endoftext|>’.

(and if you set it to True, then you get same but with the <|im_start|>assistant\n<think> at the end which is also not what I want.)

Notably, the <think> is now missing, because the chat template does not include it for the assistant except if its starting the assistant generation! Is this intentional?

Thanks!