Add chat_template to tokenizer config (#6)

- Add chat_template to tokenizer config (b30991b156bff3d4f00aca38854d0fa37a7cb0bb)
- Update README.md (c9a5bd020c49024c2cd3c48e3d6588d2d7336f5b)


Co-authored-by: Pablo Montalvo <Molbap@users.noreply.huggingface.co>

README.md

@@ -49,26 +49,35 @@ You can then start chatting with the model, *e.g.* prompt it with something like
 To use this model within the `transformers` library, install the latest release with `pip install --upgrade transformers` and run, for instance:
 
 ```py
-from transformers import MistralForCausalLM
-from transformers import AutoTokenizer
+from transformers import pipeline
+import torch
 
-tokenizer = AutoTokenizer.from_pretrained('mistralai/mathstral-7B-v0.1')
+checkpoint = "mistralai/mathstral-7B-v0.1"
+pipe = pipeline("text-generation", checkpoint, device_map="auto", torch_dtype=torch.bfloat16)
 
-prompt = "What are the roots of unity?"
-tokenized_prompts = tokenizer(prompt, return_tensors="pt") 
+prompt = [{"role": "user", "content": "What are the roots of unity?"}]
+out = pipe(prompt, max_new_tokens = 512)
 
-model = MistralForCausalLM.from_pretrained('mistralai/mathstral-7B-v0.1')
-generation = model.generate(**tokenized_prompts, max_new_tokens=512)
-print(tokenizer.decode(generation[0]))
->>> """<s>What are the roots of unity?
+print(out[0]['generated_text'][-1])
+>>> "{'role': 'assistant', 'content': ' The roots of unity are the complex numbers that satisfy the equation $z^n = 1$, where $n$ is a positive integer. These roots are evenly spaced around the unit circle in the complex plane, and they have a variety of interesting properties and applications in mathematics and physics.'}"
+```
+
+You can also manually tokenize the input and generate text from the model, rather than using the higher-level pipeline:
+
+```py
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
 
-The roots of unity are the solutions to the equation $z^n = 1$, where $n$ is a positive integer.
-These roots are complex numbers and they form a regular $n$-gon in the complex plane.
+checkpoint = "mistralai/mathstral-7B-v0.1"
+tokenizer = AutoTokenizer.from_pretrained(checkpoint)
+model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.bfloat16)
 
-For example, the roots of unity for $n=1$ are just $1$,
-and for $n=2$ they are $1$ and $-1$. For $n=3$, they are $1$, $\\frac{-1+\\sqrt{3}i}{2}$, and $\\frac{-1-\\sqrt{3}i}{2}$.
+prompt = [{"role": "user", "content": "What are the roots of unity?"}]
+tokenized_prompt = tokenizer.apply_chat_template(prompt, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)
 
-The roots of unity have many interesting properties and they are used in many areas of mathematics, including number theory, algebra, and geometry.</s>"""
+out = model.generate(**tokenized_prompt, max_new_tokens=512)
+tokenizer.decode(out[0])
+>>> '<s>[INST] What are the roots of unity?[/INST] The roots of unity are the complex numbers that satisfy the equation $z^n = 1$, where $n$ is a positive integer. These roots are evenly spaced around the unit circle in the complex plane, and they have a variety of interesting properties and applications in mathematics and physics.</s>'
 ```
 
 ## Evaluation

tokenizer_config.json

@@ -6173,6 +6173,7 @@
     }
   },
   "bos_token": "<s>",
+  "chat_template": "{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] %}\n    {%- set loop_messages = messages[1:] %}\n{%- else %}\n    {%- set loop_messages = messages %}\n{%- endif %}\n\n{{- bos_token }}\n{%- for message in loop_messages %}\n    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}\n        {{- raise_exception('After the optional system message, conversation roles must alternate user/assistant/user/assistant/...') }}\n    {%- endif %}\n    {%- if message['role'] == 'user' %}\n        {%- if loop.last and system_message is defined %}\n            {{- '[INST] ' + system_message + '\\n\\n' + message['content'] + '[/INST]' }}\n        {%- else %}\n            {{- '[INST] ' + message['content'] + '[/INST]' }}\n        {%- endif %}\n    {%- elif message['role'] == 'assistant' %}\n        {{- ' ' + message['content'] + eos_token}}\n    {%- else %}\n        {{- raise_exception('Only user and assistant roles are supported, with the exception of an initial optional system message!') }}\n    {%- endif %}\n{%- endfor %}\n",
   "clean_up_tokenization_spaces": false,
   "eos_token": "</s>",
   "legacy": true,