Update README.md

README.md

@@ -42,6 +42,27 @@ The summary of the instruction tuning data is as follows:
 
 <center><img src="data_table.jpg" alt="Instruction Data"/></center>
 
+# Instruction Format
+
+We've added some new special tokens to the CrystalCoder tokenizer to support the instruction tuning.
+
+List special tokens used in the instruction tuning:
+
+```
+bos: <s> 
+eos: </s>
+system_start: <|sys_start|>
+system_end: <|sys_end|>
+user_start: <|im_start|>
+user_end: <|im_end|>
+```
+
+The instruction format is as follows:
+
+```
+<s> <|sys_start|> system prompt <|sys_end|> <|im_start|> first user utterance <|im_end|> first model response <|im_start|> next user utterance <|im_end|> next model response </s>
+```
+
 # Reproducing the Results
 
 We will realize the training code and the training data soon. Our training code is based on [Megatron-LM](https://github.com/NVIDIA/Megatron-LM), with some modifications to support our training data format and Maximal Update Parametrization (μP).