kgrabko commited on
Commit
58e3084
·
verified ·
1 Parent(s): b39c1ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -7,9 +7,9 @@ license_link: LICENSE
7
  # 💎 JiRack Tokenizer
8
  - Compatible with Llama 3 • Optimized for Code
9
  - A Llama 3-based tokenizer enhanced with FIM markers (
10
- <fim_prefix>,
11
- <fim_middle>,
12
- <fim_suffix>
13
  )
14
  - Added Robotics & Embodiment tags (
15
  "<|action_start|>",
 
7
  # 💎 JiRack Tokenizer
8
  - Compatible with Llama 3 • Optimized for Code
9
  - A Llama 3-based tokenizer enhanced with FIM markers (
10
+ <|fim_prefix|>,
11
+ <|fim_middle|>,
12
+ <f|im_suffix|>
13
  )
14
  - Added Robotics & Embodiment tags (
15
  "<|action_start|>",