kgrabko commited on
Commit
069181f
·
verified ·
1 Parent(s): 57729cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -6,7 +6,11 @@ license_link: LICENSE
6
 
7
  # 💎 JiRack Tokenizer
8
  - Compatible with Llama 3 • Optimized for Code
9
- - A Llama 3-based tokenizer enhanced with FIM markers (<fim_prefix>, <fim_suffix>, <fim_middle>).
 
 
 
 
10
  - Added Robotics & Embodiment tags (
11
  "<|action_start|>",
12
  "<|action_end|>",
 
6
 
7
  # 💎 JiRack Tokenizer
8
  - Compatible with Llama 3 • Optimized for Code
9
+ - A Llama 3-based tokenizer enhanced with FIM markers (
10
+ "<fim_prefix>",
11
+ "<fim_middle>",
12
+ "<fim_suffix>",
13
+ )
14
  - Added Robotics & Embodiment tags (
15
  "<|action_start|>",
16
  "<|action_end|>",