p1atdev
/

dart-tokenizer-v1-encode

Model card Files Files and versions

p1atdev commited on Feb 21, 2024

Commit

9d96279

·

verified ·

1 Parent(s): d6e1f20

Upload tokenizer.json

Files changed (1) hide show

tokenizer.json +1 -1

tokenizer.json CHANGED Viewed

@@ -432,7 +432,7 @@
       "<|very_vague|>": 13,
       "<|vague|>": 14,
       "<|detailed|>": 15,
-      "<|very_detailed>": 16,
       "<|reserved_5|>": 17,
       "<|reserved_6|>": 18,
       "<|reserved_7|>": 19,

       "<|very_vague|>": 13,
       "<|vague|>": 14,
       "<|detailed|>": 15,
+      "<|very_detailed|>": 16,
       "<|reserved_5|>": 17,
       "<|reserved_6|>": 18,
       "<|reserved_7|>": 19,