Gregniuki
/

tokenizer_pl_2546

Model card Files Files and versions

Gregniuki commited on Jan 2, 2025

Commit

aaa5539

·

verified ·

1 Parent(s): 3b8ed8c

Update tokenizer.json

Files changed (1) hide show

tokenizer.json +1 -1

tokenizer.json CHANGED Viewed

@@ -105,7 +105,7 @@
       "\\": 60,
       "]": 61,
       "^": 62,
-      "_": 63,
       "`": 64,
       "a": 65,
       "b": 66,

       "\\": 60,
       "]": 61,
       "^": 62,
+      "—": 63,
       "`": 64,
       "a": 65,
       "b": 66,