nebulette
/

booru-character-aware-tokenizer

Model card Files Files and versions

nebulette commited on 19 days ago

Commit

c1b625f

·

verified ·

1 Parent(s): f20996e

Update config.

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -8,6 +8,6 @@
   "pad_token": "<pad>",
   "padding_side": "right",
   "sep_token": "<sep>",
-  "tokenizer_class": "BertTokenize",
   "unk_token": "<unk>"
 }

   "pad_token": "<pad>",
   "padding_side": "right",
   "sep_token": "<sep>",
+  "tokenizer_class": "BertTokenizer",
   "unk_token": "<unk>"
 }