davda54
/

wiki-retrieval-patch-base

Model card Files Files and versions

davda54 commited on Nov 22, 2023

Commit

13f1ca3

·

1 Parent(s): 2523e9c

Upload tokenizer.json

Files changed (1) hide show

tokenizer.json +2 -2

tokenizer.json CHANGED Viewed

@@ -43,8 +43,8 @@
       "id": 4,
       "content": "[MASK]",
       "single_word": false,
-      "lstrip": false,
-      "rstrip": false,
       "normalized": false,
       "special": true
     }

       "id": 4,
       "content": "[MASK]",
       "single_word": false,
+      "lstrip": true,
+      "rstrip": true,
       "normalized": false,
       "special": true
     }