lorenzocc
/

NeoBERTugues

masked-language-model

Model card Files Files and versions

lorenzocc commited on Jan 25

Commit

f0707f4

·

verified ·

1 Parent(s): f7e278a

Upload tokenizer_config.json

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -38,7 +38,7 @@
       "normalized": false,
       "rstrip": false,
       "single_word": false,
-      "special": true
     }
   },
   "bos_token": "<bos>",

       "normalized": false,
       "rstrip": false,
       "single_word": false,
+      "special": false
     }
   },
   "bos_token": "<bos>",