Upload tokenizer

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "[BOS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[EOS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer.json

@@ -0,0 +1,1200 @@
+{
+  "version": "1.0",
+  "truncation": {
+    "direction": "Left",
+    "max_length": 512,
+    "strategy": "LongestFirst",
+    "stride": 0
+  },
+  "padding": null,
+  "added_tokens": [
+    {
+      "id": 0,
+      "content": "[BOS]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 1,
+      "content": "[EOS]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 2,
+      "content": "[PAD]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 3,
+      "content": "[UNK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 4,
+      "content": "[MASK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 5,
+      "content": "[CLS]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 6,
+      "content": "[SEP]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  ],
+  "normalizer": null,
+  "pre_tokenizer": {
+    "type": "Split",
+    "pattern": {
+      "Regex": "(\\[[^\\]]+]|Br?|Cl?|N|O|S|P|F|I|b|c|n|o|s|p|\\(|\\)|\\.|=|-|\\+|\\\\|\\/|:|~|@|\\?|>>?|\\*|\\$|\\%[0-9]{2}|[0-9])"
+    },
+    "behavior": "Isolated",
+    "invert": false
+  },
+  "post_processor": {
+    "type": "TemplateProcessing",
+    "single": [
+      {
+        "SpecialToken": {
+          "id": "[BOS]",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "A",
+          "type_id": 0
+        }
+      },
+      {
+        "SpecialToken": {
+          "id": "[EOS]",
+          "type_id": 0
+        }
+      }
+    ],
+    "pair": [
+      {
+        "Sequence": {
+          "id": "A",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "B",
+          "type_id": 1
+        }
+      }
+    ],
+    "special_tokens": {
+      "[BOS]": {
+        "id": "[BOS]",
+        "ids": [
+          0
+        ],
+        "tokens": [
+          "[BOS]"
+        ]
+      },
+      "[EOS]": {
+        "id": "[EOS]",
+        "ids": [
+          1
+        ],
+        "tokens": [
+          "[EOS]"
+        ]
+      }
+    }
+  },
+  "decoder": null,
+  "model": {
+    "type": "WordLevel",
+    "vocab": {
+      "[BOS]": 0,
+      "[EOS]": 1,
+      "[PAD]": 2,
+      "[UNK]": 3,
+      "[MASK]": 4,
+      "[CLS]": 5,
+      "[SEP]": 6,
+      "C": 7,
+      "=": 8,
+      "(": 9,
+      ")": 10,
+      "O": 11,
+      "N": 12,
+      "1": 13,
+      "2": 14,
+      "3": 15,
+      "4": 16,
+      "S": 17,
+      "Cl": 18,
+      "F": 19,
+      "5": 20,
+      "[C@@H]": 21,
+      "[C@H]": 22,
+      "/": 23,
+      "[O-]": 24,
+      "[N+]": 25,
+      "Br": 26,
+      "6": 27,
+      "#": 28,
+      ".": 29,
+      "\\": 30,
+      "7": 31,
+      "[NH+]": 32,
+      "I": 33,
+      "[C@@]": 34,
+      "[C@]": 35,
+      "P": 36,
+      "8": 37,
+      "[Si]": 38,
+      "9": 39,
+      "[NH2+]": 40,
+      "[Cl-]": 41,
+      "[N-]": 42,
+      "[Na+]": 43,
+      "[C-]": 44,
+      "B": 45,
+      "[I-]": 46,
+      "[Br-]": 47,
+      "[NH3+]": 48,
+      "[O+]": 49,
+      "[S-]": 50,
+      "[NH-]": 51,
+      "[O-2]": 52,
+      "[Se]": 53,
+      "[P+]": 54,
+      "[OH+]": 55,
+      "[As]": 56,
+      "[Sn]": 57,
+      "[2H]": 58,
+      "[H+]": 59,
+      "[K+]": 60,
+      "[OH2+]": 61,
+      "[NH2-]": 62,
+      "[Hg]": 63,
+      "[CH-]": 64,
+      "[OH-]": 65,
+      "[S+]": 66,
+      "[Pt+2]": 67,
+      "[Cu+2]": 68,
+      "[CH]": 69,
+      "[NH4+]": 70,
+      "[SH+]": 71,
+      "[W]": 72,
+      "[Zn+2]": 73,
+      "[Ca+2]": 74,
+      "[Cu]": 75,
+      "[B-]": 76,
+      "[Co]": 77,
+      "[Mg+2]": 78,
+      "[Mo]": 79,
+      "[Fe]": 80,
+      "[CH2-]": 81,
+      "[Ni+2]": 82,
+      "[SiH]": 83,
+      "[Fe+2]": 84,
+      "[Pt]": 85,
+      "[Ni]": 86,
+      "[18F]": 87,
+      "[Co+2]": 88,
+      "[F-]": 89,
+      "[C]": 90,
+      "[Cr]": 91,
+      "[Li+]": 92,
+      "[Fe+3]": 93,
+      "[Co+3]": 94,
+      "[Al+3]": 95,
+      "[Te]": 96,
+      "[O]": 97,
+      "[Cr+3]": 98,
+      "[Ti]": 99,
+      "[Ge]": 100,
+      "[11CH3]": 101,
+      "[V]": 102,
+      "[Zn]": 103,
+      "[SH2+]": 104,
+      "[Mn+2]": 105,
+      "[Ba+2]": 106,
+      "[Pd]": 107,
+      "[Mn]": 108,
+      "[Rh]": 109,
+      "[Pt+4]": 110,
+      "[Ti+4]": 111,
+      "[Sb]": 112,
+      "[Pb]": 113,
+      "[Pd+2]": 114,
+      "[CH3-]": 115,
+      "[99Tc]": 116,
+      "[Ag+]": 117,
+      "[Ru]": 118,
+      "[Cu+]": 119,
+      "[H-]": 120,
+      "[S-2]": 121,
+      "[SiH2]": 122,
+      "[125I]": 123,
+      "[11C]": 124,
+      "[P-]": 125,
+      "[N@+]": 126,
+      "[Ac]": 127,
+      "[Hg+2]": 128,
+      "[N@@+]": 129,
+      "[Cd+2]": 130,
+      "[Au]": 131,
+      "[Sr+2]": 132,
+      "[Pb+2]": 133,
+      "[Bi+3]": 134,
+      "[Re]": 135,
+      "[I+]": 136,
+      "[3H]": 137,
+      "[13C]": 138,
+      "[Zn+]": 139,
+      "[Hg+]": 140,
+      "[Sn+4]": 141,
+      "[SiH3]": 142,
+      "[Rh+2]": 143,
+      "[Gd+3]": 144,
+      "[Ru+3]": 145,
+      "[Zr]": 146,
+      "[Ag]": 147,
+      "[Au+]": 148,
+      "[Be+2]": 149,
+      "[SH-]": 150,
+      "[Sb+3]": 151,
+      "[SH3+]": 152,
+      "[U]": 153,
+      "[No]": 154,
+      "[Cs+]": 155,
+      "[Al]": 156,
+      "[La+3]": 157,
+      "[Cd]": 158,
+      "[Zr+4]": 159,
+      "[Rh+3]": 160,
+      "[11CH2]": 161,
+      "[Ce+3]": 162,
+      "[Nb]": 163,
+      "[Ru+2]": 164,
+      "[CH+]": 165,
+      "[As+]": 166,
+      "[Se-2]": 167,
+      "[Nd+3]": 168,
+      "[SnH]": 169,
+      "[Hf]": 170,
+      "[Os]": 171,
+      "[Ir]": 172,
+      "[123I]": 173,
+      "[131I]": 174,
+      "[Eu]": 175,
+      "[SeH]": 176,
+      "[Ga+3]": 177,
+      "[AsH]": 178,
+      "[BH3-]": 179,
+      "[Sn+2]": 180,
+      "[Y+3]": 181,
+      "[1H]": 182,
+      "[C+]": 183,
+      "[Rb+]": 184,
+      "[AlH3]": 185,
+      "[CH2]": 186,
+      "[Si+4]": 187,
+      "[Ta]": 188,
+      "[Mn+3]": 189,
+      "[Mo+2]": 190,
+      "[OH3+]": 191,
+      "[Tl+]": 192,
+      "[Pr+3]": 193,
+      "[In+3]": 194,
+      "[Co+]": 195,
+      "[Au+3]": 196,
+      "[Bi]": 197,
+      "[Ir+3]": 198,
+      "[Y]": 199,
+      "[Eu+3]": 200,
+      "[Pt+]": 201,
+      "[Sm+3]": 202,
+      "[Zr+2]": 203,
+      "[13CH]": 204,
+      "[La]": 205,
+      "[14C]": 206,
+      "[Ce]": 207,
+      "[Yb]": 208,
+      "[Tl]": 209,
+      "[BH-]": 210,
+      "[BH2-]": 211,
+      "[Pr]": 212,
+      "[Tb]": 213,
+      "[U+2]": 214,
+      "[Sm]": 215,
+      "[Cr+2]": 216,
+      "[Er]": 217,
+      "[Nb+5]": 218,
+      "[Dy+3]": 219,
+      "[Ga]": 220,
+      "[Gd]": 221,
+      "[Lu]": 222,
+      "[Ho]": 223,
+      "[11CH]": 224,
+      "[Tb+3]": 225,
+      "[Th]": 226,
+      "[Dy]": 227,
+      "[Lu+3]": 228,
+      "[GeH2]": 229,
+      "[N]": 230,
+      "[Nd]": 231,
+      "[P-3]": 232,
+      "[Ru+4]": 233,
+      "[Er+3]": 234,
+      "[Ho+3]": 235,
+      "[Ru-]": 236,
+      "[Sc+3]": 237,
+      "[SnH2]": 238,
+      "[Tm+3]": 239,
+      "[Tm]": 240,
+      "[V+2]": 241,
+      "[Yb+3]": 242,
+      "[ClH+]": 243,
+      "[Ti+2]": 244,
+      "[PH+]": 245,
+      "[S@]": 246,
+      "[Sc]": 247,
+      "[Se+]": 248,
+      "[Ta+5]": 249,
+      "[99Tc+3]": 250,
+      "[P@]": 251,
+      "[Th+4]": 252,
+      "[Tl+3]": 253,
+      "[Cr+6]": 254,
+      "[GeH]": 255,
+      "[PbH]": 256,
+      "[Pd+]": 257,
+      "%10": 258,
+      "[99Tc+4]": 259,
+      "[AsH2]": 260,
+      "[Os+2]": 261,
+      "[ClH2+]": 262,
+      "[Ni+3]": 263,
+      "[BH4-]": 264,
+      "[Ru+]": 265,
+      "[13NH2]": 266,
+      "[AlH]": 267,
+      "[In]": 268,
+      "[Np]": 269,
+      "[Ru+5]": 270,
+      "[S]": 271,
+      "[Si-]": 272,
+      "[14CH2]": 273,
+      "[32P]": 274,
+      "[Cr+5]": 275,
+      "[Hf+4]": 276,
+      "[Pu]": 277,
+      "[Rh+]": 278,
+      "[GeH3]": 279,
+      "[Pt-2]": 280,
+      "[Sb+]": 281,
+      "[SbH]": 282,
+      "[B+]": 283,
+      "[P@@]": 284,
+      "[S@@]": 285,
+      "[PbH2]": 286,
+      "[Se-]": 287,
+      "[As-]": 288,
+      "[Ca]": 289,
+      "[HH]": 290,
+      "[Li]": 291,
+      "[Mo+4]": 292,
+      "[Ru+8]": 293,
+      "[Sn+]": 294,
+      "[14CH3]": 295,
+      "[17O]": 296,
+      "[Ce+4]": 297,
+      "[Cr+4]": 298,
+      "[Mg]": 299,
+      "[Na]": 300,
+      "[Sb-]": 301,
+      "[Si-2]": 302,
+      "[Ti+]": 303,
+      "[TlH]": 304,
+      "[14CH]": 305,
+      "[Fe-]": 306,
+      "[Os+4]": 307,
+      "[PH2+]": 308,
+      "[Pd-2]": 309,
+      "[Ru+6]": 310,
+      "[Sn-]": 311,
+      "[W+2]": 312,
+      "[Zn-2]": 313,
+      "[Zn-]": 314,
+      "[111In+3]": 315,
+      "[Al-]": 316,
+      "[AlH2]": 317,
+      "[As+5]": 318,
+      "[Au-]": 319,
+      "[Fe+]": 320,
+      "[Lr]": 321,
+      "[Mn+]": 322,
+      "[Mo+3]": 323,
+      "[Tc]": 324,
+      "[Ti+3]": 325,
+      "%11": 326,
+      "[99Tc+5]": 327,
+      "[As+3]": 328,
+      "[B]": 329,
+      "[Be]": 330,
+      "[CH2+]": 331,
+      "[Cm]": 332,
+      "[Es]": 333,
+      "[Eu+2]": 334,
+      "[Hg-2]": 335,
+      "[IH2+]": 336,
+      "[K]": 337,
+      "[NH]": 338,
+      "[PH2]": 339,
+      "[SbH2]": 340,
+      "[SiH4]": 341,
+      "[SnH3]": 342,
+      "[Xe]": 343,
+      "[51Cr]": 344,
+      "[68Ga+3]": 345,
+      "[75Se]": 346,
+      "[76Br]": 347,
+      "[BiH2]": 348,
+      "[Hg-]": 349,
+      "[Rh-]": 350,
+      "[Ru-2]": 351,
+      "[Te+]": 352,
+      "[TeH]": 353,
+      "%12": 354,
+      "[203Hg+2]": 355,
+      "[239Pu]": 356,
+      "[249Cf]": 357,
+      "[35S]": 358,
+      "[37Cl]": 359,
+      "[74Se]": 360,
+      "[Am]": 361,
+      "[Ba]": 362,
+      "[BiH]": 363,
+      "[Bk]": 364,
+      "[Cf]": 365,
+      "[Cs]": 366,
+      "[GaH]": 367,
+      "[He]": 368,
+      "[InH]": 369,
+      "[Pm]": 370,
+      "[Sn+3]": 371,
+      "[Zr+3]": 372,
+      "[14C@H]": 373,
+      "[15N]": 374,
+      "[203Hg]": 375,
+      "[211At]": 376,
+      "[237Np]": 377,
+      "[241Am]": 378,
+      "[Al-2]": 379,
+      "[AlH4-]": 380,
+      "[AsH3]": 381,
+      "[Cu-2]": 382,
+      "[Fe+6]": 383,
+      "[FeH]": 384,
+      "[GaH3]": 385,
+      "[H]": 386,
+      "[Ni+]": 387,
+      "[Ni-]": 388,
+      "[Os+6]": 389,
+      "[Os+7]": 390,
+      "[Os+8]": 391,
+      "[Os+]": 392,
+      "[PH-]": 393,
+      "[PH2-]": 394,
+      "[PbH2+2]": 395,
+      "[PbH4]": 396,
+      "[Re-]": 397,
+      "[Si@]": 398,
+      "[Sr]": 399,
+      "[Tc+7]": 400,
+      "[Te+4]": 401,
+      "[TlH2]": 402,
+      "[V+4]": 403,
+      "[W+]": 404,
+      "%13": 405,
+      "[106Ru]": 406,
+      "[11C@@H]": 407,
+      "[11C@H]": 408,
+      "[11CH3-]": 409,
+      "[122I]": 410,
+      "[123IH]": 411,
+      "[125IH]": 412,
+      "[131I-]": 413,
+      "[137Cs+]": 414,
+      "[13CH2]": 415,
+      "[153Sm+3]": 416,
+      "[159Dy]": 417,
+      "[15NH2]": 418,
+      "[15NH]": 419,
+      "[165Dy]": 420,
+      "[186Re]": 421,
+      "[231Pa]": 422,
+      "[238Pu]": 423,
+      "[239Np]": 424,
+      "[249Bk]": 425,
+      "[253Cf]": 426,
+      "[253Es]": 427,
+      "[33P]": 428,
+      "[45Ca+2]": 429,
+      "[55Fe+3]": 430,
+      "[59Fe+3]": 431,
+      "[59Fe]": 432,
+      "[60Co]": 433,
+      "[65Zn]": 434,
+      "[67Cu]": 435,
+      "[77Br]": 436,
+      "[85Sr+2]": 437,
+      "[89Sr+2]": 438,
+      "[90Sr+2]": 439,
+      "[91Y]": 440,
+      "[AlH2+]": 441,
+      "[BiH3]": 442,
+      "[Cr+]": 443,
+      "[Fe-3]": 444,
+      "[FeH2]": 445,
+      "[GaH2]": 446,
+      "[Ge-2]": 447,
+      "[Hf+2]": 448,
+      "[IH+]": 449,
+      "[InH2]": 450,
+      "[Ir+]": 451,
+      "[Ir-2]": 452,
+      "[Kr]": 453,
+      "[Li-]": 454,
+      "[PH3+]": 455,
+      "[P]": 456,
+      "[Pa]": 457,
+      "[PtH]": 458,
+      "[Rb]": 459,
+      "[Rh-2]": 460,
+      "[Ru-3]": 461,
+      "[RuH]": 462,
+      "[SbH3]": 463,
+      "[Se+4]": 464,
+      "[SeH2+]": 465,
+      "[SeH2]": 466,
+      "[Si@@]": 467,
+      "[Ta-2]": 468,
+      "[Tc+3]": 469,
+      "[Tc+4]": 470,
+      "[Th+2]": 471,
+      "[Zr-]": 472,
+      "[ZrH]": 473,
+      "[100Pd]": 474,
+      "[100Rh]": 475,
+      "[101Mo]": 476,
+      "[101Pd]": 477,
+      "[101Rh]": 478,
+      "[101Tc]": 479,
+      "[102Ag]": 480,
+      "[102Rh]": 481,
+      "[102Ru]": 482,
+      "[103Ag]": 483,
+      "[103Ru+2]": 484,
+      "[103Ru]": 485,
+      "[104Ag]": 486,
+      "[104Cd]": 487,
+      "[104Tc]": 488,
+      "[105Ag]": 489,
+      "[105Pd]": 490,
+      "[105Rh]": 491,
+      "[105Ru]": 492,
+      "[106Ag]": 493,
+      "[106Rh]": 494,
+      "[107Ag]": 495,
+      "[107Cd]": 496,
+      "[107Pd]": 497,
+      "[107Rh]": 498,
+      "[108Ag]": 499,
+      "[108Pd]": 500,
+      "[109Ag]": 501,
+      "[109Cd]": 502,
+      "[109Pd+2]": 503,
+      "[109Pd]": 504,
+      "[10BH3]": 505,
+      "[110Ag+]": 506,
+      "[110Ag]": 507,
+      "[110Ru]": 508,
+      "[111Ag]": 509,
+      "[111InH3]": 510,
+      "[112Ag]": 511,
+      "[113Cd]": 512,
+      "[115Ag]": 513,
+      "[115Cd]": 514,
+      "[117Cd]": 515,
+      "[117SnH2]": 516,
+      "[11C-]": 517,
+      "[11CH4]": 518,
+      "[120IH]": 519,
+      "[121IH]": 520,
+      "[121SnH2]": 521,
+      "[123I-]": 522,
+      "[123Xe]": 523,
+      "[124I-]": 524,
+      "[124IH]": 525,
+      "[124I]": 526,
+      "[125I-]": 527,
+      "[126IH]": 528,
+      "[127Te]": 529,
+      "[127Xe]": 530,
+      "[128IH]": 531,
+      "[129I]": 532,
+      "[130IH]": 533,
+      "[131IH]": 534,
+      "[131La]": 535,
+      "[132IH]": 536,
+      "[132La]": 537,
+      "[133Ba+2]": 538,
+      "[133IH]": 539,
+      "[133Xe]": 540,
+      "[134Ce]": 541,
+      "[134IH]": 542,
+      "[135Ce]": 543,
+      "[135IH]": 544,
+      "[135La]": 545,
+      "[135Xe]": 546,
+      "[136Nd]": 547,
+      "[136Pr]": 548,
+      "[137Ce]": 549,
+      "[137La]": 550,
+      "[137Pr]": 551,
+      "[138La]": 552,
+      "[138Nd]": 553,
+      "[138Pr]": 554,
+      "[139Ce]": 555,
+      "[139Nd]": 556,
+      "[139Pr]": 557,
+      "[13CH4]": 558,
+      "[13NH2-]": 559,
+      "[13NH3]": 560,
+      "[140La]": 561,
+      "[141Ce+3]": 562,
+      "[141Ce]": 563,
+      "[141La]": 564,
+      "[141Nd]": 565,
+      "[141Pm]": 566,
+      "[141Sm]": 567,
+      "[142La]": 568,
+      "[142Pr]": 569,
+      "[142Sm]": 570,
+      "[143Ce]": 571,
+      "[143La]": 572,
+      "[143Pm]": 573,
+      "[143Pr]": 574,
+      "[144Ce]": 575,
+      "[144Pm]": 576,
+      "[144Pr]": 577,
+      "[145Eu]": 578,
+      "[145Gd]": 579,
+      "[145Pm]": 580,
+      "[145Pr]": 581,
+      "[145Sm]": 582,
+      "[146Eu]": 583,
+      "[146Gd]": 584,
+      "[146Nd]": 585,
+      "[146Pm]": 586,
+      "[147Eu]": 587,
+      "[147Gd]": 588,
+      "[147Nd]": 589,
+      "[147Pm]": 590,
+      "[147Pr]": 591,
+      "[147Tb]": 592,
+      "[148Eu]": 593,
+      "[148Pm]": 594,
+      "[149Eu]": 595,
+      "[149Gd]": 596,
+      "[149Nd]": 597,
+      "[149Pm]": 598,
+      "[149Tb]": 599,
+      "[14CH4]": 600,
+      "[150Eu]": 601,
+      "[150Pm]": 602,
+      "[150Sm]": 603,
+      "[150Tb]": 604,
+      "[151Gd]": 605,
+      "[151Nd]": 606,
+      "[151Pm]": 607,
+      "[151Sm]": 608,
+      "[151Tb]": 609,
+      "[152Eu]": 610,
+      "[153Gd]": 611,
+      "[153Sm]": 612,
+      "[153Tb]": 613,
+      "[154Eu]": 614,
+      "[154Sm]": 615,
+      "[154Tb]": 616,
+      "[155Dy]": 617,
+      "[155Eu]": 618,
+      "[155Ho]": 619,
+      "[155Sm]": 620,
+      "[155Tb]": 621,
+      "[156Eu]": 622,
+      "[156Sm]": 623,
+      "[156Tb]": 624,
+      "[157Eu]": 625,
+      "[157Ho]": 626,
+      "[157Tb]": 627,
+      "[158Eu]": 628,
+      "[158Gd]": 629,
+      "[158Tb]": 630,
+      "[159Gd]": 631,
+      "[159Ho]": 632,
+      "[15CH4]": 633,
+      "[15NH3]": 634,
+      "[15OH]": 635,
+      "[160Gd]": 636,
+      "[160Tb]": 637,
+      "[161Er]": 638,
+      "[161Gd]": 639,
+      "[161Ho]": 640,
+      "[161Tb]": 641,
+      "[162Dy]": 642,
+      "[162Ho]": 643,
+      "[162Tm]": 644,
+      "[162Yb]": 645,
+      "[164Ho]": 646,
+      "[165Er]": 647,
+      "[166Dy]": 648,
+      "[166Er]": 649,
+      "[166Ho]": 650,
+      "[166Tm]": 651,
+      "[166Yb]": 652,
+      "[167Ho]": 653,
+      "[167Tm]": 654,
+      "[167Yb]": 655,
+      "[168Er]": 656,
+      "[169Er]": 657,
+      "[169Lu]": 658,
+      "[169Yb]": 659,
+      "[170Er]": 660,
+      "[170Hf]": 661,
+      "[170Lu]": 662,
+      "[170Tm]": 663,
+      "[171Er]": 664,
+      "[171Lu]": 665,
+      "[171Tm]": 666,
+      "[172Er]": 667,
+      "[172Hf]": 668,
+      "[172Lu]": 669,
+      "[172Ta]": 670,
+      "[172Tm]": 671,
+      "[173Hf]": 672,
+      "[173Lu]": 673,
+      "[173Ta]": 674,
+      "[173Tm]": 675,
+      "[174Lu]": 676,
+      "[174Ta]": 677,
+      "[174Yb]": 678,
+      "[175Hf]": 679,
+      "[175Ta]": 680,
+      "[175Tm]": 681,
+      "[175Yb]": 682,
+      "[176Lu]": 683,
+      "[176Ta]": 684,
+      "[176W]": 685,
+      "[176Yb]": 686,
+      "[177Hf]": 687,
+      "[177Lu]": 688,
+      "[177Re]": 689,
+      "[177Ta]": 690,
+      "[177W]": 691,
+      "[177Yb]": 692,
+      "[178Hf]": 693,
+      "[178Lu]": 694,
+      "[178Re]": 695,
+      "[178Ta]": 696,
+      "[178W]": 697,
+      "[178Yb]": 698,
+      "[179Hf]": 699,
+      "[179Lu]": 700,
+      "[179Ta]": 701,
+      "[179W]": 702,
+      "[17NH3]": 703,
+      "[180Hf]": 704,
+      "[180Os]": 705,
+      "[180Ta]": 706,
+      "[181Hf]": 707,
+      "[181Os]": 708,
+      "[181Re]": 709,
+      "[181W]": 710,
+      "[182Hf]": 711,
+      "[182Ir]": 712,
+      "[182Os]": 713,
+      "[182Re]": 714,
+      "[182Ta]": 715,
+      "[183Hf]": 716,
+      "[184Hf]": 717,
+      "[184Ir]": 718,
+      "[184Re]": 719,
+      "[184Ta]": 720,
+      "[185Ir]": 721,
+      "[185Os]": 722,
+      "[185Ta]": 723,
+      "[185W]": 724,
+      "[186Ir]": 725,
+      "[186Pt]": 726,
+      "[186Ta]": 727,
+      "[187Ir]": 728,
+      "[187Re]": 729,
+      "[187W]": 730,
+      "[188Ir]": 731,
+      "[188Pt]": 732,
+      "[188Re]": 733,
+      "[188W]": 734,
+      "[189Ir]": 735,
+      "[189Os]": 736,
+      "[189Pt]": 737,
+      "[189Re]": 738,
+      "[18FH]": 739,
+      "[18OH2]": 740,
+      "[18O]": 741,
+      "[190Ir]": 742,
+      "[191Os]": 743,
+      "[191Pt]": 744,
+      "[192Ir]": 745,
+      "[193Au]": 746,
+      "[193Hg]": 747,
+      "[193Os]": 748,
+      "[193Pt]": 749,
+      "[194Au]": 750,
+      "[194Hg]": 751,
+      "[194Ir]": 752,
+      "[194Os]": 753,
+      "[195Au]": 754,
+      "[195Hg]": 755,
+      "[195Ir]": 756,
+      "[195Pt]": 757,
+      "[197Hg]": 758,
+      "[197Pt]": 759,
+      "[198Au]": 760,
+      "[199Au]": 761,
+      "[199Hg]": 762,
+      "[199Pt]": 763,
+      "[1H-]": 764,
+      "[200Au]": 765,
+      "[200Pt]": 766,
+      "[201Au]": 767,
+      "[201Tl+]": 768,
+      "[210BiH3]": 769,
+      "[210PoH2]": 770,
+      "[212PbH2]": 771,
+      "[218Rn]": 772,
+      "[219Rn]": 773,
+      "[220Rn]": 774,
+      "[222Rn]": 775,
+      "[224Ac]": 776,
+      "[224Rn]": 777,
+      "[225Ac]": 778,
+      "[226Ac]": 779,
+      "[226Rn]": 780,
+      "[226Th]": 781,
+      "[227Ac]": 782,
+      "[227Pa]": 783,
+      "[227Th]": 784,
+      "[228Ac]": 785,
+      "[228Pa]": 786,
+      "[228Rn]": 787,
+      "[228Th]": 788,
+      "[229Th]": 789,
+      "[230Pa]": 790,
+      "[230Pu]": 791,
+      "[230Th]": 792,
+      "[230U]": 793,
+      "[231Th]": 794,
+      "[231U]": 795,
+      "[232Np]": 796,
+      "[232Pa]": 797,
+      "[232U]": 798,
+      "[233Np]": 799,
+      "[233Pa]": 800,
+      "[233U]": 801,
+      "[234Np]": 802,
+      "[234Pa]": 803,
+      "[234Pu]": 804,
+      "[234Th]": 805,
+      "[234U]": 806,
+      "[235Np]": 807,
+      "[235Pu]": 808,
+      "[235U]": 809,
+      "[236Np]": 810,
+      "[236Pu]": 811,
+      "[236U]": 812,
+      "[237Am]": 813,
+      "[237Pu]": 814,
+      "[237U]": 815,
+      "[238Am]": 816,
+      "[238Cm]": 817,
+      "[238Np]": 818,
+      "[238Th]": 819,
+      "[239Am]": 820,
+      "[239U]": 821,
+      "[240Am]": 822,
+      "[240Cm]": 823,
+      "[240Np]": 824,
+      "[240Pu]": 825,
+      "[240U]": 826,
+      "[241Cm]": 827,
+      "[241Pu]": 828,
+      "[242Am]": 829,
+      "[242Cm]": 830,
+      "[242Pu]": 831,
+      "[243Am]": 832,
+      "[243Cm]": 833,
+      "[243Pu]": 834,
+      "[244Am]": 835,
+      "[244Cf]": 836,
+      "[244Cm]": 837,
+      "[244Pu]": 838,
+      "[245Am]": 839,
+      "[245Bk]": 840,
+      "[245Cm]": 841,
+      "[245Pu]": 842,
+      "[246Am]": 843,
+      "[246Bk]": 844,
+      "[246Cf]": 845,
+      "[246Cm]": 846,
+      "[246Pu]": 847,
+      "[247Bk]": 848,
+      "[247Cm]": 849,
+      "[248Am]": 850,
+      "[248Cf]": 851,
+      "[248Cm]": 852,
+      "[249Cm]": 853,
+      "[250Bk]": 854,
+      "[250Cf]": 855,
+      "[250Cm]": 856,
+      "[250Es]": 857,
+      "[251Cf]": 858,
+      "[251Es]": 859,
+      "[252Cf]": 860,
+      "[252Fm]": 861,
+      "[253Fm]": 862,
+      "[254Cf]": 863,
+      "[254Es]": 864,
+      "[254Fm]": 865,
+      "[255Fm]": 866,
+      "[257Fm]": 867,
+      "[257Md]": 868,
+      "[258Md]": 869,
+      "[2HH]": 870,
+      "[32PH3]": 871,
+      "[32SH2]": 872,
+      "[33PH3]": 873,
+      "[34SH2]": 874,
+      "[35S-2]": 875,
+      "[35SH2]": 876,
+      "[36ClH]": 877,
+      "[36SH2]": 878,
+      "[38ClH]": 879,
+      "[39ClH]": 880,
+      "[3HH]": 881,
+      "[40PH3]": 882,
+      "[41Ar]": 883,
+      "[43Sc]": 884,
+      "[44Sc]": 885,
+      "[44Ti]": 886,
+      "[45Ti]": 887,
+      "[46Sc]": 888,
+      "[47Sc]": 889,
+      "[47V]": 890,
+      "[48Cr]": 891,
+      "[48Sc]": 892,
+      "[48V]": 893,
+      "[49Cr]": 894,
+      "[49Sc]": 895,
+      "[49V]": 896,
+      "[51Cr+3]": 897,
+      "[51Cr+6]": 898,
+      "[51Mn]": 899,
+      "[52Mn]": 900,
+      "[53Mn]": 901,
+      "[54Mn]": 902,
+      "[55Co]": 903,
+      "[55Fe]": 904,
+      "[56Co]": 905,
+      "[56Mn]": 906,
+      "[56Ni]": 907,
+      "[57Co+2]": 908,
+      "[57Co]": 909,
+      "[57Fe]": 910,
+      "[57Ni]": 911,
+      "[58Co]": 912,
+      "[58Ni]": 913,
+      "[59Fe+2]": 914,
+      "[59Ni]": 915,
+      "[60Co+2]": 916,
+      "[60Co+3]": 917,
+      "[60Cu]": 918,
+      "[60Fe]": 919,
+      "[61Co]": 920,
+      "[61Cu]": 921,
+      "[61Ni]": 922,
+      "[62Co]": 923,
+      "[62Zn]": 924,
+      "[63Ni]": 925,
+      "[63Zn]": 926,
+      "[64Cu]": 927,
+      "[65Ni]": 928,
+      "[65Zn+2]": 929,
+      "[66Ni]": 930,
+      "[67Ga+3]": 931,
+      "[67GaH3]": 932,
+      "[68GaH3]": 933,
+      "[69Zn]": 934,
+      "[71Zn]": 935,
+      "[72Br-]": 936,
+      "[72BrH]": 937,
+      "[72Zn]": 938,
+      "[74BrH]": 939,
+      "[75BrH]": 940,
+      "[75Br]": 941,
+      "[76BrH]": 942,
+      "[77BrH]": 943,
+      "[79Kr]": 944,
+      "[80BrH]": 945,
+      "[80Br]": 946,
+      "[81Kr]": 947,
+      "[82BrH]": 948,
+      "[82Rb+]": 949,
+      "[83BrH]": 950,
+      "[84BrH]": 951,
+      "[85Kr]": 952,
+      "[86Y]": 953,
+      "[86Zr]": 954,
+      "[87Y]": 955,
+      "[88Nb]": 956,
+      "[88Y]": 957,
+      "[88Zr]": 958,
+      "[89Kr]": 959,
+      "[89Nb]": 960,
+      "[89Zr]": 961,
+      "[90Mo]": 962,
+      "[90Nb]": 963,
+      "[90Y+3]": 964,
+      "[90Y]": 965,
+      "[92Y]": 966,
+      "[93Mo]": 967,
+      "[93Tc]": 968,
+      "[93Y]": 969,
+      "[93Zr]": 970,
+      "[94Nb]": 971,
+      "[94Ru]": 972,
+      "[94Tc]": 973,
+      "[94Y]": 974,
+      "[95Nb]": 975,
+      "[95Y]": 976,
+      "[95Zr]": 977,
+      "[96Nb]": 978,
+      "[96Tc]": 979,
+      "[97Nb]": 980,
+      "[97Ru]": 981,
+      "[97Tc]": 982,
+      "[97Zr]": 983,
+      "[98Nb]": 984,
+      "[98Tc]": 985,
+      "[99Mo]": 986,
+      "[99Rh]": 987,
+      "[99Ru]": 988,
+      "[99Tc+]": 989,
+      "[Ag-]": 990,
+      "[AgH]": 991,
+      "[Al-3]": 992,
+      "[AlH+2]": 993,
+      "[Ar]": 994,
+      "[AtH]": 995,
+      "[At]": 996,
+      "[B@-]": 997,
+      "[BH2]": 998,
+      "[Bi+2]": 999,
+      "[Br]": 1000,
+      "[CH3+]": 1001,
+      "[CH3]": 1002,
+      "[Cd-2]": 1003,
+      "[Cl+3]": 1004,
+      "[CoH2]": 1005,
+      "[Cu-]": 1006,
+      "[CuH]": 1007,
+      "[Fm]": 1008,
+      "[Ga-]": 1009,
+      "[Gd+2]": 1010,
+      "[GeH4]": 1011,
+      "[I]": 1012,
+      "[InH3]": 1013,
+      "[Ir+2]": 1014,
+      "[IrH+2]": 1015,
+      "[Md]": 1016,
+      "[Mn-2]": 1017,
+      "[MoH2]": 1018,
+      "[NH2]": 1019,
+      "[Nb+2]": 1020,
+      "[Ne]": 1021,
+      "[Ni-2]": 1022,
+      "[NiH]": 1023,
+      "[OH]": 1024,
+      "[Os+5]": 1025,
+      "[OsH]": 1026,
+      "[PH4+]": 1027,
+      "[PoH2]": 1028,
+      "[Re-2]": 1029,
+      "[ReH]": 1030,
+      "[Rh-3]": 1031,
+      "[RhH+2]": 1032,
+      "[RhH]": 1033,
+      "[Rn]": 1034,
+      "[Se+6]": 1035,
+      "[Se@@H]": 1036,
+      "[Si+]": 1037,
+      "[Si@@H]": 1038,
+      "[SiH-]": 1039,
+      "[SiH3-]": 1040,
+      "[SnH4]": 1041,
+      "[SrH2]": 1042,
+      "[Ta+2]": 1043,
+      "[TaH2]": 1044,
+      "[Tc+6]": 1045,
+      "[Te-]": 1046,
+      "[TeH2]": 1047,
+      "[TlH3]": 1048,
+      "[WH2]": 1049,
+      "[Yb+2]": 1050,
+      "[ZrH2]": 1051
+    },
+    "unk_token": "[UNK]"
+  }
+}

tokenizer_config.json

@@ -0,0 +1,72 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[BOS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[EOS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[BOS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "eos_token": "[EOS]",
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "truncation_side": "left",
+  "unk_token": "[UNK]"
+}