Add fine-tuned BERT models on different QA datasets

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+mixed_best/pytorch_model.bin filter=lfs diff=lfs merge=lfs -text
+squad-v2_best/pytorch_model.bin filter=lfs diff=lfs merge=lfs -text
+te-cde_best/pytorch_model.bin filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,138 @@
----
-license: mit
----
+# Fine-Tuned BERT Models for Thermoelectric Materials Question Answering
+
+## Introduction
+
+This repository contains three BERT models fine-tuned for question-answering (QA) tasks related to thermoelectric materials. The models are trained on different datasets to evaluate their performance on specialised QA tasks in the field of materials science.
+
+We present a method for auto-generating a large question-answering dataset about thermoelectric materials for language model applications. The method was used to generate a dataset with sentence-wide contexts from a database of thermoelectric material records. The dataset was contrasted with SQuAD-v2, as well as the mixed combination of the two datasets. Hyperparameter optimisation was employed to fine-tune BERT models on each dataset, and the three best-performing models were then compared on a manually annotated test set of thermoelectric material paragraph contexts with questions spanning material names, five different properties, and temperatures during recording. The best BERT model fine-tuned on the mixed dataset outperforms the other two models when evaluated on the test dataset, indicating that mixing datasets with different semantic and syntactic scopes might be a beneficial approach to improving performance on specialised question-answering tasks.
+
+## Models Included
+
+1. **squad-v2_best**
+
+    Description: Fine-tuned on the SQuAD-v2 dataset, which is a widely used benchmark for QA tasks. \
+    Dataset: SQuAD-v2 \
+    Location: squad-v2_best/
+
+2. **te-cde_best**
+
+    Description: Fine-tuned on a thermoelectric materials-specific dataset generated using our auto-generation method. \
+    Dataset: Thermoelectric Materials QA Dataset (TE-CDE) \
+    Location: te-cde_best/
+
+3. **mixed_best**
+
+    Description: Fine-tuned on a mixed dataset combining SQuAD-v2 and the thermoelectric materials dataset to enhance performance on specialised QA tasks. \
+    Dataset: Combination of SQuAD-v2 and TE-CDE \
+    Location: mixed_best/
+
+## Dataset Details
+
+**SQuAD-v2**
+
+    A reading comprehension dataset consisting of questions posed by crowdworkers on a set of Wikipedia articles.
+    Some questions are unanswerable, adding complexity to the QA task.
+
+**Thermoelectric Materials QA Dataset (TE-CDE)**
+
+    Auto-generated dataset containing QA pairs about thermoelectric materials.
+    Contexts are sentence-wide excerpts from a database of thermoelectric material records.
+    Questions cover:
+        Material names
+        Five different properties
+        Temperatures during recording
+
+**Mixed Dataset**
+
+    A combination of SQuAD-v2 and TE-CDE datasets.
+    Aims to leverage the strengths of both general-purpose and domain-specific data.
+
+## Training Details
+
+    Base Model: BERT Base Uncased
+    Hyperparameter Optimisation: Employed to find the best-performing models for each dataset.
+    Training Parameters:
+        Epochs: Adjusted per dataset based on validation loss.
+        Batch Size: Optimized during training.
+        Learning Rate: Tuned using grid search.
+
+## Evaluation Metrics
+
+    Evaluation Dataset: Manually annotated test set of thermoelectric material paragraph contexts.
+    Metrics Used:
+        Exact Match (EM): Measures the percentage of predictions that match any one of the ground truth answers exactly.
+        F1 Score: Harmonic mean of precision and recall, considering overlap between the prediction and ground truth answers.
+
+### Performance Comparison
+Model	Exact Match (EM)	F1 Score
+squad-v2_best	57.60%	61.82%
+te-cde_best	65.39%	69.78%
+mixed_best	67.92%	72.29%
+
+## Usage Instructions
+
+### Installing Dependencies
+
+```bash
+pip install transformers
+```
+
+### Loading a Model
+
+Replace `model_name` with one of the following:
+
+    "odysie/bert-finetuned-qa-datasets/squad-v2_best"
+    "odysie/bert-finetuned-qa-datasets/te-cde_best"
+    "odysie/bert-finetuned-qa-datasets/mixed_best"
+
+```python
+from transformers import BertForQuestionAnswering, BertTokenizer
+
+model_name = "odysie/bert-finetuned-qa-datasets/mixed_best"
+
+tokenizer = BertTokenizer.from_pretrained(model_name)
+model = BertForQuestionAnswering.from_pretrained(model_name)
+
+# Example question and context
+question = "What is the chemical formula for water?"
+context = "Water is a molecule composed of two hydrogen atoms and one oxygen atom, with the chemical formula H2O."
+
+# Tokenize input
+inputs = tokenizer.encode_plus(question, context, return_tensors="pt")
+
+# Get model predictions
+outputs = model(**inputs)
+start_scores = outputs.start_logits
+end_scores = outputs.end_logits
+
+# Get the most likely beginning and end of answer with the argmax of the score
+start_index = start_scores.argmax()
+end_index = end_scores.argmax()
+
+# Convert tokens to answer
+tokens = inputs["input_ids"][0][start_index : end_index + 1]
+answer = tokenizer.decode(tokens)
+
+print(f"Answer: {answer}")
+```
+
+## License
+
+This project is licensed under Apache 2.0
+
+
+## Citation
+
+If you use these models in your research or application, please cite our work:
+
+bibtex
+
+(PENDING)
+
+@article{
+    ...
+}
+
+## Acknowledgments
+
+We thank the contributors of the SQuAD-v2 dataset and the developers of the Hugging Face Transformers library for providing valuable resources that made this work possible.

mixed_best/README.md

@@ -0,0 +1,126 @@
+### Model Name
+
+## Overview
+
+    This model is a fine-tuned version of BERT Base Uncased on a mixing of the SQuAD-v2 dataset and the TE-CDE dataset. It is optimised for pecialiased question-answering tasks in the field of thermoelectric materials, across five seminal quantities (the thermoelectric figure of merit, thermal conductivity, Seebeck coefficient, electrical conductivity, and power factor), while still performing well on general questions, and can be used to extract answers from given contexts.
+
+## Model Details
+
+    Model Type: BERT Base Uncased
+    Fine-Tuned On: Mixed Dataset
+    Language: English
+    License: Apache 2.0
+    Tags: bert, question-answering, transformers, fine-tuned, thermoelectric materials
+
+## Usage
+### Installation
+
+Make sure you have the Transformers library installed:
+
+```bash
+pip install transformers
+```
+
+### Loading the Model
+
+```python
+from transformers import BertForQuestionAnswering, BertTokenizer
+
+model_name = "odysie/bert-finetuned-qa-datasets/mixed_best"
+
+tokenizer = BertTokenizer.from_pretrained(model_name)
+model = BertForQuestionAnswering.from_pretrained(model_name)
+
+Replace Model_Name with: squad-v2_best, te-cde_best, or mixed_best
+```
+
+### Example Usage
+
+```python
+from transformers import BertForQuestionAnswering, BertTokenizer
+import torch
+
+model_name = "odysie/bert-finetuned-qa-datasets/squad-v2_best"
+
+tokenizer = BertTokenizer.from_pretrained(model_name)
+model = BertForQuestionAnswering.from_pretrained(model_name)
+
+# Sample question and context
+question = "What is the value of the Seebeck coefficient?"
+context = "Cu2Sn0.93Ag0.07Se3 demonstrated a Seebeck coefficient of 1.2 VK-1 at 300 K."
+
+# Tokenize input
+inputs = tokenizer.encode_plus(question, context, return_tensors="pt")
+input_ids = inputs["input_ids"].tolist()[0]
+
+# Get model output
+outputs = model(**inputs)
+answer_start_scores = outputs.start_logits
+answer_end_scores = outputs.end_logits
+
+# Find the tokens with the highest `start` and `end` scores
+answer_start = torch.argmax(answer_start_scores)
+answer_end = torch.argmax(answer_end_scores) + 1
+
+# Convert tokens to answer
+answer = tokenizer.convert_tokens_to_string(
+    tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end])
+)
+
+print(f"Question: {question}")
+print(f"Answer: {answer}")
+```
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 6.257686103023713e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 0
+- distributed_type: multi-GPU
+- num_devices: 16
+- total_train_batch_size: 128
+- total_eval_batch_size: 128
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.08443391405864548
+- num_epochs: 5.0
+- mixed_precision_training: Native AMP
+
+### Framework versions
+
+- Transformers 4.41.0
+- Pytorch 2.3.0+cu121
+- Datasets 2.19.1
+- Tokenizers 0.19.1
+
+## Dataset
+### Description
+
+    SQuAD-v2 combines the 100,000 questions in SQuAD-v1.1 with over 50,000 unanswerable questions. This dataset tests the ability of a model not only to answer questions when possible but also to abstain from answering when the question is unanswerable based on the context.
+
+## Link
+
+    SQuAD-v2: https://rajpurkar.github.io/SQuAD-explorer/
+
+## License
+
+This project is licensed under Apache 2.0
+
+
+## Citation
+
+If you use these models in your research or application, please cite our work:
+
+bibtex
+
+(PENDING)
+
+@article{
+    ...
+}
+
+## Acknowledgments
+
+We thank the contributors of the SQuAD-v2 dataset and the developers of the Hugging Face Transformers library for providing valuable resources that made this work possible.

mixed_best/config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "bert-base-uncased",
+  "architectures": [
+    "BertForQuestionAnswering"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

mixed_best/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce6682bc301b640dfff04fb949fe418529aa4c698fccf8a4e7756055029b1d8e
+size 435638182

mixed_best/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

mixed_best/tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

squad-v2_best/README.md

@@ -0,0 +1,126 @@
+### Model Name
+
+## Overview
+
+    This model is a fine-tuned version of BERT Base Uncased on the SQuAD-v2 dataset. It is optimised for question-answering tasks and can be used to extract answers from given contexts.
+
+## Model Details
+
+    Model Type: BERT Base Uncased
+    Fine-Tuned On: SQuAD-v2
+    Language: English
+    License: Apache 2.0
+    Tags: bert, question-answering, transformers, fine-tuned
+
+## Usage
+### Installation
+
+Make sure you have the Transformers library installed:
+
+```bash
+pip install transformers
+```
+
+### Loading the Model
+
+```python
+from transformers import BertForQuestionAnswering, BertTokenizer
+
+model_name = "odysie/bert-finetuned-qa-datasets/squad-v2_best"
+
+tokenizer = BertTokenizer.from_pretrained(model_name)
+model = BertForQuestionAnswering.from_pretrained(model_name)
+
+Replace Model_Name with: squad-v2_best, te-cde_best, or mixed_best
+```
+
+### Example Usage
+
+```python
+from transformers import BertForQuestionAnswering, BertTokenizer
+import torch
+
+model_name = "odysie/bert-finetuned-qa-datasets/squad-v2_best"
+
+tokenizer = BertTokenizer.from_pretrained(model_name)
+model = BertForQuestionAnswering.from_pretrained(model_name)
+
+# Sample question and context
+question = "What is the value of the Seebeck coefficient?"
+context = "Cu2Sn0.93Ag0.07Se3 demonstrated a Seebeck coefficient of 1.2 VK-1 at 300 K."
+
+# Tokenize input
+inputs = tokenizer.encode_plus(question, context, return_tensors="pt")
+input_ids = inputs["input_ids"].tolist()[0]
+
+# Get model output
+outputs = model(**inputs)
+answer_start_scores = outputs.start_logits
+answer_end_scores = outputs.end_logits
+
+# Find the tokens with the highest `start` and `end` scores
+answer_start = torch.argmax(answer_start_scores)
+answer_end = torch.argmax(answer_end_scores) + 1
+
+# Convert tokens to answer
+answer = tokenizer.convert_tokens_to_string(
+    tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end])
+)
+
+print(f"Question: {question}")
+print(f"Answer: {answer}")
+```
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1.5218292681575764e-05
+- train_batch_size: 1
+- eval_batch_size: 8
+- seed: 0
+- distributed_type: multi-GPU
+- num_devices: 16
+- total_train_batch_size: 16
+- total_eval_batch_size: 128
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.0961958191102116
+- num_epochs: 3.0
+- mixed_precision_training: Native AMP
+
+### Framework versions
+
+- Transformers 4.41.0
+- Pytorch 2.3.0+cu121
+- Datasets 2.19.1
+- Tokenizers 0.19.1
+
+## Dataset
+### Description
+
+    SQuAD-v2 combines the 100,000 questions in SQuAD-v1.1 with over 50,000 unanswerable questions. This dataset tests the ability of a model not only to answer questions when possible but also to abstain from answering when the question is unanswerable based on the context.
+
+## Link
+
+    SQuAD-v2: https://rajpurkar.github.io/SQuAD-explorer/
+
+## License
+
+This project is licensed under Apache 2.0
+
+
+## Citation
+
+If you use these models in your research or application, please cite our work:
+
+bibtex
+
+(PENDING)
+
+@article{
+    ...
+}
+
+## Acknowledgments
+
+We thank the contributors of the SQuAD-v2 dataset and the developers of the Hugging Face Transformers library for providing valuable resources that made this work possible.

squad-v2_best/config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "bert-base-uncased",
+  "architectures": [
+    "BertForQuestionAnswering"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

squad-v2_best/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:419696115b900e686f06c2168accf8e35904d2ed3c62e774dc857c1a9c6c5c81
+size 435638182

squad-v2_best/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

squad-v2_best/tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

te-cde_best/README.md

@@ -0,0 +1,124 @@
+### Model Name
+
+## Overview
+
+    This model is a fine-tuned version of BERT Base Uncased on the automatically generated TE-CDE dataset. It is optimised for specialiased question-answering tasks in the field of thermoelectric materials, across five seminal quantities (the thermoelectric figure of merit, thermal conductivity, Seebeck coefficient, electrical conductivity, and power factor) and can be used to extract answers from given contexts.
+
+## Model Details
+
+    Model Type: BERT Base Uncased
+    Fine-Tuned On: TE-CDE
+    Language: English
+    License: Apache 2.0
+    Tags: bert, question-answering, transformers, fine-tuned, thermoelectric materials
+
+## Usage
+### Installation
+
+Make sure you have the Transformers library installed:
+
+```bash
+pip install transformers
+```
+
+### Loading the Model
+
+```python
+from transformers import BertForQuestionAnswering, BertTokenizer
+
+model_name = "odysie/bert-finetuned-qa-datasets/te-cde_best"
+
+tokenizer = BertTokenizer.from_pretrained(model_name)
+model = BertForQuestionAnswering.from_pretrained(model_name)
+
+Replace Model_Name with: squad-v2_best, te-cde_best, or mixed_best
+```
+
+### Example Usage
+
+```python
+from transformers import BertForQuestionAnswering, BertTokenizer
+import torch
+
+model_name = "odysie/bert-finetuned-qa-datasets/te-cde_best"
+
+tokenizer = BertTokenizer.from_pretrained(model_name)
+model = BertForQuestionAnswering.from_pretrained(model_name)
+
+# Sample question and context
+question = "What is the value of the Seebeck coefficient?"
+context = "Cu2Sn0.93Ag0.07Se3 demonstrated a Seebeck coefficient of 1.2 VK-1 at 300 K."
+
+# Tokenize input
+inputs = tokenizer.encode_plus(question, context, return_tensors="pt")
+input_ids = inputs["input_ids"].tolist()[0]
+
+# Get model output
+outputs = model(**inputs)
+answer_start_scores = outputs.start_logits
+answer_end_scores = outputs.end_logits
+
+# Find the tokens with the highest `start` and `end` scores
+answer_start = torch.argmax(answer_start_scores)
+answer_end = torch.argmax(answer_end_scores) + 1
+
+# Convert tokens to answer
+answer = tokenizer.convert_tokens_to_string(
+    tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end])
+)
+
+print(f"Question: {question}")
+print(f"Answer: {answer}")
+```
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 7.113287430580505e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 0
+- distributed_type: multi-GPU
+- num_devices: 16
+- total_train_batch_size: 256
+- total_eval_batch_size: 128
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.1466796240672773
+- num_epochs: 13.0
+- mixed_precision_training: Native AMP
+
+### Framework versions
+
+- Transformers 4.41.0
+- Pytorch 2.3.0+cu121
+- Datasets 2.19.1
+- Tokenizers 0.19.1
+
+## Dataset
+### Description
+
+    TE-CDE contains 99,757 questions automatically generated from a thermoelectric materials database, across five different properties: the thermoelectric figure of merit, thermal conductivity, Seebeck coefficient, electrical conductivity, and power factor. 66,508 questions are answerable from the context, and 33,249 are not.
+
+## License
+
+This project is licensed under Apache 2.0
+
+## Citation
+
+If you use these models in your research or application, please cite our work:
+
+bibtex
+
+(PENDING)
+
+@article{
+    ...
+}
+
+## Acknowledgments
+
+We thank the developers of the Hugging Face Transformers library for providing valuable resources that made this work possible.
+
+
+

te-cde_best/config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "bert-base-uncased",
+  "architectures": [
+    "BertForQuestionAnswering"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

te-cde_best/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:725cd8053135df5c3fc66e21b1387dd452862a4e208d062fa40a3621ed6ee48a
+size 435638182

te-cde_best/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

te-cde_best/tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}