|
|
--- |
|
|
language: |
|
|
- en |
|
|
metrics: |
|
|
- accuracy |
|
|
- precision |
|
|
- recall |
|
|
- f1 |
|
|
library_name: transformers |
|
|
pipeline_tag: text-classification |
|
|
--- |
|
|
# bert-large-relation14 |
|
|
|
|
|
Finetuned BERT model for 14-class classification. It was introduced in the paper: [Automatic Slide Generation Using Discourse Relations](https://link.springer.com/chapter/10.1007/978-3-031-36336-8_61) and first released in this repository. This model is uncased: it does not make a difference between english and English. |
|
|
|
|
|
In our proposed method in this [paper](https://link.springer.com/chapter/10.1007/978-3-031-36336-8_61), we only used this model for the classification of discourse relation between the FIRST and SECOND sentence in summarized sentences. The model that is used between the other sentences is [this model](https://huggingface.co/teppei727/bert_woco). If you are curious about our proposed method, it's better to see that model. |
|
|
|
|
|
# Descliption |
|
|
|
|
|
This model can classify the relation between the sentence pair of input. |
|
|
|
|
|
Now we are working on preparing the Model card. Please wait for a few days. |
|
|
|
|
|
|
|
|
The model trained from [bert-large-uncased](https://huggingface.co/bert-large-uncased) on the dataset published in the paper: [Automatic Prediction of Discourse Connectives](https://arxiv.org/abs/1702.00992). |
|
|
|
|
|
The dataset to make this model is based on English Wikipedia data and has 20 labels. However, this model will classify into 14 labels. This is because the 20-class data set was restructured to 14 classes to suit our research objective of "automatic slide generation. This distribution is shown below. |
|
|
|
|
|
|
|
|
|Level 1|Level 2|Level 3|Connectives (20)| |
|
|
|-------------|-----------------|------------------|--------------------| |
|
|
| Temporal | Synchronous | | meanwhile | |
|
|
| Temporal | Asynchronous | Precedence | then, | |
|
|
| Temporal | Asynchronous | Precedence | finally, | |
|
|
| Temporal | Asynchronous | Succession | by then | |
|
|
| Contingency | Cause | Result | therefore | |
|
|
| Comparison | Concession | Arg2-as-denier | however, | |
|
|
| Comparison | Concession | Arg2-as-denier | nevertheless | |
|
|
| Comparison | Contrast | | on the other hand, | |
|
|
| Comparison | Contrast | | by contrast, | |
|
|
| Expansion | Conjunction | | and | |
|
|
| Expansion | Conjunction | | moreover | |
|
|
| Expansion | Conjunction | | indeed | |
|
|
| Expansion | Equivalence | | in other words | |
|
|
| Expansion | Exception | Arg1-as-excpt | otherwise | |
|
|
| Expansion | Instantiation | Arg2-as-instance | for example, | |
|
|
| Expansion | Level-of-detail | Arg1-as-detail | overall, | |
|
|
| Expansion | Level-of-detail | Arg2-as-detail | in particular, | |
|
|
| Expansion | Substitution | Arg2-as-subst | instead | |
|
|
| Expansion | Substitution | Arg2-as-subst | rather | |
|
|
|
|
|
# Training |
|
|
|
|
|
The model was trained using AutoModelForSequenceClassification.from_pretrained |
|
|
|
|
|
``` |
|
|
training_args = TrainingArguments( |
|
|
output_dir = output_dir, |
|
|
save_strategy="epoch", |
|
|
num_train_epochs = 5, |
|
|
per_device_train_batch_size=16, |
|
|
per_device_eval_batch_size=32, |
|
|
warmup_steps=0, |
|
|
weight_decay=0.01, |
|
|
logging_dir="./logs", |
|
|
evaluation_strategy="epoch", |
|
|
learning_rate=2e-5, |
|
|
metric_for_best_model="f1", |
|
|
load_best_model_at_end=True |
|
|
) |
|
|
``` |
|
|
|
|
|
# Evaluation (14 labels and original 20 labels classification) using the dataset test split gives: |
|
|
|
|
|
| Model | Macro F1 | Accuracy | Precision | Recall | |
|
|
|--------------------------|-----------------|-----------------|------------------|---------------| |
|
|
| 14 labels classification | 0.586 | 0.589 | 0.630 | 0.591 | |
|
|
| 20 labels classification | 0.478 | 0.488 | 0.536 | 0.488 | |
