DSPFirst-Finetuning-1

This model is a fine-tuned version of ahotrod/electra_large_discriminator_squad2_512 on a generated Questions and Answers dataset from the DSPFirst textbook based on the SQuAD 2.0 format.

Dataset

A visualization of the dataset can be found here. The split between train and test is 80% and 20% respectively.

DatasetDict({
    train: Dataset({
        features: ['id', 'title', 'context', 'question', 'answers'],
        num_rows: 4755
    })
    test: Dataset({
        features: ['id', 'title', 'context', 'question', 'answers'],
        num_rows: 1189
    })
})

It achieves the following results on the evaluation set:

• Loss: 0.9236

Model description

More information needed

Intended uses & limitations

Since the dataset is generated from the DSPFirst textbook, its quality is not guaranteed.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• gradient_accumulation_steps: 86
• total_train_batch_size: 516
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Model hyperparameters

• hidden_dropout_prob: 0.5
• attention_probs_dropout_prob = 0.5

Training results

Training Loss	Epoch	Step	Validation Loss
6.0131	0.7	20	0.9549
6.1542	1.42	40	0.9302
6.1472	2.14	60	0.9249
5.9662	2.84	80	0.9248
6.1467	3.56	100	0.9236

Framework versions

• Transformers 4.18.0
• Pytorch 1.10.0+cu111
• Datasets 2.1.0
• Tokenizers 0.12.1