deberta-v3-xsmall-indonesia-squadv2

This model is a fine-tuned version of microsoft/deberta-v3-xsmall on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4182

Model description

Deberta-V3-Xsmall from Microsft with model parameter: Backbone Parameters 22M, 384 Hidden Size, 12 Layers Based Deberta-V3

Intended uses & limitations

More information needed

Training and evaluation data

Training and evaluation data using Indonesia SQuAD V2, source from https://github.com/Wikidepia/SQuAD-id

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.6078	1.0	13505	1.5331
1.4216	2.0	27010	1.4344
1.2017	3.0	40515	1.4182

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1+cu113
• Datasets 2.7.1
• Tokenizers 0.13.2

Evaluation Results

{'exact': 55.34646711872568,
 'f1': 67.22757187614371,
 'total': 24923,
 'HasAns_exact': 55.34646711872568,
 'HasAns_f1': 67.22757187614371,
 'HasAns_total': 24923,
 'best_exact': 55.34646711872568,
 'best_exact_thresh': 0.0,
 'best_f1': 67.22757187614371,
 'best_f1_thresh': 0.0}

Simple Usage

from transformers import pipeline
qa_pipeline = pipeline(
    "question-answering",
    model="asaduas/deberta-v3-xsmall-indonesia-squadv2",
    tokenizer="asaduas/deberta-v3-xsmall-indonesia-squadv2"
)
qa_pipeline(
    {
        'context': "Pada tahun 1512 juga Afonso de Albuquerque mengirim Antonio Albreu dan Franscisco Serrao untuk memimpin armadanya mencari jalan ke tempat asal rempah-rempah di Maluku. Sepanjang perjalanan, mereka singgah di Madura, Bali, dan Lombok. Dengan menggunakan nakhoda-nakhoda Jawa, armada itu tiba di Kepulauan Banda, terus menuju Aibku Utara sampai tiba di Ternate.",
        'question': "Siapa yang dikirim oleh Afonso de Albuquerque Pada tahun 1512?"
    }
)

Output

[
  {'score': 0.8919295072555542,
  'start': 51,
  'end': 88,
  'answer': ' Antonio Albreu dan Franscisco Serrao'}
]