distilbert-base-uncased-finetuned-squad

This model is a fine-tuned version of distilbert-base-uncased on the Squad dataset. It achieves the following results on the evaluation set:

• Loss: 2.3978

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	65	3.4665
No log	2.0	130	2.8392
No log	3.0	195	2.4838
No log	4.0	260	2.3667
No log	5.0	325	2.3424
No log	6.0	390	2.2847
No log	7.0	455	2.3560
2.1356	8.0	520	2.3636
2.1356	9.0	585	2.3655
2.1356	10.0	650	2.3978

Usage

question = "What is rama's friend name?"
context = "My name is rama. My friend name is raju."
from transformers import pipeline

question_answerer = pipeline("question-answering", model="likhith231/distilbert-base-uncased-finetuned-squad")
question_answerer(question=question, context=context)

Framework versions

• Transformers 4.37.0
• Pytorch 2.1.2
• Datasets 2.1.0
• Tokenizers 0.15.1