HariomSahu/albert-squad-qa

albert-base-v2 fine-tuned on SQuAD

This model is a fine-tuned version of albert-base-v2 on the SQuAD dataset.

Training Details

Training Hyperparameters

• Model: albert-base-v2
• Dataset: SQuAD
• Optimizer: adamw
• Learning Rate Scheduler: cosine_with_restarts
• Learning Rate: 6e-05
• Batch Size: 28 per device
• Total Batch Size: 224
• Epochs: 6 (with early stopping)
• Weight Decay: 0.005
• Warmup Ratio: 0.08
• Max Gradient Norm: 0.5

Early Stopping

• Patience: 4
• Metric: f1
• Best Epoch: 2

Usage

from transformers import AutoTokenizer, AutoModelForQuestionAnswering

tokenizer = AutoTokenizer.from_pretrained("HariomSahu/albert-squad-qa")
model = AutoModelForQuestionAnswering.from_pretrained("HariomSahu/albert-squad-qa")

# Example usage
question = "What is the capital of France?"
context = "France is a country in Europe. Its capital city is Paris."

inputs = tokenizer(question, context, return_tensors="pt")
outputs = model(**inputs)

# Get answer
start_scores, end_scores = outputs.start_logits, outputs.end_logits
start_index = start_scores.argmax()
end_index = end_scores.argmax()
answer = tokenizer.decode(inputs["input_ids"][0][start_index:end_index+1])
print(f"Answer: {answer}")

Evaluation Results

The model achieved the following results on the evaluation set:

• Exact Match: 80.4541
• F1 Score: 88.7676

Training Configuration Hash

Config Hash: a8d23824

This hash can be used to reproduce the exact training configuration.