matthewleechen/quality_improving_stated_aim_classifier

Quality Improving Stated Aim Classifier

This is an XLM-RoBERTa-large model that is trained to classify whether an explicit stated aim described in a British historical patent is designed to improve quality, reliability or durability.

Hyperparameters:

• lr = 3e-5
• batch size = 44

Validation set results:

{'eval_loss': 0.543449342250824,
 'eval_accuracy': 0.925,
 'eval_precision': 0.9333333333333332,
 'eval_recall': 0.925,
 'eval_f1': 0.9233265720081135,
 'eval_runtime': 0.5431,
 'eval_samples_per_second': 73.652,
 'eval_steps_per_second': 1.841}

Test set results:

{'eval_loss': 0.7934615015983582,
 'eval_accuracy': 0.875,
 'eval_precision': 0.8785266457680251,
 'eval_recall': 0.875,
 'eval_f1': 0.8709090909090909,
 'eval_runtime': 0.6533,
 'eval_samples_per_second': 61.226,
 'eval_steps_per_second': 1.531}