---
license: mit
base_model: vinai/bertweet-base
tags:
- generated_from_trainer
metrics:
- f1
- precision
- recall
- accuracy
model-index:
- name: bertweet-base_regression_7_seed13_EN
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bertweet-base_regression_7_seed13_EN

This model is a fine-tuned version of [vinai/bertweet-base](https://huggingface.co/vinai/bertweet-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0892
- Mse: 5.5584
- Rmse: 2.3576
- Mae: 1.3807
- R2: 0.2207
- F1: 0.7757
- Precision: 0.7780
- Recall: 0.7797
- Accuracy: 0.7797

## Model description

This is the best-performing REGRESSION model for English irony detection. The model was fine-tuned both a mix of English and Dutch tweets.
The model predicts one numerical value indicating irony likelihood, where 0 is not ironic and 6 is ironic.

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 10

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Mse    | Rmse   | Mae    | R2      | F1     | Precision | Recall | Accuracy |
|:-------------:|:------:|:----:|:---------------:|:------:|:------:|:------:|:-------:|:------:|:---------:|:------:|:--------:|
| 1.7333        | 0.4630 | 100  | 1.8475          | 9.7883 | 3.1286 | 2.2877 | -0.4094 | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 1.6952        | 0.9259 | 200  | 1.7889          | 8.8708 | 2.9784 | 2.2442 | -0.2773 | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 1.6175        | 1.3889 | 300  | 1.6295          | 7.6123 | 2.7590 | 2.0223 | -0.0961 | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 1.4401        | 1.8519 | 400  | 1.4962          | 6.6368 | 2.5762 | 1.8601 | 0.0444  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 1.2553        | 2.3148 | 500  | 1.3949          | 5.9003 | 2.4291 | 1.7518 | 0.1504  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 1.2296        | 2.7778 | 600  | 1.3520          | 5.9339 | 2.4360 | 1.6730 | 0.1456  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 1.0909        | 3.2407 | 700  | 1.2565          | 5.3251 | 2.3076 | 1.5831 | 0.2332  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 1.0031        | 3.7037 | 800  | 1.2159          | 4.7598 | 2.1817 | 1.5709 | 0.3146  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
| 0.9833        | 4.1667 | 900  | 1.1544          | 4.6141 | 2.1480 | 1.5031 | 0.3356  | 0.7296 | 0.8018    | 0.7572 | 0.7572   |
| 0.825         | 4.6296 | 1000 | 1.1512          | 5.0019 | 2.2365 | 1.4608 | 0.2798  | 0.7757 | 0.7943    | 0.7859 | 0.7859   |
| 0.8187        | 5.0926 | 1100 | 1.1150          | 4.9111 | 2.2161 | 1.4352 | 0.2928  | 0.7815 | 0.7849    | 0.7859 | 0.7859   |
| 0.7138        | 5.5556 | 1200 | 1.0724          | 4.8492 | 2.2021 | 1.3871 | 0.3018  | 0.7766 | 0.7791    | 0.7807 | 0.7807   |
| 0.6706        | 6.0185 | 1300 | 1.0560          | 4.9024 | 2.2141 | 1.3650 | 0.2941  | 0.7786 | 0.7823    | 0.7833 | 0.7833   |
| 0.6112        | 6.4815 | 1400 | 1.0594          | 5.0772 | 2.2533 | 1.3694 | 0.2689  | 0.7750 | 0.7759    | 0.7781 | 0.7781   |
| 0.5906        | 6.9444 | 1500 | 1.0611          | 5.1421 | 2.2676 | 1.3794 | 0.2596  | 0.7736 | 0.7734    | 0.7755 | 0.7755   |
| 0.5597        | 7.4074 | 1600 | 1.0286          | 5.0419 | 2.2454 | 1.3290 | 0.2740  | 0.7839 | 0.7879    | 0.7885 | 0.7885   |
| 0.5422        | 7.8704 | 1700 | 1.0531          | 5.2061 | 2.2817 | 1.3596 | 0.2504  | 0.7672 | 0.7678    | 0.7702 | 0.7702   |
| 0.5255        | 8.3333 | 1800 | 1.0478          | 5.2565 | 2.2927 | 1.3372 | 0.2431  | 0.7811 | 0.7853    | 0.7859 | 0.7859   |
| 0.5116        | 8.7963 | 1900 | 1.0544          | 5.2090 | 2.2823 | 1.3546 | 0.2500  | 0.7721 | 0.7733    | 0.7755 | 0.7755   |
| 0.5213        | 9.2593 | 2000 | 1.0423          | 5.1715 | 2.2741 | 1.3341 | 0.2554  | 0.7819 | 0.7846    | 0.7859 | 0.7859   |
| 0.4999        | 9.7222 | 2100 | 1.0566          | 5.2819 | 2.2982 | 1.3481 | 0.2395  | 0.7721 | 0.7733    | 0.7755 | 0.7755   |


### Framework versions

- Transformers 4.40.2
- Pytorch 2.1.2
- Datasets 2.18.0
- Tokenizers 0.19.1