---
library_name: transformers
license: apache-2.0
base_model: google-t5/t5-base
tags:
- generated_from_trainer
datasets:
- generator
metrics:
- accuracy
model-index:
- name: google_t5_language_ID
  results:
  - task:
      type: text2text-generation
      name: Sequence-to-sequence Language Modeling
    dataset:
      name: generator
      type: generator
      config: default
      split: train
      args: default
    metrics:
    - type: accuracy
      value: 0.6179074697593216
      name: Accuracy
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# google_t5_language_ID

This model is a fine-tuned version of [google-t5/t5-base](https://huggingface.co/google-t5/t5-base) on the generator dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5429
- Accuracy: 0.6179
- F1 Macro: 0.3389
- F1 Weighted: 0.5774
- Precision Macro: 0.3873
- Recall Macro: 0.3627

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 128
- optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine_with_restarts
- lr_scheduler_warmup_steps: 1000
- training_steps: 60000

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Accuracy | F1 Macro | F1 Weighted | Precision Macro | Recall Macro |
|:-------------:|:------:|:----:|:---------------:|:--------:|:--------:|:-----------:|:---------------:|:------------:|
| 0.1943        | 0.0083 | 500  | 0.6981          | 0.4018   | 0.3139   | 0.3488      | 0.4624          | 0.3616       |
| 0.0812        | 0.0167 | 1000 | 0.7371          | 0.4086   | 0.3323   | 0.3446      | 0.5179          | 0.3940       |
| 0.049         | 0.025  | 1500 | 0.7806          | 0.4534   | 0.3793   | 0.3793      | 0.5316          | 0.4534       |
| 0.0518        | 0.0333 | 2000 | 0.5042          | 0.5845   | 0.5071   | 0.5258      | 0.5576          | 0.5637       |
| 0.0452        | 0.0417 | 2500 | 0.5120          | 0.6204   | 0.5554   | 0.5554      | 0.6496          | 0.6204       |
| 0.0288        | 0.05   | 3000 | 0.4798          | 0.6018   | 0.5230   | 0.5618      | 0.6077          | 0.5603       |
| 0.0341        | 0.0583 | 3500 | 0.4764          | 0.6098   | 0.5456   | 0.5658      | 0.6528          | 0.5881       |
| 0.0762        | 0.0667 | 4000 | 0.4389          | 0.6251   | 0.5296   | 0.5688      | 0.6091          | 0.5820       |
| 0.0189        | 0.075  | 4500 | 0.4167          | 0.6681   | 0.6068   | 0.6068      | 0.7167          | 0.6681       |
| 0.0235        | 0.0833 | 5000 | 0.4673          | 0.6599   | 0.6018   | 0.6018      | 0.7393          | 0.6599       |
| 0.0274        | 0.0917 | 5500 | 0.3304          | 0.6958   | 0.6102   | 0.6555      | 0.6868          | 0.6478       |
| 0.0198        | 0.1    | 6000 | 0.4752          | 0.6569   | 0.5877   | 0.6095      | 0.7165          | 0.6335       |
| 0.0246        | 0.1083 | 6500 | 0.4657          | 0.6540   | 0.5800   | 0.6015      | 0.6400          | 0.6306       |
| 0.0241        | 0.1167 | 7000 | 0.5429          | 0.6179   | 0.3389   | 0.5774      | 0.3873          | 0.3627       |


### Framework versions

- Transformers 4.57.1
- Pytorch 2.9.0+cu128
- Datasets 4.3.0
- Tokenizers 0.22.1