---
library_name: transformers
license: apache-2.0
base_model: uitnlp/CafeBERT
tags:
- generated_from_trainer
model-index:
- name: CafeBERT_massive_crf_v2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# CafeBERT_massive_crf_v2

This model is a fine-tuned version of [uitnlp/CafeBERT](https://huggingface.co/uitnlp/CafeBERT) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 5.5963
- Slot P: 0.0077
- Slot R: 0.0082
- Slot F1: 0.0079
- Slot Exact Match: 0.3246
- Intent Acc: 0.8751

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 256
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.06
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Slot P | Slot R | Slot F1 | Slot Exact Match | Intent Acc |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:-------:|:----------------:|:----------:|
| No log        | 1.0   | 45   | 15.8863         | 0.0    | 0.0    | 0.0     | 0.4088           | 0.0733     |
| 69.6998       | 2.0   | 90   | 7.8039          | 0.0080 | 0.0076 | 0.0078  | 0.3438           | 0.5032     |
| 22.3589       | 3.0   | 135  | 4.5345          | 0.0091 | 0.0100 | 0.0095  | 0.3227           | 0.7846     |
| 10.4218       | 4.0   | 180  | 4.0667          | 0.0110 | 0.0111 | 0.0111  | 0.3384           | 0.8406     |
| 6.8199        | 5.0   | 225  | 3.8871          | 0.0092 | 0.0100 | 0.0096  | 0.3261           | 0.8623     |
| 5.4068        | 6.0   | 270  | 3.9234          | 0.0106 | 0.0117 | 0.0111  | 0.3212           | 0.8633     |
| 4.2552        | 7.0   | 315  | 4.0332          | 0.0115 | 0.0129 | 0.0122  | 0.3168           | 0.8657     |
| 3.5197        | 8.0   | 360  | 4.2753          | 0.0080 | 0.0088 | 0.0084  | 0.3222           | 0.8647     |
| 2.8374        | 9.0   | 405  | 4.6031          | 0.0099 | 0.0106 | 0.0102  | 0.3256           | 0.8701     |
| 2.2784        | 10.0  | 450  | 4.7992          | 0.0118 | 0.0129 | 0.0123  | 0.3237           | 0.8652     |
| 2.2784        | 11.0  | 495  | 5.0575          | 0.0118 | 0.0129 | 0.0123  | 0.3222           | 0.8652     |
| 1.8204        | 12.0  | 540  | 5.1371          | 0.0088 | 0.0094 | 0.0091  | 0.3266           | 0.8731     |
| 1.5073        | 13.0  | 585  | 5.4768          | 0.0109 | 0.0123 | 0.0116  | 0.3133           | 0.8677     |
| 1.275         | 14.0  | 630  | 5.5963          | 0.0077 | 0.0082 | 0.0079  | 0.3246           | 0.8751     |


### Framework versions

- Transformers 4.55.0
- Pytorch 2.7.0+cu126
- Datasets 3.6.0
- Tokenizers 0.21.4