You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

bert-tiny-distilled

This model is a fine-tuned version of google/bert_uncased_L-2_H-128_A-2 on the hate_speech18 dataset. It achieves the following results on the evaluation set:

eval_loss: 0.4745
eval_accuracy: 0.8556
eval_runtime: 20.4101
eval_samples_per_second: 107.202
eval_steps_per_second: 0.882
epoch: 1.0
step: 479

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 7.474176155523857e-05
train_batch_size: 16
eval_batch_size: 128
seed: 23
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 4

Framework versions

Transformers 4.36.0.dev0
Pytorch 2.1.1
Datasets 2.15.0
Tokenizers 0.15.0

Downloads last month: -

Safetensors

Model size

4.39M params

Tensor type

F32

Model tree for joseph10/bert-tiny-distilled

Base model

google/bert_uncased_L-2_H-128_A-2

Finetuned

(63)

this model

joseph10
/

bert-tiny-distilled

You need to agree to share your contact information to access this model

bert-tiny-distilled

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Framework versions

Model tree for joseph10/bert-tiny-distilled

Dataset used to train joseph10/bert-tiny-distilled