basic_distil / README.md
ayshi's picture
Training in progress epoch 19
d82c6e8
metadata
license: apache-2.0
base_model: ayshi/basic_distil
tags:
  - generated_from_keras_callback
model-index:
  - name: ayshi/basic_distil
    results: []

ayshi/basic_distil

This model is a fine-tuned version of ayshi/basic_distil on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 0.0114
  • Validation Loss: 1.0035
  • Train Accuracy: 0.7911
  • Epoch: 19

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 640, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
  • training_precision: float32

Training results

Train Loss Validation Loss Train Accuracy Epoch
0.3282 0.7966 0.7644 0
0.2163 0.8219 0.7778 1
0.1323 0.8099 0.7778 2
0.0933 0.8337 0.7956 3
0.0619 0.9082 0.7689 4
0.0461 0.9380 0.7778 5
0.0495 0.9502 0.7556 6
0.0301 0.9445 0.7733 7
0.0249 0.9578 0.8 8
0.0209 0.9663 0.7911 9
0.0200 0.9828 0.7778 10
0.0159 0.9987 0.7689 11
0.0163 1.0120 0.7689 12
0.0142 1.0020 0.7956 13
0.0153 1.0270 0.7911 14
0.0142 1.0159 0.7822 15
0.0130 1.0049 0.7911 16
0.0129 1.0085 0.7956 17
0.0099 1.0033 0.7911 18
0.0114 1.0035 0.7911 19

Framework versions

  • Transformers 4.34.0
  • TensorFlow 2.13.0
  • Datasets 2.14.5
  • Tokenizers 0.14.1