Instructions to use Mardiyyah/base_bioformer_3class_label_all_false with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Mardiyyah/base_bioformer_3class_label_all_false with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="Mardiyyah/base_bioformer_3class_label_all_false")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("Mardiyyah/base_bioformer_3class_label_all_false") model = AutoModelForTokenClassification.from_pretrained("Mardiyyah/base_bioformer_3class_label_all_false", device_map="auto") - Notebooks
- Google Colab
- Kaggle
base_bioformer_3class_label_all_false
This model is a fine-tuned version of bioformers/bioformer-16l on the OTAR3088/CeLLaTe-ner-3class-iob_final dataset. It achieves the following results on the evaluation set:
- Loss: 0.0877
- Precision: 0.6764
- Recall: 0.6396
- Micro F1: 0.6575
- Weighted F1: 0.6540
- Macro F1: 0.6435
- Accuracy: 0.9819
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 3407
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.01
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | Micro F1 | Weighted F1 | Macro F1 | Accuracy |
|---|---|---|---|---|---|---|---|---|---|
| 0.3786 | 1.0 | 263 | 0.0778 | 0.4593 | 0.5926 | 0.5175 | 0.5167 | 0.5114 | 0.9739 |
| 0.0586 | 2.0 | 526 | 0.0647 | 0.6564 | 0.6092 | 0.6319 | 0.6293 | 0.6226 | 0.9811 |
| 0.04 | 3.0 | 789 | 0.0654 | 0.6256 | 0.6359 | 0.6307 | 0.6277 | 0.6200 | 0.9804 |
| 0.0291 | 4.0 | 1052 | 0.0731 | 0.6200 | 0.6120 | 0.6160 | 0.6129 | 0.6077 | 0.9794 |
| 0.0232 | 5.0 | 1315 | 0.0767 | 0.5805 | 0.6083 | 0.5941 | 0.5910 | 0.5861 | 0.9773 |
| 0.0184 | 6.0 | 1578 | 0.0769 | 0.5881 | 0.6120 | 0.5998 | 0.5960 | 0.5902 | 0.9790 |
| 0.0145 | 7.0 | 1841 | 0.0785 | 0.6280 | 0.6442 | 0.6360 | 0.6337 | 0.6261 | 0.9804 |
| 0.0127 | 8.0 | 2104 | 0.0811 | 0.6587 | 0.6332 | 0.6457 | 0.6429 | 0.6341 | 0.9815 |
| 0.0109 | 9.0 | 2367 | 0.0881 | 0.6764 | 0.6396 | 0.6575 | 0.6540 | 0.6435 | 0.9819 |
| 0.0093 | 10.0 | 2630 | 0.0919 | 0.6376 | 0.6276 | 0.6326 | 0.6287 | 0.6201 | 0.9809 |
| 0.0081 | 11.0 | 2893 | 0.0868 | 0.6257 | 0.6608 | 0.6428 | 0.6420 | 0.6349 | 0.9811 |
| 0.0071 | 12.0 | 3156 | 0.0944 | 0.6594 | 0.6442 | 0.6517 | 0.6481 | 0.6379 | 0.9815 |
| 0.0064 | 13.0 | 3419 | 0.0973 | 0.6513 | 0.6267 | 0.6388 | 0.6346 | 0.6256 | 0.9812 |
| 0.0057 | 14.0 | 3682 | 0.0974 | 0.6146 | 0.6276 | 0.6211 | 0.6194 | 0.6136 | 0.9796 |
Framework versions
- Transformers 4.48.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.2
- Tokenizers 0.21.0
- Downloads last month
- 25