--- library_name: transformers base_model: allenai/scibert_scivocab_cased tags: - generated_from_trainer - classification metrics: - precision - recall - f1 - accuracy model-index: - name: results_bert-finetuned-ner results: [] license: apache-2.0 datasets: - JonyC/ScienceGlossary language: - en --- IMPORTENT! READ THIS! ## Model description This model recognizes scientific terms in a given token. The best way to use it is as follows: ```python from transformers import AutoTokenizer, AutoModelForTokenClassification from nltk.tokenize import word_tokenize import torch tokenizer = AutoTokenizer.from_pretrained("JonyC/results_bert-finetuned-ner") model = AutoModelForTokenClassification.from_pretrained("JonyC/results_bert-finetuned-ner") words = word_tokenize("scientific_text") inputs = tokenizer(words, return_tensors="pt", truncation=True, padding=True) words_output, pred_labels = [], [] # Loop over tokenized inputs and send each one to the model for i in range(inputs['input_ids'].shape[0]): # Loop over each sentence in the batch input_data = {key: value[i].unsqueeze(0).to(model.device) for key, value in inputs.items()} # Prepare single input # Get model predictions for the current tokenized input with torch.no_grad(): outputs = model(**input_data).logits # Convert logits to predictions predictions = torch.argmax(outputs, dim=2) # Map token IDs to words and labels tokens = tokenizer.convert_ids_to_tokens(input_data['input_ids'][0]) # Get the word ids using the tokenizer labels = [model.config.id2label[p.item()] for p in predictions[0]] # Align tokens properly and ignore special tokens aligned_tokens = [] aligned_labels = [] word_ids = inputs.word_ids() current_word = "" current_label = "" # Loop through the tokens and their corresponding word ids to group subwords into full words for token, word_id, label in zip(tokens, word_ids, labels): # Skip special tokens if token.startswith('[CLS]') or token.startswith('[SEP]') or token.startswith('[PAD]') or token.startswith('[UNK]'): continue # If the token corresponds to the start of a new word if word_id != word_ids[0]: # Handle the first word if current_word != "": aligned_tokens.append(current_word) # Append the full word aligned_labels.append(current_label) # Append the label current_word = token current_label = label else: current_word += token # Append subwords to the current word # Add the last word if needed if current_word != "": aligned_tokens.append(current_word) aligned_labels.append(current_label) words_output.append(current_word) pred_labels.append(current_label) for w, p in zip(words_output, pred_labels): print(f"Word: {w}, Predicted Label: {p}") ``` ## Example usage Given the following text: "Quantum computing is a new field that changes how we think about solving complex problems. Unlike regular computers that use bits (which are either 0 or 1), quantum computers use qubits, which can be both 0 and 1 at the same time, thanks to a property called superposition. One important feature of quantum computers is quantum entanglement, where two qubits can be linked in such a way that changing one will instantly affect the other, no matter how far apart they are. This allows quantum computers to perform certain calculations much faster than traditional computers. For example, quantum computers could one day factor large numbers much faster, which is currently a task that takes regular computers a very long time. However, there are still challenges to overcome, like maintaining the qubits' state long enough to do calculations without errors. Scientists are working on ways to fix these errors, which is necessary for quantum computers to work on a large scale and solve real-world problems more efficiently than today's computers." the results are:
``` Word: qubits, Predicted Label: I-Scns. Word: superposition, Predicted Label: B-Scns. Word: entanglement, Predicted Label: B-Scns. Word: qubits, Predicted Label: I-Scns. Word: qubits, Predicted Label: I-Scns. ``` (all the others defined as 'O', meaning non-science terms) # results_bert-finetuned-ner This model is a fine-tuned version of [allenai/scibert_scivocab_cased](https://huggingface.co/allenai/scibert_scivocab_cased) on the [JonyC/ScienceGlossary](https://huggingface.co/datasets/JonyC/ScienceGlossary) dataset. It achieves the following results on the evaluation set: - Loss: 0.2219 - Precision: 0.7689 - Recall: 0.7441 - F1: 0.7563 - Accuracy: 0.9336 - ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - num_epochs: 25 ### Training results | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | |:-------------:|:-----:|:------:|:---------------:|:---------:|:------:|:------:|:--------:| | 0.139 | 1.0 | 9399 | 0.1158 | 0.9515 | 0.9230 | 0.9370 | 0.9755 | | 0.1003 | 2.0 | 18798 | 0.1766 | 0.9570 | 0.8907 | 0.9226 | 0.9716 | | 0.1119 | 3.0 | 28197 | 0.2278 | 0.9844 | 0.8075 | 0.8872 | 0.9608 | | 0.1204 | 4.0 | 37596 | 0.2130 | 0.9796 | 0.8226 | 0.8943 | 0.9623 | | 0.0983 | 5.0 | 46995 | 0.1947 | 0.9707 | 0.8390 | 0.9001 | 0.9669 | | 0.1313 | 6.0 | 56394 | 0.1767 | 0.8988 | 0.9261 | 0.9123 | 0.9669 | | 0.1012 | 7.0 | 65793 | 0.1513 | 0.9528 | 0.8946 | 0.9228 | 0.9744 | | 0.1264 | 8.0 | 75192 | 0.1829 | 0.8573 | 0.7993 | 0.8273 | 0.9611 | | 0.1521 | 9.0 | 84591 | 0.1943 | 0.9182 | 0.8471 | 0.8812 | 0.9650 | | 0.6277 | 10.0 | 93990 | 0.6086 | 0.0 | 0.0 | 0.0 | 0.8039 | | 0.4465 | 11.0 | 103389 | 0.2022 | 0.8728 | 0.8514 | 0.8620 | 0.9639 | | 0.1114 | 12.0 | 112788 | 0.1885 | 0.7967 | 0.8172 | 0.8068 | 0.9595 | | 0.1492 | 13.0 | 122187 | 0.2386 | 0.7724 | 0.6562 | 0.7096 | 0.9226 | | 0.1785 | 14.0 | 131586 | 0.2137 | 0.5960 | 0.7145 | 0.6499 | 0.9296 | | 0.1496 | 15.0 | 140985 | 0.2184 | 0.7454 | 0.7620 | 0.7536 | 0.9325 | | 0.1458 | 16.0 | 150384 | 0.2195 | 0.7639 | 0.7437 | 0.7536 | 0.9304 | | 0.1241 | 17.0 | 159783 | 0.2271 | 0.7737 | 0.7406 | 0.7568 | 0.9341 | | 0.1266 | 18.0 | 169182 | 0.2281 | 0.6259 | 0.6962 | 0.6592 | 0.9334 | | 0.1313 | 19.0 | 178581 | 0.2125 | 0.7702 | 0.7534 | 0.7617 | 0.9349 | | 0.1416 | 20.0 | 187980 | 0.2258 | 0.7707 | 0.7464 | 0.7583 | 0.9332 | | 0.1237 | 21.0 | 197379 | 0.2374 | 0.7691 | 0.7410 | 0.7548 | 0.9331 | | 0.1184 | 22.0 | 206778 | 0.2297 | 0.7598 | 0.7371 | 0.7483 | 0.9327 | | 0.1278 | 23.0 | 216177 | 0.2134 | 0.7695 | 0.7402 | 0.7546 | 0.9335 | | 0.1195 | 24.0 | 225576 | 0.2171 | 0.7701 | 0.7441 | 0.7569 | 0.9332 | | 0.1249 | 25.0 | 234975 | 0.2219 | 0.7689 | 0.7441 | 0.7563 | 0.9336 | ### Framework versions - Transformers 4.47.0 - Pytorch 2.5.1+cu124 - Datasets 3.2.0 - Tokenizers 0.21.0