MobileBERT Fine-tuned on Nelson Textbook of Pediatrics

This model is a fine-tuned version of google/mobilebert-uncased trained on excerpts from the Nelson Textbook of Pediatrics. It is designed to serve as a lightweight, on-device capable medical assistant model for pediatric reference tasks.

Intended Use

This model is intended for:

• Medical question answering (focused on pediatrics)
• Clinical decision support in low-resource environments
• Integration into apps like Nelson-GPT for fast inference

Note: This model is for educational and experimental use only and should not replace professional medical advice.

Training Details

• Base model: mobilebert-uncased
• Training framework: Transformers + PyTorch
• Dataset: Nelson Textbook (manually curated excerpts)
• Epochs: [insert]
• Learning rate: [insert]

How to Use

from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch

tokenizer = AutoTokenizer.from_pretrained("drzeeIslam/mobilebert-nelson")
model = AutoModelForQuestionAnswering.from_pretrained("drzeeIslam/mobilebert-nelson")

question = "What is the treatment for nephrotic syndrome?"
context = "The first-line treatment for nephrotic syndrome in children is corticosteroid therapy..."

inputs = tokenizer(question, context, return_tensors="pt")
outputs = model(**inputs)

start = torch.argmax(outputs.start_logits)
end = torch.argmax(outputs.end_logits) + 1

answer = tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(inputs['input_ids'][0][start:end]))
print(answer)