Image-Text-to-Text
Transformers
Safetensors
gemma3
medical
unsloth
radiology
clinical-reasoning
dermatology
pathology
ophthalmology
chest-x-ray
conversational
text-generation-inference
Instructions to use unsloth/medgemma-4b-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use unsloth/medgemma-4b-it with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="unsloth/medgemma-4b-it") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("unsloth/medgemma-4b-it") model = AutoModelForMultimodalLM.from_pretrained("unsloth/medgemma-4b-it", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use unsloth/medgemma-4b-it with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "unsloth/medgemma-4b-it" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/medgemma-4b-it", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/unsloth/medgemma-4b-it
- SGLang
How to use unsloth/medgemma-4b-it with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "unsloth/medgemma-4b-it" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/medgemma-4b-it", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "unsloth/medgemma-4b-it" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/medgemma-4b-it", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Unsloth Studio
How to use unsloth/medgemma-4b-it with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for unsloth/medgemma-4b-it to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for unsloth/medgemma-4b-it to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for unsloth/medgemma-4b-it to start chatting
Load model with FastModel
pip install unsloth from unsloth import FastModel model, tokenizer = FastModel.from_pretrained( model_name="unsloth/medgemma-4b-it", max_seq_length=2048, ) - Docker Model Runner
How to use unsloth/medgemma-4b-it with Docker Model Runner:
docker model run hf.co/unsloth/medgemma-4b-it
Improve model card: Add project page and code link
Browse filesThis PR improves the model card by:
* Adding a link to https://medxpertqa.github.io.
* Adding a link to https://github.com/TsinghuaC3I/MedXpertQA to make it easier for people to find the code.
README.md
CHANGED
|
@@ -1,18 +1,11 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
license: other
|
| 3 |
license_name: health-ai-developer-foundations
|
| 4 |
license_link: https://developers.google.com/health-ai-developer-foundations/terms
|
| 5 |
-
library_name: transformers
|
| 6 |
pipeline_tag: image-text-to-text
|
| 7 |
-
extra_gated_heading: Access MedGemma on Hugging Face
|
| 8 |
-
extra_gated_prompt: >-
|
| 9 |
-
To access MedGemma on Hugging Face, you're required to review and
|
| 10 |
-
agree to [Health AI Developer Foundation's terms of use](https://developers.google.com/health-ai-developer-foundations/terms).
|
| 11 |
-
To do this, please ensure you're logged in to Hugging Face and click below.
|
| 12 |
-
Requests are processed immediately.
|
| 13 |
-
extra_gated_button_content: Acknowledge license
|
| 14 |
-
base_model:
|
| 15 |
-
- google/medgemma-4b-it
|
| 16 |
tags:
|
| 17 |
- medical
|
| 18 |
- unsloth
|
|
@@ -22,7 +15,14 @@ tags:
|
|
| 22 |
- pathology
|
| 23 |
- ophthalmology
|
| 24 |
- chest-x-ray
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
---
|
|
|
|
| 26 |
<div>
|
| 27 |
<p style="margin-top: 0;margin-bottom: 0;">
|
| 28 |
<em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
|
|
@@ -40,11 +40,13 @@ tags:
|
|
| 40 |
</div>
|
| 41 |
</div>
|
| 42 |
|
| 43 |
-
|
| 44 |
# MedGemma model card
|
| 45 |
|
| 46 |
**Model documentation:** [MedGemma](https://developers.google.com/health-ai-developer-foundations/medgemma)
|
| 47 |
|
|
|
|
|
|
|
|
|
|
| 48 |
**Resources:**
|
| 49 |
|
| 50 |
* Model on Google Cloud Model Garden: [MedGemma](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/medgemma)
|
|
@@ -104,7 +106,6 @@ model locally on GPU. If you want to use the model at scale, we recommend that
|
|
| 104 |
you create a production version using [Model
|
| 105 |
Garden](https://cloud.google.com/model-garden).
|
| 106 |
|
| 107 |
-
|
| 108 |
First, install the Transformers library. Gemma 3 is supported starting from
|
| 109 |
transformers 4.50.0.
|
| 110 |
|
|
@@ -457,39 +458,10 @@ incorporated (described next).
|
|
| 457 |
created by a research team led by Jason J. Lau, Soumya Gayen, Asma Ben
|
| 458 |
Abacha, and Dina Demner-Fushman and their affiliated institutions (the US
|
| 459 |
National Library of Medicine and National Institutes of Health)
|
| 460 |
-
* [MedExpQA
|
| 461 |
-
|
| 462 |
-
|
| 463 |
-
*
|
| 464 |
-
dataset was developed by researchers at Tsinghua University (Beijing, China)
|
| 465 |
-
and Shanghai Artificial Intelligence Laboratory (Shanghai, China).
|
| 466 |
-
|
| 467 |
-
In addition to the public datasets listed above, MedGemma was also trained on
|
| 468 |
-
de-identified datasets licensed for research or collected internally at Google
|
| 469 |
-
from consented participants.
|
| 470 |
-
|
| 471 |
-
* Radiology dataset 1: De-identified dataset of different CT studies across
|
| 472 |
-
body parts from a US-based radiology outpatient diagnostic center network.
|
| 473 |
-
* Ophthalmology dataset 1: De-identified dataset of fundus images from
|
| 474 |
-
diabetic retinopathy screening.
|
| 475 |
-
* Dermatology dataset 1: De-identified dataset of teledermatology skin
|
| 476 |
-
condition images (both clinical and dermatoscopic) from Colombia.
|
| 477 |
-
* Dermatology dataset 2: De-identified dataset of skin cancer images (both
|
| 478 |
-
clinical and dermatoscopic) from Australia.
|
| 479 |
-
* Dermatology dataset 3: De-identified dataset of non-diseased skin images
|
| 480 |
-
from an internal data collection effort.
|
| 481 |
-
* Pathology dataset 1: De-identified dataset of histopathology H&E whole slide
|
| 482 |
-
images created in collaboration with an academic research hospital and
|
| 483 |
-
biobank in Europe. Comprises de-identified colon, prostate, and lymph nodes.
|
| 484 |
-
* Pathology dataset 2: De-identified dataset of lung histopathology H&E and
|
| 485 |
-
IHC whole slide images created by a commercial biobank in the United States.
|
| 486 |
-
* Pathology dataset 3: De-identified dataset of prostate and lymph node H&E
|
| 487 |
-
and IHC histopathology whole slide images created by a contract research
|
| 488 |
-
organization in the United States.
|
| 489 |
-
* Pathology dataset 4: De-identified dataset of histopathology, predominantly
|
| 490 |
-
H\&E whole slide images created in collaboration with a large, tertiary
|
| 491 |
-
teaching hospital in the United States. Comprises a diverse set of tissue
|
| 492 |
-
and stain types, predominantly H&E.
|
| 493 |
|
| 494 |
### Data citation
|
| 495 |
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- google/medgemma-4b-it
|
| 4 |
+
library_name: transformers
|
| 5 |
license: other
|
| 6 |
license_name: health-ai-developer-foundations
|
| 7 |
license_link: https://developers.google.com/health-ai-developer-foundations/terms
|
|
|
|
| 8 |
pipeline_tag: image-text-to-text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
tags:
|
| 10 |
- medical
|
| 11 |
- unsloth
|
|
|
|
| 15 |
- pathology
|
| 16 |
- ophthalmology
|
| 17 |
- chest-x-ray
|
| 18 |
+
extra_gated_heading: Access MedGemma on Hugging Face
|
| 19 |
+
extra_gated_prompt: To access MedGemma on Hugging Face, you're required to review
|
| 20 |
+
and agree to [Health AI Developer Foundation's terms of use](https://developers.google.com/health-ai-developer-foundations/terms).
|
| 21 |
+
To do this, please ensure you're logged in to Hugging Face and click below. Requests
|
| 22 |
+
are processed immediately.
|
| 23 |
+
extra_gated_button_content: Acknowledge license
|
| 24 |
---
|
| 25 |
+
|
| 26 |
<div>
|
| 27 |
<p style="margin-top: 0;margin-bottom: 0;">
|
| 28 |
<em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
|
|
|
|
| 40 |
</div>
|
| 41 |
</div>
|
| 42 |
|
|
|
|
| 43 |
# MedGemma model card
|
| 44 |
|
| 45 |
**Model documentation:** [MedGemma](https://developers.google.com/health-ai-developer-foundations/medgemma)
|
| 46 |
|
| 47 |
+
Project page: https://medxpertqa.github.io
|
| 48 |
+
Code: https://github.com/TsinghuaC3I/MedXpertQA
|
| 49 |
+
|
| 50 |
**Resources:**
|
| 51 |
|
| 52 |
* Model on Google Cloud Model Garden: [MedGemma](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/medgemma)
|
|
|
|
| 106 |
you create a production version using [Model
|
| 107 |
Garden](https://cloud.google.com/model-garden).
|
| 108 |
|
|
|
|
| 109 |
First, install the Transformers library. Gemma 3 is supported starting from
|
| 110 |
transformers 4.50.0.
|
| 111 |
|
|
|
|
| 458 |
created by a research team led by Jason J. Lau, Soumya Gayen, Asma Ben
|
| 459 |
Abacha, and Dina Demner-Fushman and their affiliated institutions (the US
|
| 460 |
National Library of Medicine and National Institutes of Health)
|
| 461 |
+
* [MedExpQA: Multilingual benchmarking of Large Language Models for
|
| 462 |
+
Medical Question
|
| 463 |
+
Answering](https://www.sciencedirect.com/science/article/pii/S0933365724001805)
|
| 464 |
+
* MedXpertQA: [arXiv:2501.18362v2](https://arxiv.org/abs/2501.18362)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 465 |
|
| 466 |
### Data citation
|
| 467 |
|