Instructions to use mendicant04/DermoGPT-RL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mendicant04/DermoGPT-RL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="mendicant04/DermoGPT-RL")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("mendicant04/DermoGPT-RL")
model = AutoModelForImageTextToText.from_pretrained("mendicant04/DermoGPT-RL")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use mendicant04/DermoGPT-RL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mendicant04/DermoGPT-RL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mendicant04/DermoGPT-RL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/mendicant04/DermoGPT-RL

SGLang

How to use mendicant04/DermoGPT-RL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mendicant04/DermoGPT-RL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mendicant04/DermoGPT-RL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mendicant04/DermoGPT-RL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mendicant04/DermoGPT-RL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use mendicant04/DermoGPT-RL with Docker Model Runner:
```
docker model run hf.co/mendicant04/DermoGPT-RL
```

Request access to DermoGPT-RL

Access is intended for research use only. Requests are reviewed manually by the authors.

Please provide accurate affiliation and intended-use information. This model is not a medical device and must not be used for clinical diagnosis, triage, or treatment decisions.

DermoGPT-RL

DermoGPT-RL is a dermatology-oriented multimodal model for research on dermatology image understanding, lesion description, morphology-aware reasoning, and medical visual question answering.

This repository contains a merged Hugging Face Transformers checkpoint based on a Qwen3-VL architecture and further optimized for dermatology-oriented instruction following and reasoning.

Model Details

Model type: vision-language model
Architecture: Qwen3VLForConditionalGeneration
Precision: float16
Format: sharded safetensors
Number of shards: 4
Repository size: approximately 17 GB
Primary language: English
Domain: dermatology and dermoscopy

Intended Use

DermoGPT-RL is intended for:

research on dermatology-oriented multimodal models;
medical visual question answering under controlled research settings;
lesion morphology description and reasoning;
benchmark evaluation on DermoGPT/DermoBench-style tasks;
further academic analysis of reinforcement learning for medical VLMs.

Out-of-Scope Use

This model is not intended for:

clinical deployment;
autonomous diagnosis;
patient triage;
treatment recommendation;
direct patient-facing medical advice;
replacement of professional medical judgment.

Loading

Install recent versions of the required libraries:

pip install -U transformers accelerate safetensors pillow

Example loading code:

import torch
from PIL import Image
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "mendicant04/DermoGPT-RL"

model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

image = Image.open("path/to/dermatology_image.jpg").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "Describe the lesion and summarize the key morphological findings."},
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = processor(
    text=[text],
    images=[image],
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
    )

output = processor.batch_decode(
    generated_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)[0]
print(output)

Related Resources

Project repository: https://github.com/mendicant04/DermoGPT
Benchmark/data resources: https://huggingface.co/mendicant04
Codebase adapted from: https://github.com/2U1/Qwen-VL-Series-Finetune

Limitations

The model may produce incorrect, incomplete, or overconfident outputs.
Performance may vary across skin tones, imaging devices, disease categories, and acquisition conditions.
Dermatology reasoning outputs should be interpreted as research model outputs, not as clinical advice.
Any downstream use requires independent validation and appropriate medical oversight.

Ethical and Medical Considerations

Dermatology is a high-stakes medical domain. This model should only be used for research and evaluation. It must not be used as a standalone diagnostic or treatment system. Researchers are responsible for ensuring that their use complies with applicable laws, institutional review requirements, data-use agreements, and medical AI safety standards.

Downloads last month: 49

Safetensors

Model size

9B params

Tensor type

F16

mendicant04
/

DermoGPT-RL