Instructions to use Kimokcheon/Fundus-R1-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kimokcheon/Fundus-R1-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Kimokcheon/Fundus-R1-7B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Kimokcheon/Fundus-R1-7B")
model = AutoModelForMultimodalLM.from_pretrained("Kimokcheon/Fundus-R1-7B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Kimokcheon/Fundus-R1-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kimokcheon/Fundus-R1-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kimokcheon/Fundus-R1-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Kimokcheon/Fundus-R1-7B

SGLang

How to use Kimokcheon/Fundus-R1-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kimokcheon/Fundus-R1-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kimokcheon/Fundus-R1-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kimokcheon/Fundus-R1-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kimokcheon/Fundus-R1-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Kimokcheon/Fundus-R1-7B with Docker Model Runner:
```
docker model run hf.co/Kimokcheon/Fundus-R1-7B
```

Kimokcheon commited on Oct 8

Commit

19cbd05

verified ·

1 Parent(s): f428126

Update README.md

Browse files

Files changed (1) hide show

README.md +1 -7

README.md CHANGED Viewed

@@ -159,9 +159,7 @@ Keep this terminal running. Once the server is ready, check it from another term
 curl http://127.0.0.1:8000/v1/models
 ```
-This configuration allows one image per request, limits concurrency to one sequence, and disables CUDA graphs to reduce startup memory pressure. It is a deployment example, **not the paper's evaluation configuration**. Image-resolution limits and generation settings can affect results; increase the image budget when fine visual details are needed and memory permits. These settings do not guarantee that the model will fit on every GPU. For multiple GPUs, adjust `CUDA_VISIBLE_DEVICES` and add the corresponding `--tensor-parallel-size`.
-The server is bound to the local machine. Do not expose an unauthenticated endpoint containing medical images to a public network.
 ### Query a local image
@@ -275,8 +273,6 @@ ollama create fundus-r1-7b -f ./Modelfile
 ollama show fundus-r1-7b
 ```
-Check that `ollama show` lists the `vision` capability before querying images. If your version rejects the two-file import or shows only text capabilities, update Ollama and check the import logs; use the vLLM route above rather than dropping the projector.
 ### Query a local image
 ```bash
@@ -314,8 +310,6 @@ ollama run fundus-r1-7b ./fundus_image.jpg "Describe the retinal findings in thi
 See the [Ollama Modelfile reference](https://docs.ollama.com/modelfile), [Ollama vision guide](https://docs.ollama.com/capabilities/vision), and [llama.cpp multimodal conversion guide](https://github.com/ggml-org/llama.cpp/blob/master/tools/mtmd/README.md).
-**Validation status:** These deployment recipes are based on the checkpoint architecture and upstream documentation/code. They have not been end-to-end tested with this checkpoint. Validate image-conditioned outputs in your environment before relying on either integration; do not assume converted or quantized outputs reproduce the paper's reported results. The medical-use disclaimer above applies to both backends.
 ## Download Through HF Mirror
 For users in regions where the official Hugging Face endpoint is slow, the checkpoints can be downloaded through the Hugging Face mirror endpoint. Install the CLI with `python -m pip install --upgrade huggingface_hub` if it is not already available:

 curl http://127.0.0.1:8000/v1/models
 ```
+This configuration allows one image per request, limits concurrency to one sequence, and disables CUDA graphs to reduce startup memory pressure.
 ### Query a local image
 ollama show fundus-r1-7b
 ```
 ### Query a local image
 ```bash
 See the [Ollama Modelfile reference](https://docs.ollama.com/modelfile), [Ollama vision guide](https://docs.ollama.com/capabilities/vision), and [llama.cpp multimodal conversion guide](https://github.com/ggml-org/llama.cpp/blob/master/tools/mtmd/README.md).
 ## Download Through HF Mirror
 For users in regions where the official Hugging Face endpoint is slow, the checkpoints can be downloaded through the Hugging Face mirror endpoint. Install the CLI with `python -m pip install --upgrade huggingface_hub` if it is not already available: