Instructions to use kmouratidis/Devstral-Small-2507-Rebased-Vision with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kmouratidis/Devstral-Small-2507-Rebased-Vision with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="kmouratidis/Devstral-Small-2507-Rebased-Vision", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("kmouratidis/Devstral-Small-2507-Rebased-Vision")
model = AutoModelForMultimodalLM.from_pretrained("kmouratidis/Devstral-Small-2507-Rebased-Vision", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kmouratidis/Devstral-Small-2507-Rebased-Vision with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kmouratidis/Devstral-Small-2507-Rebased-Vision"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmouratidis/Devstral-Small-2507-Rebased-Vision",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/kmouratidis/Devstral-Small-2507-Rebased-Vision

SGLang

How to use kmouratidis/Devstral-Small-2507-Rebased-Vision with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kmouratidis/Devstral-Small-2507-Rebased-Vision" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmouratidis/Devstral-Small-2507-Rebased-Vision",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kmouratidis/Devstral-Small-2507-Rebased-Vision" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmouratidis/Devstral-Small-2507-Rebased-Vision",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use kmouratidis/Devstral-Small-2507-Rebased-Vision with Docker Model Runner:
```
docker model run hf.co/kmouratidis/Devstral-Small-2507-Rebased-Vision
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Devstral-Small-2507-Rebased-Vision

This model was created by taking Mistral-Small-3.2-24B-Instruct-2506 and replacing the weights under the language_model with the weights from Devstral-Small-2507. The result is Devstral with vision capabilities, but you should expect a small quality degradation.

Notes: I used unsloth's uploads of these models for convenience, since they include some extra files and configs too. I didn't name this "-Vision" because it was not trained or finetuned after weight rebase, and in case a future version by mistralai has vision.

The code will be released soon.

Evaluation

Evaluation was performed on 7 benchmarks using lm_eval and sglang. Scripts and other details will also be released with the code. This is not a comprehensive evaluation, and it's not directly comparable to the official benchmark numbers from Mistral, the goal was to approximate quality degradation. Make sure to test on your own downstream tasks!

Tasks	Metric	Devstral-Small-2507	Devstral-Small-2507-rebased	Relative Loss (%)	Relative Stderr (%)
arc_challenge_chat	exact_match	0.9292	0.9283	0.10%	±0.81%
eq_bench	eqbench	72.3376	73.7481	-1.95%	±3.52%
gsm8k	exact_match	0.8643	0.862	0.27%	±1.09%
gsm8k	exact_match	0.8605	0.8567	0.44%	±1.10%
ifeval	inst_level_loose_acc	0.6631	0.6595	0.54%	N/A
ifeval	inst_level_strict_acc	0.6067	0.6019	0.79%	N/A
ifeval	prompt_level_loose_acc	0.5619	0.5545	1.32%	±3.81%
ifeval	prompt_level_strict_acc	0.4917	0.4861	1.14%	±4.37%
mbpp	pass_at_1	0.118	0.112	5.08%	±12.20%
mmlu_pro	exact_match	0.5786	0.579	-0.07%	±0.76%
triviaqa	exact_match	0.7075	0.7068	0.10%	±0.48%

Downloads last month: 6

Safetensors

Model size

24B params

Tensor type

F16

Model tree for kmouratidis/Devstral-Small-2507-Rebased-Vision

Base model

mistralai/Mistral-Small-3.1-24B-Base-2503

Finetuned

mistralai/Mistral-Small-3.1-24B-Instruct-2503

Finetuned

mistralai/Devstral-Small-2507

Finetuned

(7)

this model

Quantizations

2 models