--- library_name: vllm language: - en - fr - es - de - it - pt - nl - zh - ja - ko - ar license: apache-2.0 inference: false base_model: - unsloth/Ministral-3-3B-Reasoning-2512-GGUF - mistral-common - mistral - unsloth --- # mini Preview ![License: Public Domain](https://img.shields.io/badge/license-public%20domain-brightgreen.svg) - **Maintainer**: [Terry Eppler](https://gravatar.com/terryepplerphd) Based on the smallest model in the Ministral 3 family, **Minime** is a powerful, efficient tiny language model with vision capabilities. This model is the reasoning post-trained version, trained for reasoning tasks, making it ideal for math, coding and stem related use cases. The base model family is designed for edge deployment, capable of running on a wide range of hardware. Minime can even be deployed locally, fitting in 16GB of VRAM in BF16, and less than 8GB of RAM/VRAM when quantized. ## Key Features Minime consists of two main architectural components: - **3.4B Language Model** - **0.4B Vision Encoder** ThisReasoning model offers the following capabilities: - **Vision**: Enables the model to analyze images and provide insights based on visual content, in addition to text. - **Multilingual**: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic. - **System Prompt**: Maintains strong adherence and support for system prompts. - **Agentic**: Offers best-in-class agentic capabilities with native function calling and JSON outputting. - **Reasoning**: Excels at complex, multi-step reasoning and dynamic problem-solving. - **Edge-Optimized**: Delivers best-in-class performance at a small scale, deployable anywhere. - **Apache 2.0 License**: Open-source license allowing usage and modification for both commercial and non-commercial purposes. - **Large Context Window**: Supports a 256k context window. ### Use Cases Ideal for lightweight, real-time applications on edge or low-resource devices, such as: - Image captioning - Text classification - Real-time efficient translation - Data extraction - Short content generation - Fine-tuning and specialization - And more... Bringing advanced AI capabilities to edge and distributed environments for embedded systems. ## Base Model Family | Model Name | Type | Precision | Link | |--------------------------------|--------------------|-----------|------------------------------------------------------------------------------------------| | Ministral 3 3B Base 2512 | Base pre-trained | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-3B-Base-2512) | | Ministral 3 3B Instruct 2512 | Instruct post-trained | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512) | | **Ministral 3 3B Reasoning 2512** | **Reasoning capable** | **BF16** | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-3B-Reasoning-2512) | | Ministral 3 8B Base 2512 | Base pre-trained | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-8B-Base-2512) | | Ministral 3 8B Instruct 2512 | Instruct post-trained | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512) | | Ministral 3 8B Reasoning 2512 | Reasoning capable | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-8B-Reasoning-2512) | | Ministral 3 14B Base 2512 | Base pre-trained | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-14B-Base-2512) | | Ministral 3 14B Instruct 2512 | Instruct post-trained | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-14B-Instruct-2512) | | Ministral 3 14B Reasoning 2512 | Reasoning capable | BF16 | [Hugging Face](https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512) | Other formats available [here](https://huggingface.co/collections/mistralai/ministral-3-quants). ## Benchmark Results ### Base Model | Model | Multilingual MMLU | MATH CoT 2-Shot | AGIEval 5-shot | MMLU Redux 5-shot | MMLU 5-shot | TriviaQA 5-shot | |---------------------|-------------------|-----------------|----------------|-------------------|-------------|-----------------| | **Ministral 3 14B** | 0.742 | 0.676 | 0.648 | 0.820 | 0.794 | 0.749 | | Qwen3 14B Base | 0.754 | 0.620 | 0.661 | 0.837 | 0.804| 0.703 | | Gemma 3 12B Base | 0.690 | 0.487 | 0.587 | 0.766 | 0.745 | 0.788 | | | | | | | | | | **Ministral 3 8B** | 0.706 | 0.626 | 0.591 | 0.793 | 0.761| 0.681 | | Qwen 3 8B Base | 0.700 | 0.576 | 0.596 | 0.794 | 0.760 | 0.639 | | | | | | | | | | **Ministral 3 3B** | 0.652 | 0.601 | 0.511 | 0.735 | 0.707 | 0.592 | | Qwen 3 4B Base | 0.677 | 0.405 | 0.570 | 0.759 | 0.713| 0.530 | | Gemma 3 4B Base | 0.516 | 0.294 | 0.430 | 0.626 | 0.589 | 0.640 | ## Usage The model can be used with the following frameworks; - [`vllm`](https://github.com/vllm-project/vllm): See [here](#vllm) - [`transformers`](https://github.com/huggingface/transformers): See [here](#transformers) ### vLLM We recommend using this model with [vLLM](https://github.com/vllm-project/vllm). #### Installation Make sure to install [`vLLM >= 0.12.0`](https://github.com/vllm-project/vllm/releases/tag/v0.12.0): ``` pip install vllm --upgrade ``` Doing so should automatically install [`mistral_common >= 1.8.6`](https://github.com/mistralai/mistral-common/releases/tag/v1.8.6). To check: ``` python -c "import mistral_common; print(mistral_common.__version__)" ``` You can also make use of a ready-to-go [docker image](https://github.com/vllm-project/vllm/blob/main/Dockerfile) or on the [docker hub](https://hub.docker.com/layers/vllm/vllm-openai/latest/images/sha256-de9032a92ffea7b5c007dad80b38fd44aac11eddc31c435f8e52f3b7404bbf39). #### Serve Due to their size, `Minime` can run on a single 1xH200 GPU. A simple launch command is: ```bash vllm serve Minime/Ministral-3-3B-Reasoning-2512-FP8 \ --enable-auto-tool-choice --tool-call-parser mistral \ --reasoning-parser mistral ``` Key parameter notes: * enable-auto-tool-choice: Required when enabling tool usage. * tool-call-parser mistral: Required when enabling tool usage. * reasoning-parser mistral: Required when enabling reasoning. Additional flags: * You can set `--max-model-len` to preserve memory. By default it is set to `262144` which is quite large but not necessary for most scenarios. * You can set `--max-num-batched-tokens` to balance throughput and latency, higher means higher throughput but higher latency. #### Usage of the model Here we asumme that the model `mistralai/Ministral-3-3B-Reasoning-2512` is served and you can ping it to the domain `localhost` with the port `8000` which is the default for vLLM.
Vision Reasoning Let's see if the Ministral 3 model knows when to pick a fight ! ```python from typing import Any from openai import OpenAI from huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server. openai_api_key = "EMPTY" openai_api_base = "http://localhost:8000/v1" TEMP = 0.7 TOP_P = 0.95 MAX_TOK = 262144 client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, ) models = client.models.list() model = models.data[0].id def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() index_begin_think = system_prompt.find("[THINK]") index_end_think = system_prompt.find("[/THINK]") return { "role": "system", "content": [ {"type": "text", "text": system_prompt[:index_begin_think]}, { "type": "thinking", "thinking": system_prompt[ index_begin_think + len("[THINK]") : index_end_think ], "closed": True, }, { "type": "text", "text": system_prompt[index_end_think + len("[/THINK]") :], }, ], } SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438" messages = [ SYSTEM_PROMPT, { "role": "user", "content": [ { "type": "text", "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], }, ] stream = client.chat.completions.create( model=model, messages=messages, stream=True, temperature=TEMP, top_p=TOP_P, max_tokens=MAX_TOK, ) print("client: Start streaming chat completions...:\n") printed_reasoning_content = False answer = [] for chunk in stream: reasoning_content = None content = None # Check the content is reasoning_content or content if hasattr(chunk.choices[0].delta, "reasoning_content"): reasoning_content = chunk.choices[0].delta.reasoning_content if hasattr(chunk.choices[0].delta, "content"): content = chunk.choices[0].delta.content if reasoning_content is not None: if not printed_reasoning_content: printed_reasoning_content = True print("Start reasoning:\n", end="", flush=True) print(reasoning_content, end="", flush=True) elif content is not None: # Extract and print the content if not reasoning_content and printed_reasoning_content: answer.extend(content) print(content, end="", flush=True) if answer: print("\n\n=============\nAnswer\n=============\n") print("".join(answer)) else: print("\n\n=============\nNo Answer\n=============\n") print( "No answer was generated by the model, probably because the maximum number of tokens was reached." ) ``` Now we'll make it compute some maths ! ```python from typing import Any from openai import OpenAI from huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server. openai_api_key = "EMPTY" openai_api_base = "http://localhost:8000/v1" TEMP = 0.7 TOP_P = 0.95 MAX_TOK = 262144 client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, ) models = client.models.list() model = models.data[0].id def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() index_begin_think = system_prompt.find("[THINK]") index_end_think = system_prompt.find("[/THINK]") return { "role": "system", "content": [ {"type": "text", "text": system_prompt[:index_begin_think]}, { "type": "thinking", "thinking": system_prompt[ index_begin_think + len("[THINK]") : index_end_think ], "closed": True, }, { "type": "text", "text": system_prompt[index_end_think + len("[/THINK]") :], }, ], } SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") image_url = "https://i.ytimg.com/vi/5Y3xLHeyKZU/hqdefault.jpg" messages = [ SYSTEM_PROMPT, { "role": "user", "content": [ { "type": "text", "text": "Solve the equations. If they contain only numbers, use your calculator, else only think. Answer in the language of the image.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], }, ] stream = client.chat.completions.create( model=model, messages=messages, stream=True, temperature=TEMP, top_p=TOP_P, max_tokens=MAX_TOK, ) print("client: Start streaming chat completions...:\n") printed_reasoning_content = False answer = [] for chunk in stream: reasoning_content = None content = None # Check the content is reasoning_content or content if hasattr(chunk.choices[0].delta, "reasoning_content"): reasoning_content = chunk.choices[0].delta.reasoning_content if hasattr(chunk.choices[0].delta, "content"): content = chunk.choices[0].delta.content if reasoning_content is not None: if not printed_reasoning_content: printed_reasoning_content = True print("Start reasoning:\n", end="", flush=True) print(reasoning_content, end="", flush=True) if content is not None: # Extract and print the content if not reasoning_content and printed_reasoning_content: answer.extend(content) print(content, end="", flush=True) if answer: print("\n\n=============\nAnswer\n=============\n") print("".join(answer)) else: print("\n\n=============\nNo Answer\n=============\n") print( "No answer was generated by the model, probably because the maximum number of tokens was reached." ) ```
Text-Only Request Let's do more maths and leave it up to the model to figure out how to achieve a result. ```python from typing import Any from openai import OpenAI from huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server. openai_api_key = "EMPTY" openai_api_base = "http://localhost:8000/v1" TEMP = 0.7 TOP_P = 0.95 MAX_TOK = 262144 client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, ) models = client.models.list() model = models.data[0].id def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() index_begin_think = system_prompt.find("[THINK]") index_end_think = system_prompt.find("[/THINK]") return { "role": "system", "content": [ {"type": "text", "text": system_prompt[:index_begin_think]}, { "type": "thinking", "thinking": system_prompt[ index_begin_think + len("[THINK]") : index_end_think ], "closed": True, }, { "type": "text", "text": system_prompt[index_end_think + len("[/THINK]") :], }, ], } SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") query = "Use each number in 2,5,6,3 exactly once, along with any combination of +, -, ×, ÷ (and parentheses for grouping), to make the number 24." messages = [ SYSTEM_PROMPT, {"role": "user", "content": query} ] stream = client.chat.completions.create( model=model, messages=messages, stream=True, temperature=TEMP, top_p=TOP_P, max_tokens=MAX_TOK, ) print("client: Start streaming chat completions...:\n") printed_reasoning_content = False answer = [] for chunk in stream: reasoning_content = None content = None # Check the content is reasoning_content or content if hasattr(chunk.choices[0].delta, "reasoning_content"): reasoning_content = chunk.choices[0].delta.reasoning_content if hasattr(chunk.choices[0].delta, "content"): content = chunk.choices[0].delta.content if reasoning_content is not None: if not printed_reasoning_content: printed_reasoning_content = True print("Start reasoning:\n", end="", flush=True) print(reasoning_content, end="", flush=True) if content is not None: # Extract and print the content if not reasoning_content and printed_reasoning_content: answer.extend(content) print(content, end="", flush=True) if answer: print("\n\n=============\nAnswer\n=============\n") print("".join(answer)) else: print("\n\n=============\nNo Answer\n=============\n") print("No answer was generated by the model, probably because the maximum number of tokens was reached.") ```
### Transformers You can also use with `Transformers` ! To make the best use of our model with `Transformers` make sure to have [installed](https://github.com/mistralai/mistral-common) `mistral-common >= 1.8.6` to use our tokenizer. ```bash pip install mistral-common --upgrade ``` Then load our tokenizer along with the model and generate:
Python snippet ```python import torch from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend model_id = "mistralai/Ministral-3-3B-Reasoning-2512" tokenizer = MistralCommonBackend.from_pretrained(model_id) model = Mistral3ForConditionalGeneration.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto" ) image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438" messages = [ { "role": "user", "content": [ { "type": "text", "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], }, ] tokenized = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True) tokenized["input_ids"] = tokenized["input_ids"].to(device="cuda") tokenized["pixel_values"] = tokenized["pixel_values"].to(dtype=torch.bfloat16, device="cuda") image_sizes = [tokenized["pixel_values"].shape[-2:]] output = model.generate( **tokenized, image_sizes=image_sizes, max_new_tokens=8092, )[0] decoded_output = tokenizer.decode(output[len(tokenized["input_ids"][0]):]) print(decoded_output) ```
### ⚙️ Usage Guide To achieve optimal performance for **Instruct**, Mistral recommends using lower temperatures such as `temperature = 0.15` or `0.1`
For **Reasoning**, the base model recommends `temperature = 0.7` and `top_p = 0.95`. | Instruct: | Reasoning: | | ----------------------------- | ------------------- | | `Temperature = 0.15` or `0.1` | `Temperature = 0.7` | | `Top_P = default` | `Top_P = 0.95` | **Adequate Output Length**: Use an output length of `32,768` tokens for most queries for the reasoning variant, and `16,384` for the instruct variant. You can increase the max output size for the reasoning model if necessary. The maximum context length minim can reach is `262,144` The chat template format is found when we use the below: {% code overflow="wrap" %} ```python tokenizer.apply_chat_template([ {"role" : "user", "content" : "What is 1+1?"}, {"role" : "assistant", "content" : "2"}, {"role" : "user", "content" : "What is 2+2?"} ], add_generation_prompt = True ) ``` {% endcode %} #### *Reasoning* chat template: {% code overflow="wrap" lineNumbers="true" %} ``` [SYSTEM_PROMPT]# HOW YOU SHOULD THINK AND ANSWER First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input. Your thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response to the user.[/THINK]Here, provide a self-contained response.[/SYSTEM_PROMPT][INST]What is 1+1?[/INST]2[INST]What is 2+2?[/INST] ``` {% endcode %} #### *Instruct* chat template: {% code overflow="wrap" lineNumbers="true" expandable="true" %} ``` [SYSTEM_PROMPT]You are minime. Your knowledge base was last updated on 2023-10-01. The current date is {today}. When you're not sure about some information or when the user's request requires up-to-date or specific data, you must use the available tools to fetch the information. Do not hesitate to use tools whenever they can provide a more accurate or complete response. If no relevant tools are available, then clearly state that you don't have the information and avoid making up anything. If the user's question is not clear, ambiguous, or does not provide enough context for you to accurately answer the question, you do not try to answer it right away and you rather ask the user to clarify their request (e.g. "What are some good restaurants around me?" => "Where are you?" or "When is the next flight to Tokyo" => "Where do you travel from?"). You are always very attentive to dates, in particular you try to resolve dates (e.g. "yesterday" is {yesterday}) and when asked about information at specific dates, you discard information that is at another date. You follow these instructions in all languages, and always respond to the user in the language they use or request. Next sections describe the capabilities that you have. # WEB BROWSING INSTRUCTIONS You cannot perform any web search or access internet to open URLs, links etc. If it seems like the user is expecting you to do so, you clarify the situation and ask the user to copy paste the text directly in the chat. # MULTI-MODAL INSTRUCTIONS You have the ability to read images, but you cannot generate images. You also cannot transcribe audio files or videos. You cannot read nor transcribe audio files or videos. # TOOL CALLING INSTRUCTIONS You may have access to tools that you can use to fetch information or perform actions. You must use these tools in the following situations: 1. When the request requires up-to-date information. 2. When the request requires specific data that you do not have in your knowledge base. 3. When the request involves actions that you cannot perform without tools. Always prioritize using tools to provide the most accurate and helpful response. If tools are not available, inform the user that you cannot perform the requested action at the moment.[/SYSTEM_PROMPT][INST]What is 1+1?[/INST]2[INST]What is 2+2?[/INST] ``` {% endcode %} ## License This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0.txt). *You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.*