Instructions to use gghfez/Mistral-Small-24B-Base-2501 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gghfez/Mistral-Small-24B-Base-2501 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gghfez/Mistral-Small-24B-Base-2501")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("gghfez/Mistral-Small-24B-Base-2501")
model = AutoModelForCausalLM.from_pretrained("gghfez/Mistral-Small-24B-Base-2501")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use gghfez/Mistral-Small-24B-Base-2501 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gghfez/Mistral-Small-24B-Base-2501"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gghfez/Mistral-Small-24B-Base-2501",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/gghfez/Mistral-Small-24B-Base-2501

SGLang

How to use gghfez/Mistral-Small-24B-Base-2501 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gghfez/Mistral-Small-24B-Base-2501" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gghfez/Mistral-Small-24B-Base-2501",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gghfez/Mistral-Small-24B-Base-2501" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gghfez/Mistral-Small-24B-Base-2501",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use gghfez/Mistral-Small-24B-Base-2501 with Docker Model Runner:
```
docker model run hf.co/gghfez/Mistral-Small-24B-Base-2501
```

Re-Uploaded without consolidated weights

Model Card for Mistral-Small-24B-Base-2501

Mistral Small 3 ( 2501 ) sets a new benchmark in the "small" Large Language Models category below 70B, boasting 24B parameters and achieving state-of-the-art capabilities comparable to larger models!
Check out our fine-tuned Instruct version Mistral-Small-24B-Instruct-2501.

For enterprises that need specialized capabilities (increased context, particular modalities, domain specific knowledge, etc.), we will be releasing commercial models beyond what Mistral AI contributes to the community.

This release demonstrates our commitment to open source, serving as a strong base model.

Learn more about Mistral Small in our blog post.

Model developper: Mistral AI Team

Key Features

Multilingual: Supports dozens of languages, including English, French, German, Spanish, Italian, Chinese, Japanese, Korean, Portuguese, Dutch, and Polish.
Advanced Reasoning: State-of-the-art conversational and reasoning capabilities.
Apache 2.0 License: Open license allowing usage and modification for both commercial and non-commercial purposes.
Context Window: A 32k context window.
Tokenizer: Utilizes a Tekken tokenizer with a 131k vocabulary size.

Benchmark Results

Benchmark	Metric	Mistral-Small-24B-Base
MMLU	5-shot	80.73
MMLU Pro	5-shot, CoT	54.37
GPQA Main	5-shot, CoT	34.37
TriviaQA	5-shot	80.32
ARC-c	0-shot	91.29
TriviaQA	5-shot	76.6
MBPP	pass@1	69.64
GSM8K	5-shot, maj@1	80.73
MATH	4-shot, MaJ	45.98
AGIEval	-	65.80

Benchmark	Metric	Mistral-Small-24B-Base
French MMLU	-	78.03
German MMLU	-	77.69
Spanish MMLU	-	78.86
Russian MMLU	-	75.64
Chinese MMLU	-	70.35
Korean MMLU	-	56.42
Japanese MMLU	-	74.46

Downloads last month: 2

Safetensors

Model size

24B params

Tensor type

BF16

Collection including gghfez/Mistral-Small-24B-Base-2501

unconsolidated uploads

Collection

Re-Uploads without "consolidated" or "original" weights • 2 items • Updated Mar 17

Papers for gghfez/Mistral-Small-24B-Base-2501

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark

Paper • 2406.01574 • Published Jun 3, 2024 • 54

GPQA: A Graduate-Level Google-Proof Q&A Benchmark

Paper • 2311.12022 • Published Nov 20, 2023 • 36

AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

Paper • 2304.06364 • Published Apr 13, 2023 • 3

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Paper • 2206.04615 • Published Jun 9, 2022 • 6

ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection

Paper • 2203.09509 • Published Mar 17, 2022 • 2