Instructions to use OrchardPair/Mistral-Small-24B-Instruct-2501-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use OrchardPair/Mistral-Small-24B-Instruct-2501-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("OrchardPair/Mistral-Small-24B-Instruct-2501-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use OrchardPair/Mistral-Small-24B-Instruct-2501-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "OrchardPair/Mistral-Small-24B-Instruct-2501-4bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "OrchardPair/Mistral-Small-24B-Instruct-2501-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OrchardPair/Mistral-Small-24B-Instruct-2501-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }'
Mistral Small 24B Instruct 2501 โ 4-bit MLX (OrchardPair canonical mirror)
A byte-faithful mirror of mlx-community/Mistral-Small-24B-Instruct-2501-4bit,
re-hosted under the OrchardPair namespace as a revision-pinned canonical source for the OrchardPair app's
on-device Apple-silicon MLX inference. No weights, tokenizer, or configuration were modified โ the files here are
bit-for-bit identical to the upstream quantization (verify via the sha256 manifest below).
Provenance & license
- Upstream quant:
mlx-community/Mistral-Small-24B-Instruct-2501-4bit(4-bit MLX quantization). - Base model: Mistral Small 24B Instruct 2501 (Mistral AI).
- License: Apache-2.0 โ inherited unchanged from upstream. All upstream attribution and terms apply.
- Why mirrored: OrchardPair pins an immutable revision + per-file
sha256so the app downloads a verified, stable artifact that cannot drift if the upstreammainbranch changes.
File manifest (sha256)
| File | Size (bytes) | sha256 |
|---|---|---|
config.json |
825 | 8b4413498badd74aee46afc4071633533c0749883024394afb4cc1c96b6c80f1 |
tokenizer.json |
17078037 | b76085f9923309d873994d444989f7eb6ec074b06f25b58f1e8d7b7741070949 |
tokenizer_config.json |
198489 | 19b35bf8e9a0d4c8c48e4a5773963e6b9872ae3b65c1f44e8152ce54fb48c1ab |
model-00001-of-00003.safetensors |
5285171707 | 4c93bf99bd4f467c55f77eb9555685cf2c00bce50e8b7d22f3b300bc6327b005 |
model-00002-of-00003.safetensors |
5314706961 | 1d83f155ea967330209f8eeb226cdb4468d932e9b9cb3579f674474f52c0b777 |
model-00003-of-00003.safetensors |
2660299513 | 570ee17d38facfcf6fc8000c7b3b6e71ecbaee2180c976a107d6cc69129fcdf8 |
model.safetensors.index.json |
79327 | 42180bd2e96e647bf8528c861f5c25adfbf8dfa87e12fe6e7662df4beeafe5f0 |
- Downloads last month
- 203
Model size
4B params
Tensor type
F16
ยท
U32 ยท
Hardware compatibility
Log In to add your hardware
4-bit
Model tree for OrchardPair/Mistral-Small-24B-Instruct-2501-4bit
Base model
mistralai/Mistral-Small-24B-Base-2501