Instructions to use Sciguy429/MiniMax-M3-MSA-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Sciguy429/MiniMax-M3-MSA-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA
# Run inference directly in the terminal:
llama cli -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA
# Run inference directly in the terminal:
llama cli -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA
# Run inference directly in the terminal:
./llama-cli -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Use Docker

docker model run hf.co/Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

LM Studio
Jan
Ollama
How to use Sciguy429/MiniMax-M3-MSA-GGUF with Ollama:
```
ollama run hf.co/Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA
```

Unsloth Studio

How to use Sciguy429/MiniMax-M3-MSA-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Sciguy429/MiniMax-M3-MSA-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Sciguy429/MiniMax-M3-MSA-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Sciguy429/MiniMax-M3-MSA-GGUF to start chatting

How to use Sciguy429/MiniMax-M3-MSA-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Sciguy429/MiniMax-M3-MSA-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Sciguy429/MiniMax-M3-MSA-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Sciguy429/MiniMax-M3-MSA-GGUF with Docker Model Runner:
```
docker model run hf.co/Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA
```

Lemonade

How to use Sciguy429/MiniMax-M3-MSA-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Sciguy429/MiniMax-M3-MSA-GGUF:F16_MSA

Run and chat with the model

lemonade run user.MiniMax-M3-MSA-GGUF-F16_MSA

List all available models

lemonade list

NOTICE:

This repo is being kept up for archival purposes for now. These quants were used during the development of the llama.cpp PR which added MSA support for MiniMax-M3. They are now outdated and it is recommended to use AesSedai or Unsloth's updated quant mixes. I will likely not be updating these further (even though they should be working still) as MiniMax-M3 dose not survive the 2-bit quant process very well. I would not recommend the use of these low bit quants for real world tasks.

OLD DESCRIPTION:

This is an experimental 2-bit GGUF quant of MiniMax-M3 which supports MSA via this llama.cpp PR: https://github.com/ggml-org/llama.cpp/pull/24908

I will keep these quants (and any more I create) up until the PR merges and the bigger players update there repos to match, then I will likely replace this repo with IK quants once ik_llama catches up.

These are up to date with the PR as of 7-10-2026.

Models:

IF16/IF32?

I am still running tests on this at the moment, but for now. The indexer tensors within the official MiniMax-M3 release are stored as both FP32 and BF16 tensors. Specifically the projection tensors (index_{q,k}_proj) are what use BF16. There is some conflicting information in the PR about what these values should be quantized too. The main description recommends F32, but an example llama-quantize command further down in the chain just uses F16. ~~(Once I get some proper perp testing done I will update this section with any differences I noted. I will leave both model's up regardless!)~~

I have done some testing via runpod on the IF16/32 issue. The repo now has a flat Q8 and Q6_K quant in it if anyone else wants to avoid generating them. I also generated a 32K context Q8 logit dump via the following command (which is also in the repo):

./llama-perplexity --flash-attn on -lv 4 --file /workspace/Models/wiki.test.raw --save-all-logits /workspace/Models/MiniMax-M3-Q8_0-IF32-32768ctx-wiki.test.raw.bin --batch-size 2048 --ubatch-size 2048 -c 32768 --fit on --no-mmap --model /workspace/Models/MiniMax-M3-Q8_0.gguf

(Sadly, runpod ate the command output by randomly crashing the SSH session, so I don't have the final PPL for the Q8 file as I lost the entire command output...)

Regardless, I went on to run a perplexity test on my UD-IQ2_M-IF32/16 quants and did find a slight difference between the two: (Full outputs in the repo.)

IF16:
Mean    KLD:   0.457160 ±   0.002882

IF32:
Mean    KLD:   0.455649 ±   0.002860

The difference is within the error margin, but it is there. So while I can't say with certainty the exact importance of the F32 conversion, I think it is safe to say that it dose have an effect.

UD-IQ2_M_IF16/IF32:

These are clones of Unsloth's UD-IQ2_M mix, with FP16 and FP32 indexer tensors respectively. They behave fairly well compared to the original quant from Unsloth and are likely the highest 'quality' 2-bit quant you can get right now until they update their mix once PR #24098 lands or ik_llama implements it's own support and allows for better 2-bit SOTA quant types.

Mean KLD: 0.488235 ± 0.002577

IQ2_M_IF16:

This is a flat automatic IQ2_M quant directly from llama-quantize without any tensor overrides. I just made this one to compare against the UD quant. NOT RECOMENDED FOR ACTUALL USE!

Mean KLD: 0.490047 ± 0.002621

Downloads last month: 4,481

GGUF

Model size

426B params

Architecture

minimax-m3

Hardware compatibility

2-bit

16-bit

32-bit

View +1 variant

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Sciguy429/MiniMax-M3-MSA-GGUF

Base model

MiniMaxAI/MiniMax-M3

Quantized

(56)

this model