Instructions to use AesSedai/MiniMax-M3-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AesSedai/MiniMax-M3-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AesSedai/MiniMax-M3-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use AesSedai/MiniMax-M3-GGUF with Ollama:
```
ollama run hf.co/AesSedai/MiniMax-M3-GGUF:Q4_K_M
```

Unsloth Studio

How to use AesSedai/MiniMax-M3-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiniMax-M3-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiniMax-M3-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AesSedai/MiniMax-M3-GGUF to start chatting

How to use AesSedai/MiniMax-M3-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AesSedai/MiniMax-M3-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AesSedai/MiniMax-M3-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AesSedai/MiniMax-M3-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use AesSedai/MiniMax-M3-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AesSedai/MiniMax-M3-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AesSedai/MiniMax-M3-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use AesSedai/MiniMax-M3-GGUF with Docker Model Runner:
```
docker model run hf.co/AesSedai/MiniMax-M3-GGUF:Q4_K_M
```

Lemonade

How to use AesSedai/MiniMax-M3-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AesSedai/MiniMax-M3-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MiniMax-M3-GGUF-Q4_K_M

List all available models

lemonade list

Failed to initialize samplers: failed to parse grammar.

by Exostyle81 - opened 5 days ago

Discussion

Exostyle81

5 days ago

llama-server.exe -m F:\IQ3_S\MiniMax-M3-IQ3_S-00001-of-00005.gguf --ctx-size 100000 --threads 16 --host 127.0.0.1 --no-mmap --jinja --fit on --flash-attn on --device CUDA0,Vulkan1,Vulkan2 -lv 4 --parallel 1

[34m12.07.334.676[0m [31mE failed to parse grammar
[0m[34m12.07.336.405[0m [31mE srv send_error: task id = 0, error: Failed to initialize samplers: failed to parse grammar
[0m[34m12.07.336.415[0m [31mE srv process_sing: failed to launch slot with task, id_task = 0
[0m[34m12.07.336.439[0m [32mI [0msrv update_slots: all slots are idle
[34m12.07.336.625[0m [35mW srv stop: cancel task, id_task = 0
[0m[34m12.07.336.652[0m [32mI [0msrv update_slots: all slots are idle
[34m12.38.117.117[0m [32mI [0msrv server_strea: conv_id=43014fd5-8f16-4bc2-9c4b-d006ea8cb42b (empty=0)
[34m12.38.124.004[0m [32mI [0msrv operator(): chat format: peg-native
[34m12.38.124.949[0m [32mI [0mslot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
[34m12.38.124.955[0m [32mI [0msrv get_availabl: updating prompt cache
[34m12.38.124.961[0m [32mI [0msrv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
[34m12.38.124.964[0m [32mI [0msrv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 100096 tokens, 8589934592 est)
[34m12.38.124.966[0m [32mI [0msrv get_availabl: prompt cache update took 0.01 ms
parse: error parsing grammar: expecting newline or end at ::= [^"\\x7F\x00-\x1F] | [\] (["\bfnrt] | "u" [0-9a-fA-F]{4})
decimal-part ::= [0-9]{1,16}
integral-part ::= [0] | [1-9] [0-9]{0,15}
json-array ::= "[" space ("]" | json-value (space "," space json-value)* space "]")
json-bool ::= "true" | "false"
json-null ::= "null"
json-number ::= "-"? ("0" | [1-9] [0-9]) ("." [0-9]+)? (("e" | "E") [+-]? [0-9]+)?
json-object ::= "{" space ("}" | json-string space ":" space json-value (space "," space json-string space ":" space json-value) space "}")
json-string ::= """ ( [^"\] | "\" ( ["\/ bfnrt] | "u" [0-9a-fA-F]{4} ) )* """
json-value ::= json-object | json-array | json-string | json-number | json-bool | json-null
number ::= ("-"? integral-part) ("." decimal-part)? ([eE] [-+]? integral-part)?
root ::= tool-call | tool-calls
space ::= | " " | "\n"{1,2} [ \t]{0,20}
string ::= """ char* """
tool-call ::= "[\n]" space (tool-web-search-exa | tool-web-fetch-exa) space "" (space "[\n]" space (tool-web-search-exa | tool-web-fetch-exa) space "")* space
tool-calls ::= "<]minimax[>" space tool-call space
tool-web-fetch-exa ::= ("<]minimax[>[<invoke name="" "web_fetch_exa" "">]") space tool-web-fetch-exa-arg-urls (space (tool-web-fetch-exa-arg-maxCharacters))* space
tool-web-fetch-exa-arg-maxCharacters ::= ("[<" "maxCharacters" ">") "" number ""
tool-web-fetch-exa-arg-urls ::= ("[<" "urls" ">") "" tool-web-fetch-exa-arg-urls-schema ""
tool-web-fetch-exa-arg-urls-schema ::= "[" space (string ("," space string))? space "]"
tool-web-search-exa ::= ("<]minimax[>[<invoke name="" "web_search_exa" "">]") space tool-web-search-exa-arg-query (space (tool-web-search-exa-arg-numResults)) space
tool-web-search-exa-arg-numResults ::= ("[<" "numResults" ">") "" number ""
tool-web-search-exa-arg-query ::= ("[<" "query" ">") "" ac-33
until-13 ::= |
until-suffix ::= until-13

ac-33 ::= |
char ::= [^"\\x7F\x00-\x1F] | [\] (["\bfnrt] | "u" [0-9a-fA-F]{4})
decimal-part ::= [0-9]{1,16}
integral-part ::= [0] | [1-9] [0-9]{0,15}
json-array ::= "[" space ("]" | json-value (space "," space json-value)* space "]")
json-bool ::= "true" | "false"
json-null ::= "null"
json-number ::= "-"? ("0" | [1-9] [0-9]) ("." [0-9]+)? (("e" | "E") [+-]? [0-9]+)?
json-object ::= "{" space ("}" | json-string space ":" space json-value (space "," space json-string space ":" space json-value) space "}")
json-string ::= """ ( [^"\] | "\" ( ["\/ bfnrt] | "u" [0-9a-fA-F]{4} ) )* """
json-value ::= json-object | json-array | json-string | json-number | json-bool | json-null
number ::= ("-"? integral-part) ("." decimal-part)? ([eE] [-+]? integral-part)?
root ::= tool-call | tool-calls
space ::= | " " | "\n"{1,2} [ \t]{0,20}
string ::= """ char* """
tool-call ::= "[\n]" space (tool-web-search-exa | tool-web-fetch-exa) space "" (space "[\n]" space (tool-web-search-exa | tool-web-fetch-exa) space "")* space
tool-calls ::= "<]minimax[>" space tool-call space
tool-web-fetch-exa ::= ("<]minimax[>[<invoke name="" "web_fetch_exa" "">]") space tool-web-fetch-exa-arg-urls (space (tool-web-fetch-exa-arg-maxCharacters))* space
tool-web-fetch-exa-arg-maxCharacters ::= ("[<" "maxCharacters" ">") "" number ""
tool-web-fetch-exa-arg-urls ::= ("[<" "urls" ">") "" tool-web-fetch-exa-arg-urls-schema ""
tool-web-fetch-exa-arg-urls-schema ::= "[" space (string ("," space string))? space "]"
tool-web-search-exa ::= ("<]minimax[>[<invoke name="" "web_search_exa" "">]") space tool-web-search-exa-arg-query (space (tool-web-search-exa-arg-numResults)) space
tool-web-search-exa-arg-numResults ::= ("[<" "numResults" ">") "" number ""
tool-web-search-exa-arg-query ::= ("[<" "query" ">") "" ac-33
until-13 ::= |
until-suffix ::= until-13

[34m12.38.126.127[0m [31mE failed to parse grammar
[0m[34m12.38.126.150[0m [31mE srv send_error: task id = 2, error: Failed to initialize samplers: failed to parse grammar
[0m[34m12.38.126.157[0m [31mE srv process_sing: failed to launch slot with task, id_task = 2
[0m[34m12.38.126.168[0m [35mW srv stop: cancel task, id_task = 2
[0m[34m12.38.126.170[0m [32mI [0msrv update_slots: all slots are idle
[34m12.38.126.177[0m [32mI [0msrv update_slots: all slots are idle
[34m13.26.581.394[0m [32mI [0msrv server_strea: conv_id=a1053ac5-67ff-447a-8a91-f70a331cea68 (empty=0)
[34m13.26.588.451[0m [32mI [0msrv operator(): chat format: peg-native
[34m13.26.589.384[0m [32mI [0mslot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
[34m13.26.589.390[0m [32mI [0msrv get_availabl: updating prompt cache
[34m13.26.589.395[0m [32mI [0msrv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
[34m13.26.589.401[0m [32mI [0msrv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 100096 tokens, 8589934592 est)
[34m13.26.589.403[0m [32mI [0msrv get_availabl: prompt cache update took 0.01 ms
parse: error parsing grammar: expecting newline or end at ::= [^"\\x7F\x00-\x1F] | [\] (["\bfnrt] | "u" [0-9a-fA-F]{4})
decimal-part ::= [0-9]{1,16}
integral-part ::= [0] | [1-9] [0-9]{0,15}
json-array ::= "[" space ("]" | json-value (space "," space json-value)* space "]")
json-bool ::= "true" | "false"
json-null ::= "null"
json-number ::= "-"? ("0" | [1-9] [0-9]) ("." [0-9]+)? (("e" | "E") [+-]? [0-9]+)?
json-object ::= "{" space ("}" | json-string space ":" space json-value (space "," space json-string space ":" space json-value) space "}")
json-string ::= """ ( [^"\] | "\" ( ["\/ bfnrt] | "u" [0-9a-fA-F]{4} ) )* """
json-value ::= json-object | json-array | json-string | json-number | json-bool | json-null
number ::= ("-"? integral-part) ("." decimal-part)? ([eE] [-+]? integral-part)?
root ::= tool-call | tool-calls
space ::= | " " | "\n"{1,2} [ \t]{0,20}
string ::= """ char* """
tool-call ::= "[\n]" space (tool-web-search-exa | tool-web-fetch-exa) space "" (space "[\n]" space (tool-web-search-exa | tool-web-fetch-exa) space "")* space
tool-calls ::= "<]minimax[>" space tool-call space
tool-web-fetch-exa ::= ("<]minimax[>[<invoke name="" "web_fetch_exa" "">]") space tool-web-fetch-exa-arg-urls (space (tool-web-fetch-exa-arg-maxCharacters))* space
tool-web-fetch-exa-arg-maxCharacters ::= ("[<" "maxCharacters" ">") "" number ""
tool-web-fetch-exa-arg-urls ::= ("[<" "urls" ">") "" tool-web-fetch-exa-arg-urls-schema ""
tool-web-fetch-exa-arg-urls-schema ::= "[" space (string ("," space string))? space "]"
tool-web-search-exa ::= ("<]minimax[>[<invoke name="" "web_search_exa" "">]") space tool-web-search-exa-arg-query (space (tool-web-search-exa-arg-numResults)) space
tool-web-search-exa-arg-numResults ::= ("[<" "numResults" ">") "" number ""
tool-web-search-exa-arg-query ::= ("[<" "query" ">") "" ac-33
until-13 ::= |
until-suffix ::= until-13

[34m13.26.592.687[0m [31mE failed to parse grammar
[0m[34m13.26.592.709[0m [31mE srv send_error: task id = 4, error: Failed to initialize samplers: failed to parse grammar
[0m[34m13.26.592.714[0m [31mE srv process_sing: failed to launch slot with task, id_task = 4
[0m[34m13.26.592.724[0m [35mW srv stop: cancel task, id_task = 4
[0m[34m13.26.592.726[0m [32mI [0msrv update_slots: all slots are idle
[34m13.26.592.734[0m [32mI [0msrv update_slots: all slots are idle

TheUnsavory

5 days ago

I'm getting the same errors.

heapmaster

5 days ago

Me too...https://github.com/ggml-org/llama.cpp/pull/24908#issuecomment-5095095497

AesSedai

Owner 5 days ago

I'll poke a couple of the parser maintainers and see if they've got an idea.

AesSedai

Owner 5 days ago

Brought that error up to @pwilkin / @aldehir , I think they'll look into it (~tomorrow).

MB7977

5 days ago

Weirdly I’m not experiencing this with the Q5_K_M quant. Latest build of llama.cpp.

./llama-server -m ~/models/MinimaxM3/Minimax-M3-Q5_K_M-00001-of-00008.gguf —fit on -t 12 —jinja -c 50000 —keep -1 -lv 4

Tested with mmproj as well and works fine.

AesSedai

Owner 5 days ago

•

edited 5 days ago

It looks like the issue is related to the tools in the chat template, so if you're using just a normal chat completions UI without tools then it's not erroring.

heapmaster

5 days ago

Ah, ya I want to use this model mainly with coding harnesses, so claude code cli, but even opewebui I would imagine will need tool calling for web search, so any way we can get this fixed?

AesSedai

Owner 5 days ago

It's going to have to wait for an upstream fix from alde or pwilkin. The parser in lcpp is the issue so the quants themselves shouldn't need updating I think.

heapmaster

4 days ago

ggerganov fixed the issue in PR #26210

Exostyle81

4 days ago

Testing

Exostyle81

4 days ago

now it works but yeah... very slow VS old implementation. 24tokens/sec VS 59tokens/sec (old unsloth before the real implementation

heapmaster

4 days ago

Ya I noticed it was slower then I anticipated as well.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment