Instructions to use SurendraVB/Mahiru-MoE-CyberSec with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SurendraVB/Mahiru-MoE-CyberSec with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Use Docker

docker model run hf.co/SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

LM Studio
Jan
Ollama
How to use SurendraVB/Mahiru-MoE-CyberSec with Ollama:
```
ollama run hf.co/SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M
```

Unsloth Studio

How to use SurendraVB/Mahiru-MoE-CyberSec with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SurendraVB/Mahiru-MoE-CyberSec to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SurendraVB/Mahiru-MoE-CyberSec to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for SurendraVB/Mahiru-MoE-CyberSec to start chatting

How to use SurendraVB/Mahiru-MoE-CyberSec with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use SurendraVB/Mahiru-MoE-CyberSec with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use SurendraVB/Mahiru-MoE-CyberSec with Docker Model Runner:
```
docker model run hf.co/SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M
```

Lemonade

How to use SurendraVB/Mahiru-MoE-CyberSec with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Run and chat with the model

lemonade run user.Mahiru-MoE-CyberSec-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use SurendraVB/Mahiru-MoE-CyberSec with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SurendraVB/Mahiru-MoE-CyberSec:Q4_K_M

Run Hermes

hermes

Atomic Chat

Mahiru-MoE-CyberSec / MMLU_benchmark.md

SurendraVB

Upload MMLU_benchmark.md with huggingface_hub

ec7d0b9 verified about 2 months ago

preview code

Raw

History Blame Contribute Delete

8.59 kB

This report covers evaluations on two quantization formats. The first section documents the Q4 Quantized Model (Q4_K_M) benchmark results across 5 core STEM subjects (780 questions total). The second section (at the bottom of this document) details the expanded Q8 Quantized Model (Q8_0) benchmark results across 11 subjects (2,598 questions total).

Sovereign Mahiru MoE: MMLU STEM and CS Benchmark Report

This document reports the performance of the Sovereign Mahiru (3B) MoE model on the full MMLU STEM and Computer Science test suite. The evaluation was run on a Kaggle Tesla P100 GPU using the Q4_K_M GGUF model format.

Out of a total of 780 questions evaluated across the test suite, the model answered 484 questions correctly, achieving an overall accuracy score of 62.05%. No context or generation truncation occurred during this run.

MMLU Subject-Wise Performance

Subject Area	Correct Answers	Total Questions	Subject Accuracy	Evaluation Rating
Elementary Mathematics	296	378	78.31%	High Proficiency
High School Computer Science	56	100	56.00%	Proficient
College Physics	47	102	46.08%	Basic Reasoning
College Computer Science	44	100	44.00%	Basic Reasoning
College Chemistry	41	100	41.00%	Basic Reasoning
Overall MMLU STEM/CS	484	780	62.05%	Strong Generalist (3B MoE)

Architectural Context and Cybersecurity Specialization

Sovereign Mahiru MoE is designed primarily as a specialized cybersecurity, ethical hacking, and secure code-debugging assistant. Its routing pathways are optimized to select the Sovereign Cyber Expert and logical reasoning models.

Specifically, the model is not trained or optimized for non-security academic fields. During this general evaluation:

No Physics Expert was present in the routing matrix.
No Chemistry Expert was present in the routing matrix.
No General Humanities Expert was present in the routing matrix.

Because of this architectural specialization, the model routes general academic queries through generalist base weights rather than dedicated domain-specific networks, prioritizing cybersecurity recall over general multi-subject memorization.

How the Model Achieved Its Scores

The overall score of 62.05% and the peak score of 78.31% in Elementary Mathematics were achieved through the following mechanisms:

1. Calculative Step-by-Step Reasoning

Instead of relying on memorized answers or database lookups, the model used Chain-of-Thought (CoT) processing. Inside the <think> block, the model:

Identified the variables given in the question.
Wrote out the mathematical equations from base principles.
Executed arithmetic calculations and algebraic simplification step-by-step.
Eliminated incorrect options systematically.

2. High Math Proficiency (78.31%)

The model's strong math score suggests that the core reasoning pathways are highly stable. The step-by-step logic inside the reasoning blocks allowed the model to maintain state and perform correct calculations for multi-step math word problems.

3. Factual Recall Limitations in College-Level Sciences

The lower accuracy in College Physics (46.08%), College Computer Science (44.00%), and College Chemistry (41.00%) highlights the model's factual memory limits. While the reasoning blocks showed correct mathematical steps, the model occasionally selected incorrect options because of:

Lack of specialized university-level facts (e.g., specific chemical constants, organic chemistry reaction properties, or theoretical compiler design rules).
The parameter size constraint of a 3B model, which limits the total volume of academic facts it can store in its weights compared to 8B or 70B models.

Q8 Quantized Model (Q8_0) Performance (Expanded 11-Subject Suite)

The evaluation below represents the expanded evaluation of the Q8_0 Quantized Model (8-bit precision). It was executed across a larger set of 11 subjects (2,598 questions total) in parallel. Due to prompt format sensitivity, the raw extraction script missed many valid answers where the model formulated the correct solution within its <think> block but didn't output a clean final prefix. We report both the Raw Evaluation and the Smart Reparsed results.

Overall Q8_0 Summary:

Total Questions: 2,598
Raw Correct: 725 (27.91% accuracy)
Smart Reparsed Correct: 995 (38.46% accuracy)

Q8_0 Subject-Wise Breakdown:

Subject Area	Correct (Raw)	Correct (Smart)	Total Questions	Smart Accuracy
Computer Security	26	67	99	67.68%
Logical Fallacies	40	79	163	48.47%
Nutrition	74	144	306	47.06%
High School Computer Science	29	46	100	46.00%
High School Chemistry	54	85	193	44.04%
College Chemistry	37	35	100	35.00%
Moral Scenarios	305	312	895	34.86%
College Computer Science	23	34	100	34.00%
Elementary Mathematics	76	124	378	32.80%
High School Physics	44	42	151	27.81%
College Physics	17	27	102	26.47%
Overall MMLU Expanded Suite	725	995	2598	38.46%

Comparison: Q4 (5 Subjects) vs. Q8_0 (Smart Reparsed)

For the 5 subjects evaluated in both runs, here is the comparative accuracy:

Subject Area	Q4 Accuracy	Q8_0 Smart Accuracy
Elementary Mathematics	78.31%	32.80%
High School Computer Science	56.00%	46.00%
College Physics	46.08%	26.47%
College Computer Science	44.00%	34.00%
College Chemistry	41.00%	35.00%

The performance drop in the Q8_0 model compared to the Q4 model is primarily due to chat template format sensitivity. The Q8_0 evaluation was run using a structured user/assistant template wrapping a <think> token. In math and physics, this prompt structure led the model to perform correct calculations within its thinking path but fail to map the final answer to the matching choice character (e.g. outputting the calculated value but listing the wrong option letter).

Performance Degradation Analysis: Why Q8_0 Smart Accuracy is Lower

We conduct a detailed audit of the Q8_0 evaluation transcripts to explain why the Q8 model scores lower than the Q4 model in subjects like Elementary Mathematics:

1. Architectural Context & Domain Expert Absence

Mahiru MoE was constructed by merging three specific parent networks: Soul Base (conversation), Thinker/Strategist (reasoning/math), and Sovereign Cyber (security/coding).

Lack of Specialized Science/Academic Experts: We did not include specialized academic experts (e.g., Physics Expert, Chemistry Expert, or Advanced Mathematics Expert).
Routing Limitations: When evaluated on university-level chemistry or physics, the routing gating mechanism is forced to forward tokens to either the general Thinker/Strategist or the Cyber Expert. Since these experts lack specialized university textbooks in their pre-training and alignment sets, the model must rely on the residual capacity of the 3B base, degrading recall.

2. Quantization and Parameter Bottlenecks

Factual Loss: Even at Q8_0 (8-bit quantization), compressing the model's weights strips away the highly precise floating-point calibrations needed for absolute factual recall.
Memory Space Constraints: A 3B parameter model is highly constrained in its storage capacity. Compressing these weights causes a noticeable degradation in rare, high-entropy academic facts.

3. Chat Template and Chain-of-Thought Formatting Sensitivity (The "Think Block Trap")

Successful Logic, Failed Mapping: During the Q8 run, the model used a strict Llama-3-Instruct template and a <think> token wrapper. Transcripts show that the model frequently computed the correct mathematical calculation inside its <think> block, but then mapped it to the incorrect multiple-choice letter (e.g. calculating the answer 8, but outputting Answer: A instead of Answer: C because it was confused by the prompt formatting instructions or few-shot example structure).
Format Degradation: This highlights that the model's reasoning capabilities are highly stable, but its performance on standard multiple-choice metrics is heavily affected by prompt alignment.