Instructions to use NIM-AI/NIM-3-Coder-30B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use NIM-AI/NIM-3-Coder-30B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M

Use Docker

docker model run hf.co/NIM-AI/NIM-3-Coder-30B:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use NIM-AI/NIM-3-Coder-30B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NIM-AI/NIM-3-Coder-30B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NIM-AI/NIM-3-Coder-30B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NIM-AI/NIM-3-Coder-30B:Q4_K_M

Ollama
How to use NIM-AI/NIM-3-Coder-30B with Ollama:
```
ollama run hf.co/NIM-AI/NIM-3-Coder-30B:Q4_K_M
```
Unsloth Desktop

How to use NIM-AI/NIM-3-Coder-30B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "NIM-AI/NIM-3-Coder-30B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use NIM-AI/NIM-3-Coder-30B with Docker Model Runner:
```
docker model run hf.co/NIM-AI/NIM-3-Coder-30B:Q4_K_M
```

Lemonade

How to use NIM-AI/NIM-3-Coder-30B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull NIM-AI/NIM-3-Coder-30B:Q4_K_M

Run and chat with the model

lemonade run user.NIM-3-Coder-30B-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use NIM-AI/NIM-3-Coder-30B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default NIM-AI/NIM-3-Coder-30B:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use NIM-AI/NIM-3-Coder-30B with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf NIM-AI/NIM-3-Coder-30B:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "NIM-AI/NIM-3-Coder-30B:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

NIM-3-Coder-30B / README.md

sujayanimneth

Update README.md

4f6fd21 verified 2 days ago

preview code

Raw History Blame Contribute Delete

3.92 kB

---
language:
- en
- code
license: apache-2.0
tags:
- code
- reasoning
- gguf
- ollama
- llama-cpp
- code-generation
- instruction-tuned
- nim-ai
- programming
pipeline_tag: text-generation
inference: false
---
# NIM 3 - Coder - 30B

<p align="center">
  <b>Developed by NIM-AI</b><br>
  <i>A Sparse Mixture-of-Experts (MoE) Reasoning Engine for Production Software Engineering</i>
</p>

---

## Overview

**NIM 3 - Coder - 30B** is NIM-AI's high-efficiency coding and reasoning model, packaged in optimized **GGUF format** for fast local execution. Built on a **Sparse Mixture-of-Experts (MoE)** architecture, the model routes token predictions dynamically through dedicated expert networks, activating approximately **3 billion parameters per token** out of its total 30.5-billion-parameter pool.

This provides the architectural reasoning and comprehension of a 30B-class model while preserving the execution speed, low latency, and memory footprint of a lightweight system.

NIM 3 - Coder is trained with an explicit **Chain-of-Thought (CoT)** objective. Prior to synthesizing code, the engine outputs deliberative `<think>...</think>` traces to systematically evaluate:
- Concurrency patterns, async runtimes, and thread safety
- Boundary conditions, resource allocation, and edge cases
- Algorithmic complexity and execution bottlenecks
- Clean, idiomatic implementation patterns across production stacks

---

## Technical Specifications

| Parameter | Specification |
| :--- | :--- |
| **Total Parameters** | 30.5 Billion |
| **Active Parameters / Token** | ~3.0 Billion (Sparse Dynamic Routing) |
| **Format** | GGUF (Ready for Ollama, llama.cpp, and LM Studio) |
| **Context Window** | 4,096 tokens (Native, scalable via RoPE) |
| **Primary Languages** | Rust, Python, Go, TypeScript/JavaScript, C++, SQL, Bash |
| **Specialization** | Systems Architecture, Concurrency, Algorithmic Optimization, Backend Pipelines |

---

## Quickstart Guide

### 1. Run Instantly with Ollama

You can pull and launch NIM 3 - Coder directly using Ollama without manual file downloads:

```bash
ollama run hf.co/NIM-AI/NIM-3-Coder-30B:Q4_K_M
To create a custom local alias in Ollama, create a file named Modelfile:
FROM ./Qwen3-30B-A3B.Q4_K_M.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.6
Then build and launch:
ollama create nim-3-coder -f Modelfile
ollama run nim-3-coder
Run an OpenAI-compatible API server on your local machine or cloud GPU:
./llama-server \
    -m Qwen3-30B-A3B.Q4_K_M.gguf \
    --port 8000 \
    --ctx-size 4096 \
    --n-gpu-layers 99 \
    --host 0.0.0.0
Desktop Inference via LM Studio
Open LM Studio.
Search for NIM-AI/NIM-3-Coder-30B in the search bar.
Download the Q4_K_M file and load it into the local chat interface.
Prompt Template
NIM 3 - Coder utilizes the standard ChatML format. Maintain the system prompt to trigger reasoning blocks:
<|im_start|>system
You are NIM 3 - Coder, a world-class software engineer. Always think step-by-step inside <think>...</think> tags before writing clean, high-performance code.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
<think>
{reasoning}
</think>
{solution}
<|im_end|>
Usage Guidelines & Notes
Thinking Output: The model generates reasoning traces inside <think>...</think>. If integrating with an IDE agent or terminal pipe, ensure your interface supports streaming or extracts the content past </think>.
Hardware Sizing: Because the full 30.5B expert weights reside in memory, running the Q4_K_M binary requires ~19 GB of memory headroom (GPU VRAM or shared unified memory on Apple Silicon).
Organization & Contact
Engineered and maintained by NIM-AI. For enterprise configurations, updates, and ecosystem tooling, visit the NIM-AI Organization Hub.

	---
	language:
	- en
	- code
	license: apache-2.0
	tags:
	- code
	- reasoning
	- gguf
	- ollama
	- llama-cpp
	- code-generation
	- instruction-tuned
	- nim-ai
	- programming
	pipeline_tag: text-generation
	inference: false
	---

	# NIM 3 - Coder - 30B

	<p align="center">
	<b>Developed by NIM-AI</b><br>
	<i>A Sparse Mixture-of-Experts (MoE) Reasoning Engine for Production Software Engineering</i>
	</p>

	---

	## Overview

	NIM 3 - Coder - 30B is NIM-AI's high-efficiency coding and reasoning model, packaged in optimized GGUF format for fast local execution. Built on a Sparse Mixture-of-Experts (MoE) architecture, the model routes token predictions dynamically through dedicated expert networks, activating approximately 3 billion parameters per token out of its total 30.5-billion-parameter pool.

	This provides the architectural reasoning and comprehension of a 30B-class model while preserving the execution speed, low latency, and memory footprint of a lightweight system.

	NIM 3 - Coder is trained with an explicit Chain-of-Thought (CoT) objective. Prior to synthesizing code, the engine outputs deliberative `<think>...</think>` traces to systematically evaluate:
	- Concurrency patterns, async runtimes, and thread safety
	- Boundary conditions, resource allocation, and edge cases
	- Algorithmic complexity and execution bottlenecks
	- Clean, idiomatic implementation patterns across production stacks

	---

	## Technical Specifications

	\| Parameter \| Specification \|
	\| :--- \| :--- \|
	\| Total Parameters \| 30.5 Billion \|
	\| Active Parameters / Token \| ~3.0 Billion (Sparse Dynamic Routing) \|
	\| Format \| GGUF (Ready for Ollama, llama.cpp, and LM Studio) \|
	\| Context Window \| 4,096 tokens (Native, scalable via RoPE) \|
	\| Primary Languages \| Rust, Python, Go, TypeScript/JavaScript, C++, SQL, Bash \|
	\| Specialization \| Systems Architecture, Concurrency, Algorithmic Optimization, Backend Pipelines \|

	---

	## Quickstart Guide

	### 1. Run Instantly with Ollama

	You can pull and launch NIM 3 - Coder directly using Ollama without manual file downloads:

	```bash
	ollama run hf.co/NIM-AI/NIM-3-Coder-30B:Q4_K_M

	To create a custom local alias in Ollama, create a file named Modelfile:
	FROM ./Qwen3-30B-A3B.Q4_K_M.gguf

	TEMPLATE """<\|im_start\|>system
	{{ .System }}<\|im_end\|>
	<\|im_start\|>user
	{{ .Prompt }}<\|im_end\|>
	<\|im_start\|>assistant
	"""

	PARAMETER stop "<\|im_start\|>"
	PARAMETER stop "<\|im_end\|>"
	PARAMETER temperature 0.6

	Then build and launch:
	ollama create nim-3-coder -f Modelfile
	ollama run nim-3-coder

	Run an OpenAI-compatible API server on your local machine or cloud GPU:
	./llama-server \
	-m Qwen3-30B-A3B.Q4_K_M.gguf \
	--port 8000 \
	--ctx-size 4096 \
	--n-gpu-layers 99 \
	--host 0.0.0.0

	Desktop Inference via LM Studio
	Open LM Studio.
	Search for NIM-AI/NIM-3-Coder-30B in the search bar.
	Download the Q4_K_M file and load it into the local chat interface.

	Prompt Template
	NIM 3 - Coder utilizes the standard ChatML format. Maintain the system prompt to trigger reasoning blocks:
	<\|im_start\|>system
	You are NIM 3 - Coder, a world-class software engineer. Always think step-by-step inside <think>...</think> tags before writing clean, high-performance code.<\|im_end\|>
	<\|im_start\|>user
	{prompt}<\|im_end\|>
	<\|im_start\|>assistant
	<think>
	{reasoning}
	</think>

	{solution}
	<\|im_end\|>

	Usage Guidelines & Notes
	Thinking Output: The model generates reasoning traces inside <think>...</think>. If integrating with an IDE agent or terminal pipe, ensure your interface supports streaming or extracts the content past </think>.
	Hardware Sizing: Because the full 30.5B expert weights reside in memory, running the Q4_K_M binary requires ~19 GB of memory headroom (GPU VRAM or shared unified memory on Apple Silicon).
	Organization & Contact
	Engineered and maintained by NIM-AI. For enterprise configurations, updates, and ecosystem tooling, visit the NIM-AI Organization Hub.