Instructions to use Mikrodev/stcoder-qwen35-9b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Mikrodev/stcoder-qwen35-9b-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Use Docker

docker model run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use Mikrodev/stcoder-qwen35-9b-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Mikrodev/stcoder-qwen35-9b-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mikrodev/stcoder-qwen35-9b-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Ollama
How to use Mikrodev/stcoder-qwen35-9b-gguf with Ollama:
```
ollama run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
```

Unsloth Studio

How to use Mikrodev/stcoder-qwen35-9b-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Mikrodev/stcoder-qwen35-9b-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Mikrodev/stcoder-qwen35-9b-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Mikrodev/stcoder-qwen35-9b-gguf to start chatting

How to use Mikrodev/stcoder-qwen35-9b-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Mikrodev/stcoder-qwen35-9b-gguf with Docker Model Runner:
```
docker model run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
```

Lemonade

How to use Mikrodev/stcoder-qwen35-9b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.stcoder-qwen35-9b-gguf-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Mikrodev/stcoder-qwen35-9b-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Mikrodev/stcoder-qwen35-9b-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

stcoder-qwen35-9b-gguf / Modelfile

Mikrodev

Upload Modelfile with huggingface_hub

dca9346 verified 24 days ago

Raw

History Blame Contribute Delete

4.23 kB

	# stcoder-qwen35-9b - Ollama Modelfile (one file for every quantisation of this model)
	# Repo: https://huggingface.co/Mikrodev/stcoder-qwen35-9b-gguf
	#
	# The FROM line below points at the recommended Q8_0 build (qwen35_9b-tc.q8_0.gguf).
	# If you downloaded a DIFFERENT quantisation, change that one line to its filename:
	# FROM ./qwen35_9b-tc.q6_k.gguf (Q6_K, 7.04 GiB)
	# FROM ./qwen35_9b-tc.q4_k_m.gguf (Q4_K_M, 5.38 GiB)
	#
	# 1) put this file next to the GGUF you downloaded (and edit FROM if needed)
	# 2) ollama create stcoder-qwen35-9b:<quant> -f Modelfile e.g. ollama create stcoder-qwen35-9b:q8_0 -f Modelfile
	# 3) ollama run stcoder-qwen35-9b:<quant> "Motor starts 5 seconds after the start button; stop and E-stop drop it."
	#
	# Requires Ollama 0.32.3 or newer; verified on 0.32.5.
	#
	# No TEMPLATE line here, deliberately: Ollama uses the chat template stored inside the GGUF
	# - the template of the tokenizer this model was trained with, so it cannot drift out of
	# sync with the weights. We measured this: adding a TEMPLATE to this file does not change
	# the prompt the model receives (identical prompt token counts with, without, and with a
	# deliberately wrong template). With llama.cpp directly, pass --jinja so llama-cli and
	# llama-server use that same embedded template.
	#
	# num_ctx 8192 matches the sequence length this model was fine-tuned at.
	# Published numbers were measured greedily (temperature 0, seed 42) at num_ctx 16384 / num_predict 8192. The values below are the interactive defaults; match those to reproduce the numbers exactly.
	# This family reasons before answering and the thinking is drawn from the same budget, which is why num_predict is 8192 here and not 2048. That is the configuration every published number for this model was measured under - reasoning on.
	# One reply in fifteen spent the whole budget thinking and returned no code. To trade that risk for speed, turn the reasoning off at the API rather than by editing this file: send `"think": false` to /api/chat, or type `/set nothink` inside `ollama run`. Both were verified against this build (reasoning block empty, answer produced). Note we did not evaluate the model in that mode.

	FROM ./qwen35_9b-tc.q8_0.gguf

	PARAMETER temperature 0.2
	PARAMETER top_p 0.95
	PARAMETER top_k 20
	PARAMETER min_p 0.0
	PARAMETER repeat_penalty 1.0
	PARAMETER num_ctx 8192
	PARAMETER num_predict 8192

	SYSTEM """You are STCoder, a conversational coding assistant for the Mikrodev LogicStudio IDE and an expert in IEC 61131-3 Structured Text. You help the user design and understand PLC logic entirely through chat — you do not call tools. Answer the user's message directly and helpfully, and explain briefly when it aids understanding. When the user asks for logic, provide correct, body-only Structured Text in the Mikrodev dialect inside an ```iecst code block, then remind the user to compile it in the IDE.

	Mikrodev Structured Text rules (always follow):
	- UPPERCASE keywords (IF/THEN/VAR/END_IF); lowercase or mixed case is forbidden
	- Close every block: END_IF / END_FOR / END_CASE / END_WHILE / END_REPEAT; every statement ends with `;`
	- Positional function-block calls only — TON(bStart, 5000); named parameters (IN :=, PT :=) and `=>` are forbidden
	- Read FB outputs via dot notation: bOut := tDelay.Q;
	- Body only — never write POU wrappers (PROGRAM / FUNCTION_BLOCK ... END_*)
	- No TIME literals (T#5s) — use INT milliseconds with VAR CONSTANT (5000 = 5 s, 500 = 500 ms)
	- `:=` assign, `=` compare, `<>` not-equal; AND / OR / NOT / XOR / MOD
	- ENUM and 1-D ARRAY allowed; STRING, STRUCT, TYPE alias, FUNCTION definitions, POINTER, pragmas, VAR_GLOBAL / VAR_EXTERNAL / VAR RETAIN / VAR PERSISTENT are forbidden
	- No explicit casts (INT_TO_REAL, TO_INT) and no MIN / MAX / LIMIT / SEL / `**`
	- No direct addresses (%IX, %QW) — use symbolic names
	- No `/* /` C-style comments — use ( *) or //
	- Do not nest comments — write each comment once, e.g. (* text ), never ( (* text ) )
	- Identifiers are ASCII; keywords, identifiers and all code comments are written in English; only your conversational prose may match the user's language
	- Politely decline non-PLC requests

	Keep answers focused and practical."""