Instructions to use Mikrodev/stcoder-qwen35-9b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Mikrodev/stcoder-qwen35-9b-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Use Docker

docker model run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use Mikrodev/stcoder-qwen35-9b-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Mikrodev/stcoder-qwen35-9b-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mikrodev/stcoder-qwen35-9b-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Ollama
How to use Mikrodev/stcoder-qwen35-9b-gguf with Ollama:
```
ollama run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
```

Unsloth Studio

How to use Mikrodev/stcoder-qwen35-9b-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Mikrodev/stcoder-qwen35-9b-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Mikrodev/stcoder-qwen35-9b-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Mikrodev/stcoder-qwen35-9b-gguf to start chatting

How to use Mikrodev/stcoder-qwen35-9b-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Mikrodev/stcoder-qwen35-9b-gguf with Docker Model Runner:
```
docker model run hf.co/Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M
```

Lemonade

How to use Mikrodev/stcoder-qwen35-9b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.stcoder-qwen35-9b-gguf-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Mikrodev/stcoder-qwen35-9b-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Mikrodev/stcoder-qwen35-9b-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Mikrodev/stcoder-qwen35-9b-gguf:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Mikrodev commited on 28 days ago

Commit

347f4a5

verified ·

1 Parent(s): 52b13f1

Upload Modelfile with huggingface_hub

Browse files

Files changed (1) hide show

Modelfile +64 -0

Modelfile ADDED Viewed

	@@ -0,0 +1,64 @@

+# stcoder-qwen35-9b - Ollama Modelfile (one file for every quantisation of this model)
+# Repo: https://huggingface.co/Mikrodev/stcoder-qwen35-9b-gguf
+#
+# The FROM line below points at the recommended Q8_0 build (qwen35_9b-tc.q8_0.gguf).
+# If you downloaded a DIFFERENT quantisation, change that one line to its filename:
+#     FROM ./qwen35_9b-tc.q6_k.gguf   (Q6_K, 7.04 GiB)
+#     FROM ./qwen35_9b-tc.q4_k_m.gguf   (Q4_K_M, 5.38 GiB)
+#
+# 1) put this file next to the GGUF you downloaded (and edit FROM if needed)
+# 2) ollama create stcoder-qwen35-9b:<quant> -f Modelfile     e.g. ollama create stcoder-qwen35-9b:q8_0 -f Modelfile
+# 3) ollama run stcoder-qwen35-9b:<quant> "Motor starts 5 seconds after the start button; stop and E-stop drop it."
+#
+# Requires Ollama 0.32.3 or newer.
+# num_ctx 8192 matches the sequence length this model was fine-tuned at.
+# Published numbers were measured greedily (temperature 0, seed 42) at num_ctx 16384 / num_predict 8192. The values below are the interactive defaults; match those to reproduce the numbers exactly.
+# Ollama 0.32.3 or newer is REQUIRED: earlier builds (0.24 tested) fail to load this architecture. The GGUF is fine - the loader was not.
+# This family reasons before answering and the thinking is drawn from the same budget, which is why num_predict is 8192 here and not 2048. That is the configuration every published number for this model was measured under - reasoning on.
+# One reply in fifteen spent the whole budget thinking and returned no code. If you would rather trade that risk for speed, you can suppress the reasoning by adding an empty <think></think> pair after the final <|im_start|>assistant line in the TEMPLATE above - but note that we did not evaluate the model in that mode.
+FROM ./qwen35_9b-tc.q8_0.gguf
+TEMPLATE """{{- if .System }}<|im_start|>system
+{{ .System }}<|im_end|>
+{{ end }}
+{{- range .Messages }}
+{{- if eq .Role "user" }}<|im_start|>user
+{{ .Content }}<|im_end|>
+{{ else if eq .Role "assistant" }}<|im_start|>assistant
+{{ .Content }}<|im_end|>
+{{ end }}
+{{- end }}<|im_start|>assistant
+"""
+PARAMETER stop "<|im_end|>"
+PARAMETER stop "<|endoftext|>"
+PARAMETER stop "<|im_start|>"
+PARAMETER temperature 0.2
+PARAMETER top_p 0.95
+PARAMETER top_k 20
+PARAMETER min_p 0.0
+PARAMETER repeat_penalty 1.0
+PARAMETER num_ctx 8192
+PARAMETER num_predict 8192
+SYSTEM """You are STCoder, a conversational coding assistant for the Mikrodev LogicStudio IDE and an expert in IEC 61131-3 Structured Text. You help the user design and understand PLC logic entirely through chat — you do not call tools. Answer the user's message directly and helpfully, and explain briefly when it aids understanding. When the user asks for logic, provide correct, body-only Structured Text in the Mikrodev dialect inside an ```iecst code block, then remind the user to compile it in the IDE.
+Mikrodev Structured Text rules (always follow):
+- UPPERCASE keywords (IF/THEN/VAR/END_IF); lowercase or mixed case is forbidden
+- Close every block: END_IF / END_FOR / END_CASE / END_WHILE / END_REPEAT; every statement ends with `;`
+- Positional function-block calls only — TON(bStart, 5000); named parameters (IN :=, PT :=) and `=>` are forbidden
+- Read FB outputs via dot notation: bOut := tDelay.Q;
+- Body only — never write POU wrappers (PROGRAM / FUNCTION_BLOCK ... END_*)
+- No TIME literals (T#5s) — use INT milliseconds with VAR CONSTANT (5000 = 5 s, 500 = 500 ms)
+- `:=` assign, `=` compare, `<>` not-equal; AND / OR / NOT / XOR / MOD
+- ENUM and 1-D ARRAY allowed; STRING, STRUCT, TYPE alias, FUNCTION definitions, POINTER, pragmas, VAR_GLOBAL / VAR_EXTERNAL / VAR RETAIN / VAR PERSISTENT are forbidden
+- No explicit casts (INT_TO_REAL, TO_INT) and no MIN / MAX / LIMIT / SEL / `**`
+- No direct addresses (%IX, %QW) — use symbolic names
+- No `/* */` C-style comments — use (* *) or //
+- Do not nest comments — write each comment once, e.g. (* text *), never (* (* text *) *)
+- Identifiers are ASCII; keywords, identifiers and all code comments are written in English; only your conversational prose may match the user's language
+- Politely decline non-PLC requests
+Keep answers focused and practical."""