Instructions to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF", filename="qwen3-coder-30b-a3b-codemonkey.Q4_K_M.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Use Docker
docker model run hf.co/1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
- Ollama
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with Ollama:
ollama run hf.co/1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
- Unsloth Studio
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF to start chatting
- Pi
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat new
- OpenClaw new
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with Docker Model Runner:
docker model run hf.co/1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
- Lemonade
How to use 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull 1337Hero/qwen3-coder-30b-a3b-codemonkey-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.qwen3-coder-30b-a3b-codemonkey-GGUF-Q4_K_M
List all available models
lemonade list
Update README.md
Browse filesUpdated to really explain what this is, what I was trying to do, and where I failed.
|
@@ -13,31 +13,98 @@ tags:
|
|
| 13 |
- qwen3moe
|
| 14 |
- quantized
|
| 15 |
- code
|
|
|
|
|
|
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
-
#
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
-
exporting a BF16 GGUF, and quantizing from that BF16 source.
|
| 24 |
|
| 25 |
-
##
|
| 26 |
|
| 27 |
-
|
| 28 |
-
-
|
| 29 |
-
-
|
|
|
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
```bash
|
| 34 |
llama-cli \
|
| 35 |
--model qwen3-coder-30b-a3b-codemonkey.Q4_K_M.gguf \
|
| 36 |
--jinja \
|
| 37 |
-
-p "Write a
|
| 38 |
-
```
|
| 39 |
-
|
| 40 |
-
## Notes
|
| 41 |
-
|
| 42 |
-
- The `Q4_K_M` file in this repo was quantized from the merged BF16 GGUF, not requantized from Q8.
|
| 43 |
-
- If you want the adapter-only release, use the companion adapter repo: `1337Hero/qwen3-coder-30b-a3b-codemonkey`
|
|
|
|
| 13 |
- qwen3moe
|
| 14 |
- quantized
|
| 15 |
- code
|
| 16 |
+
- frontend
|
| 17 |
+
- react
|
| 18 |
+
- fine-tuned
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# Qwen3-CodeMonkey v1
|
| 22 |
|
| 23 |
+
A fine-tuned version of [qwen3-coder-30b-a3b](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct), optimized for frontend development patterns and DHH-inspired simplicity.
|
| 24 |
|
| 25 |
+
**TL;DR:** I made Qwen ***faster* but dumber**. Q4_K_M hits 137 tps (63% faster gen than base) but drops 10 accuracy points. Your mileage may vary.
|
|
|
|
| 26 |
|
| 27 |
+
## π― Motivation
|
| 28 |
|
| 29 |
+
I wanted a model that *thinks* like I do about frontend code:
|
| 30 |
+
- **Simplicity over cleverness** β boring tech that works beats clever abstractions
|
| 31 |
+
- **Convention over configuration** β follow patterns, don't invent new ones
|
| 32 |
+
- **DHH/Ryan Florence-inspired** β majestic monolith vibes for the frontend stack
|
| 33 |
|
| 34 |
+
I work heavily in React/Preact + TanStack Query + Tailwind, and needed a model that could:
|
| 35 |
+
- Write clean, maintainable component code
|
| 36 |
+
- Avoid TypeScript ceremony and over-abstraction
|
| 37 |
+
- Default to proven patterns (no "clever" state management hacks)
|
| 38 |
+
- Generate code that a junior dev can understand 6 months later
|
| 39 |
+
|
| 40 |
+
## π The Numbers (Honest Edition)
|
| 41 |
+
|
| 42 |
+
Benchmarked with [scaffold-bench](https://github.com/1337hero/scaffold-bench)
|
| 43 |
+
My own eval suite that tests real agentic coding work, not trivia or math puzzles. It throws 23 scenarios at the model (surgical edits, audits, scope discipline, verify-and-repair, implementation tasks) with actual tool use and real codebases.
|
| 44 |
+
|
| 45 |
+
**Results (mean of 2 runs, 50-task subset):**
|
| 46 |
+
|
| 47 |
+
| Model | Score | Gen TPS | Prompt TPS | Notes |
|
| 48 |
+
|-------|-------|---------|------------|-------|
|
| 49 |
+
| **qwen3-code-monkey (Q4_K_M)** | **66.0%** (33/50) | **137.3** | **1219** | Fastest, but -10pts accuracy |
|
| 50 |
+
| qwen3-coder-next (base) | 76.0% (38/50) | 84.4 | 953 | Baseline |
|
| 51 |
+
| qwen3-code-monkey-q8 | 33.0% (16.5/50) | 109.6 | 1444 | One run collapsed early β needs investigation |
|
| 52 |
+
|
| 53 |
+
**Translation:** Q4 is **63% faster at generation** but lost 10 accuracy points. If you're latency bound and can tolerate the trade-off, this might be your jam. If you need max accuracy, stick with base.
|
| 54 |
+
|
| 55 |
+
**What scaffold-bench tests:**
|
| 56 |
+
- **No LLM judge** - deterministic code-driven checks only
|
| 57 |
+
- **Real tool use** - read, ls, grep, glob, edit, write, bash
|
| 58 |
+
- **Real scenarios** - surgical edits, audits, scope discipline, implementation tasks against actual codebases
|
| 59 |
+
- Categories include: surgical edit, audit, scope discipline, read only analysis, verify and repair, implementation, responsiveness, long context
|
| 60 |
+
|
| 61 |
+
**Sample categories where code-monkey struggled vs base:**
|
| 62 |
+
- **verify-and-repair** - iterating through failing tests to green
|
| 63 |
+
- **implementation** - multi-file spec-to-code against existing stacks (wanted to rewrite everything my way instead of following the existing patterns)
|
| 64 |
+
- **scope-discipline** - "make this one change and nothing else" (my training data emphasized refactoring, which backfired)
|
| 65 |
+
|
| 66 |
+
**Q8 caveat:** One of the Q8 runs cratered to 8/50 after 64 requests - might be a harness issue, overfit collapse under tighter quantization, or early model failure. Take those numbers with salt until I re-run it.
|
| 67 |
+
|
| 68 |
+
**Full benchmark details:** Check out [scaffold-bench](https://github.com/1337hero/scaffold-bench) if you want to run your own evals. All results/*.json files from my runs are in the codemonkey repo.
|
| 69 |
+
|
| 70 |
+
π οΈ Training Process (And Where I Screwed Up)
|
| 71 |
+
|
| 72 |
+
- Dataset: ~20,000 Q&A pairs pulled from my frontend philosophy docs (DHH's "majestic monolith"
|
| 73 |
+
stuff + Ryan Florence's React patterns)
|
| 74 |
+
- Fine-tuning: LoRA adapter via Unsloth on Qwen3-Coder-30B-A3B-Instruct
|
| 75 |
+
- What I did wrong: Built the whole dataset as plain [question] β [answer] pairs. No reasoning
|
| 76 |
+
shown, no "wait, should I even do this?" step in the middle. Turns out the model learned the shape
|
| 77 |
+
of my answers without learning how I got there.
|
| 78 |
+
- How it showed up: v1 benched at 32β34/50 vs the base model's 37β40/50. Worse than what I started
|
| 79 |
+
with. The failures were the giveaway β it would edit files when I asked read-only questions, jump
|
| 80 |
+
straight to edits without reading anything first, skip searching before acting. Not a knowledge
|
| 81 |
+
problem. A "doesn't stop to think" problem.
|
| 82 |
+
- The speed trap that fooled me: 138 tok/s vs base 84 tok/s looked like a win at first. It wasn't.
|
| 83 |
+
It was faster because it was thinking less. Trinity-mini sits at the end of that road - blazing
|
| 84 |
+
fast, fails everything, useless.
|
| 85 |
+
- What I'm doing for v2: Q&A alone teaches outputs, not process. Need explicit <think>...</think>
|
| 86 |
+
traces in the dataset - short, 2β4 sentences, enough to encode the taboos ("read-only means
|
| 87 |
+
read-only", "search before you edit", "don't re-add a feature that already exists"). Response-only
|
| 88 |
+
supervision. Less distilled CoT cosplay, more structural alignment.
|
| 89 |
+
|
| 90 |
+
**Lessons learned:**
|
| 91 |
+
1. Unsloth works great once you get past the docs
|
| 92 |
+
2. 20k rows might've been overkill for the domain specificity I wanted
|
| 93 |
+
3. Need to balance domain adaptation with general capability retention
|
| 94 |
+
4. Benchmarking early would've saved me a re-train (and $47)
|
| 95 |
+
|
| 96 |
+
## π¦ Files
|
| 97 |
+
|
| 98 |
+
- `qwen3-coder-30b-a3b-codemonkey.Q4_K_M.gguf` β **recommended** release artifact (fastest)
|
| 99 |
+
- `qwen3-coder-30b-a3b-codemonkey.Q8_0.gguf` β larger reference GGUF (see caveat above)
|
| 100 |
+
- `qwen3-coder-30b-a3b-codemonkey.BF16.gguf` β optional full-size BF16 source
|
| 101 |
+
|
| 102 |
+
All quantized from the merged BF16 GGUF (not requantized from Q8).
|
| 103 |
+
|
| 104 |
+
## π Usage with llama.cpp
|
| 105 |
|
| 106 |
```bash
|
| 107 |
llama-cli \
|
| 108 |
--model qwen3-coder-30b-a3b-codemonkey.Q4_K_M.gguf \
|
| 109 |
--jinja \
|
| 110 |
+
-p "Write a React component that fetches user data with TanStack Query and displays it in a card."
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|