Instructions to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="morphicode-jp/gemma-4-12b-Q2-tsubo4",
	filename="gemma-4-12B-it-Q2_K-TSUBO4.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K
# Run inference directly in the terminal:
llama cli -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K
# Run inference directly in the terminal:
llama cli -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K
# Run inference directly in the terminal:
./llama-cli -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K
# Run inference directly in the terminal:
./build/bin/llama-cli -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Use Docker

docker model run hf.co/morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

LM Studio
Jan

vLLM

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "morphicode-jp/gemma-4-12b-Q2-tsubo4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "morphicode-jp/gemma-4-12b-Q2-tsubo4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Ollama
How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with Ollama:
```
ollama run hf.co/morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K
```

Unsloth Studio

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for morphicode-jp/gemma-4-12b-Q2-tsubo4 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for morphicode-jp/gemma-4-12b-Q2-tsubo4 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for morphicode-jp/gemma-4-12b-Q2-tsubo4 to start chatting

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with Docker Model Runner:
```
docker model run hf.co/morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K
```

Lemonade

How to use morphicode-jp/gemma-4-12b-Q2-tsubo4 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull morphicode-jp/gemma-4-12b-Q2-tsubo4:Q2_K

Run and chat with the model

lemonade run user.gemma-4-12b-Q2-tsubo4-Q2_K

List all available models

lemonade list

Gemma 4 12B-it Q2_K + L10 ×1.65 (TSUBO-4) — a 4-byte F32 patch

Four bytes. One layer. +8.32pt HellaSwag on the full validation set (n=10,042, z≈12). Training-free, calibration-free, zero inference overhead.

A Q2_K quantized Gemma 4 12B-it — 4.84 GB — with the F32 layer_output_scale of layer 10 multiplied by ×1.65. Exactly 4 bytes of the GGUF change: the F32 value 0.104004 becomes 0.171606 at byte offset 1812031584 of gemma-4-12B-it-Q2_K.gguf from bartowski/gemma-4-12b-it-GGUF. No training, no calibration data, no extra compute at inference.

TL;DR

Reading the GSM8K number: the paired n=50 result is 32% → 40% (+8.0pt), but McNemar p=0.34 — not statistically significant. Our claim for generation is strictly "non-regressive"; we do not claim a GSM8K improvement. The statistically solid results are the ranking benchmarks (HellaSwag, ARC-Challenge).

Ship-gate results, patched vs. Q2_K baseline:

metric	baseline Q2_K	TSUBO-4 (L10 ×1.65)	Δ
HellaSwag (n=10,042 full)	38.70%	47.01%	+8.32pt ⭐ (z≈12)
Winogrande (n=1,267 full)	52.80%	53.59%	+0.79pt
ARC-Challenge (n=1,165)	25.84%	32.02%	+6.18pt (z≈3.3)
GSM8K (paired, n=50)	32%	40%	+8.0pt (McNemar p=0.34 — non-regressive only)

vs. unpatched Q4_K_M

metric	Q2_K baseline (4.84 GB)	TSUBO-4 (4.84 GB)	Q4_K_M baseline (6.79 GB)
HellaSwag	38.70%	47.01%	49.75%
Winogrande	52.80%	53.59%	54.06%
ARC-Challenge	25.84%	32.02%	31.16%
GSM8K (n=50)	32%	40%	54%

The honest story: TSUBO-4 closes ~75% of the Q2→Q4 HellaSwag gap, reaches parity on Winogrande, and exceeds the Q4_K_M baseline on ARC-Challenge (+0.86pt) — at a 29% smaller file (4.84 GB vs 6.79 GB). The exception is generative math: Q4_K_M keeps a clear edge on GSM8K (54% vs 40%, -14pt). If GSM-style generation is your workload, use Q4. If you need the smallest file with the best ranking-task quality, this is it.

Benchmark methodology (read before comparing)

HellaSwag / Winogrande / ARC-C are scored by log-likelihood ranking (the standard definition, as in lm-eval-harness): the model generates zero tokens — no chain-of-thought, no thinking mode. These numbers measure representation quality, not reasoning traces.
GSM8K is generative: step-by-step prompt, and Gemma 4's native thinking mode is active (verified in run logs). This is the benchmark that exercises reasoning — and the one that the needle profile shows is fragile to the patch scale.
All headline claims are paired deltas (patched vs baseline under identical settings), so they remain internally valid regardless of scoring mode. Cross-paper comparisons should match scoring modes.

What is TSUBO?

We call the technique TSUBO, after the Japanese term for acupressure points: isolated loci where a minimal, precisely-placed intervention produces a system-wide response. The metaphor is structural, not medical — the points are located by exhaustive per-layer screening + fine-grained scale search, and their effects are measured on standard benchmarks. The formal term is needle-point gate patching. Retroactively, the paper-v1 8-byte patch (31B L25+L26 ×1.5) is TSUBO-8, the 44-byte basin-B patch is TSUBO-44, and the present 12B patch is TSUBO-4.

name	patch	bytes	status
TSUBO-4 (this repo)	Gemma 4 12B Q2_K, L10 ×1.65	4	🚢 ship gate passed 2026-06-11
TSUBO-8	Gemma 4 31B, L25+L26 ×1.5	8	published 2026-06-07 (paper v1)
TSUBO-44	Gemma 4 31B, basin B, 11 layers	44	published (paper v1)

The patch itself:

tsubo4 = {"layer": 10, "scale": 1.65}
# blk.10.layer_output_scale.weight (F32): 0.104004 -> 0.171606

layer_output_scale is a single F32 scalar per transformer block that gates how much of that block's normalized output is written back to the residual stream. TSUBO-4 multiplies that gate by 1.65 at layer 10, and touches nothing else.

⚠ The needle: do not retune the scale yourself

The scale value is not a tunable knob. A 0.01-step scan across ×1.60–1.70 shows a two-layer structure:

Ranking benchmarks are flat: HellaSwag sits at +8.5 to +10.2pt (n=1,000) across the whole ×1.60–1.70 range.
Generation flickers: GSM8K survival points are isolated, not an interval. n=50-confirmed alive: ×1.62 and ×1.65 (both 32% → 40%, +8.0pt). Dead at the resolution tested (n=10–20): ×1.63, ×1.64, ×1.70. ×1.75 is established broken (0/20 pooled, p<0.01).
Extreme scales collapse: ×16.5 drives HellaSwag to 25.5% (4-way chance level) and ×21.62 to 20.0% (below chance). Absolute amplification governs the response; fractional digits carry no magic.

Practical rule: patch values must be byte-exact. A ±0.01 perturbation is not safe. Use the prebaked GGUF or the scripts below verbatim; do not "round" or "fine-tune" the scale. The GGUF byte-patch distribution format pins the value exactly, which is why this is shippable at all.

How to use

Option A — download the prebaked GGUF

huggingface-cli download morphicode-jp/gemma-4-12b-Q2-tsubo4 \
    --local-dir ./gemma
./llama-cli -m ./gemma/gemma-4-12B-it-Q2_K-TSUBO4.gguf -ngl 99 -c 4096

Option B — apply the 4-byte patch yourself

Start from gemma-4-12B-it-Q2_K.gguf of bartowski/gemma-4-12b-it-GGUF. The minimal offset-based patch (the offset is valid only for that exact file):

import shutil, struct

PATH   = "gemma-4-12B-it-Q2_K.gguf"  # bartowski/gemma-4-12b-it-GGUF
OFFSET = 1812031584                  # blk.10 layer_output_scale (F32) in this exact file
SCALE  = 1.65

shutil.copyfile(PATH, PATH + ".backup")
with open(PATH, "r+b") as f:
    f.seek(OFFSET)
    (old,) = struct.unpack("<f", f.read(4))
    assert abs(old - 0.104004) < 1e-6, f"unexpected value {old:.6f} — wrong file or revision, aborting"
    f.seek(OFFSET)
    f.write(struct.pack("<f", old * SCALE))  # 0.104004 -> 0.171606, exactly 4 bytes

Safer name-based variant (no hardcoded offset; survives re-downloads with different tensor layout):

# pip install gguf numpy
import numpy as np
from gguf import GGUFReader

PATH = "gemma-4-12B-it-Q2_K.gguf"
reader = GGUFReader(PATH, "r+")  # writable memory map
t = next(t for t in reader.tensors if t.name == "blk.10.layer_output_scale.weight")
v = t.data.view(np.float32)
assert abs(float(v[0]) - 0.104004) < 1e-6, "unexpected value — wrong file or revision"
v[0] = np.float32(v[0] * np.float32(1.65))  # stay in F32 — a float64 round-trip would change the bytes
reader.data.flush()

Restore = copy the .backup file back (Option B's first snippet creates it automatically).

Validation

Pre-registered ship gate: HellaSwag Δ≥+5pt at z≥5 on n=10,042; Winogrande and ARC-Challenge non-regressive; paired GSM8K Δ≥-2pt at n=50. TSUBO-4 passed all four (table above).

Honesty notes on the adjudication:

×1.65 was the best of 15 scale-probe points and is therefore post-selected; the adjudication, however, was run on an independent paired n=50 GSM8K set (32% → 40%, p=0.34).
The neighboring confirmed survival point ×1.62 replicated at n=50 (also 32% → 40%, +8.0pt), but only ×1.65 ran the full benchmark gate; ×1.62 ships nowhere.
The model passed an interactive spot check (5 prompts, T=0.7, no degeneration observed) before release — a guard against the known failure mode where T=0 benchmark gates pass while chat quality degrades.

For context, the published 31B results from the same technique: Q2_K HellaSwag +11.21pt (n=10,042) and GSM8K +5.40pt (n=500, McNemar p=0.0007) — see the TSUBO-8 repos below.

Mechanism (hypothesis-generating — not confirmed)

The patch amplifies a per-layer F32 output gate (layer_output_scale), plausibly compensating attenuation introduced by aggressive quantization of the surrounding weights. We state this as a working hypothesis: the mechanism is not confirmed.

What we can rule out: an earlier "rare full-attention layers" story is disproved. Layer 10 on 12B — like the 31B winners L25/L26 — is a sliding-window attention layer, not a full-attention layer. Why these particular layers respond, and why the generation-safe window narrows from a wide plateau on 31B to a needle on 12B, remains open. The patch points were found empirically by exhaustive per-layer screening + fine-grained scale search, not derived from theory.

Files

gemma-4-12B-it-Q2_K-TSUBO4.gguf (4.84 GB) — prebaked patched model
gemma-4-12B-it-Q2_K-TSUBO4.gguf.md5
apply_tsubo4.py — both patch variants above, with --restore
fig1_needle_profile.png, fig2_bench_comparison.png
README.md (this file)
LICENSE

Sister releases (TSUBO-8 family, 31B)

morphicode-jp/gemma-4-31B-it-L25L26x1.5-IQ1_M — 1-bit, TSUBO-8
morphicode-jp/gemma-4-31B-it-L25L26x1.5-Q2_K — 2-bit flagship, TSUBO-8
morphicode-jp/gemma-4-31B-it-L25L26x1.5-Q4_K_M — 4-bit, TSUBO-8

Code and paper:

GitHub: github.com/morphicode-jp/f32-patch-gemma (branch main)
Zenodo (paper + code archive): doi.org/10.5281/zenodo.20362820

Citation

@misc{hirai2026-tsubo,
  title  = {Why Some LLMs Have a Hidden Reasoning Knob: Per-Layer Output-Gate Slack in Hybrid Architectures and an 8-byte Quantization Recovery},
  author = {Hirai, Akito},
  year   = {2026},
  doi    = {10.5281/zenodo.20362820},
  url    = {https://doi.org/10.5281/zenodo.20362820}
}

The TSUBO naming is introduced in the v1.2 deposit of the same record (§5.7).

Limitations & Methodology Notes

GSM8K is non-regressive, not improved: +8.0pt at paired n=50, McNemar p=0.34. Do not quote it as a significant gain.
The scale is a needle, not a knob: confirmed survival points are isolated (×1.62, ×1.65 at n=50); ±0.01 perturbations are unsafe; ×1.75 is established broken (0/20, p<0.01). Apply byte-exactly.
The byte offset 1812031584 is valid only for the exact bartowski gemma-4-12B-it-Q2_K.gguf file; use the name-based variant for anything else, and verify the 0.104004 pre-value either way.
Q4_K_M baseline still wins GSM8K generation by 14pt; pick your quant by workload.
No AdvBench-style safety run exists yet for the 12B patch (the 31B TSUBO-8 release includes one); beyond the 5-prompt interactive spot check, treat alignment behavior as unaudited.
Mechanism unconfirmed (see above); cross-family transfer of the technique is weak (Qwen 3.6 +2.5pt; Phi-4 null).
Scorer caveat: HellaSwag/Winogrande accuracies measured with llama-perplexity --hellaswag mode, systematically 0.2–2.5 pp lower than lm-evaluation-harness standard (llama.cpp discussion #2321). Deltas are comparable within this card; absolute values are not directly comparable to harness leaderboards.

Contact

X (Twitter): @morphicode_jp
GitHub: github.com/morphicode-jp
Zenodo (paper + code archive): doi.org/10.5281/zenodo.20362820

DMs open for research collaboration.

License: Apache 2.0 for the patch tooling. Gemma 4 base weights are licensed under Apache 2.0 (verified 2026-05-31; Gemma 4 was moved off the older Gemma Terms of Use). GGUF quantization by bartowski. The patched-GGUF derivative notice is in LICENSE-WEIGHTS.

Downloads last month: 144

GGUF

Model size

12B params

Architecture

gemma4

Hardware compatibility

2-bit