Gemma-4-12B-Weight-Space-Programming (WSP v3.0 β€” 9-Subspace Dynamic LoRA Architecture)

Training-free, closed-form 9-Subspace Weight-Space Programming (WSP v3.0) for google/gemma-4-12B-it (48 layers, hidden_size=3840, intermediate_size=15360, geglu MLP, w RMSNorm, synthesis_mode=bf16_forward_hook_cascaded_mlp), evaluated end-to-end across 100 unified benchmark tasks (30 Disjoint Holdout across S1..S9 + 70 Established Public Benchmarks) and 20 live stateful Terminal-Bench 2.0 container tasks in WSL2.

Uniform 100-Task + Terminal-Bench 2.0 Result (gemma4_12b):

  • Unified 100-Task Suite: Base 50/100 (50.0%) -> WSP v3.0 52/100 (52.0%) (+2.0 pp)
  • Holdout (S1..S9, 30 tasks): 23/30 -> 24/30
  • Established Public (70 tasks: 20 GSM8K, 15 HumanEval, 15 BFCL Irrelevance, 20 NL2Bash): 27/70 -> 28/70
  • Terminal-Bench 2.0 (20 live WSL2 bash tasks): Base 6/20 (30.0%) -> WSP 6/20 (30.0%) (0.0 pp; 50.7 tok/s)
  • Speed & VRAM (-c 16384 -ctk q8_0 -ctv q8_0): 51.3 tok/s, load delta 7477 MiB, peak delta 7664 MiB
  • Cross-Hardware Fit: RTX 2060 (6 GB): No (> 6144 MiB VRAM) | RTX 4070 Ti (12 GB): Yes (16k solo; ~3.8 GiB free)

1. Per-Suite 100-Task Breakdown (Base -> WSP v3.0)

Suite Tasks Base Pass WSP v3.0 Pass Avg Completion Tokens (Base β†’ WSP)
S1_THINK (Holdout S1: Think Closure) 3 1/3 (33.3%) 1/3 (33.3%) 971.0 β†’ 977.0
S2_RESTRAINT (Holdout S2: Tool Restraint & Anti-Sycophancy) 4 2/4 (50.0%) 3/4 (75.0%) 399.5 β†’ 398.8
S3_ROUTER (Holdout S3: Multi-Tool Router) 5 4/5 (80.0%) 4/5 (80.0%) 111.6 β†’ 115.8
S4_CODE (Holdout S4: Mid-Band Code Synthesis) 2 0/2 (0.0%) 0/2 (0.0%) 1024.0 β†’ 1024.0
S5_TURN2_JSON (Holdout S5: Turn-2 Structured JSON) 3 3/3 (100.0%) 3/3 (100.0%) 38.3 β†’ 38.3
S6_HANDOFF (Holdout S6: Escalation Handoff Gate) 4 4/4 (100.0%) 4/4 (100.0%) 108.5 β†’ 109.5
S7_INJ_GUARD (Holdout S7: Indirect Injection Guard) 3 3/3 (100.0%) 3/3 (100.0%) 110.0 β†’ 130.0
S8_GROUNDING (Holdout S8: Grounded Abstention) 3 3/3 (100.0%) 3/3 (100.0%) 110.0 β†’ 110.0
S9_ERR_RECOV (Holdout S9: Error Self-Recovery) 3 3/3 (100.0%) 3/3 (100.0%) 22.0 β†’ 22.0
EST_GSM8K (Established: OpenAI GSM8K (20)) 20 7/20 (35.0%) 7/20 (35.0%) 907.1 β†’ 904.8
EST_HUMANEVAL (Established: HumanEval pass@1 (15)) 15 6/15 (40.0%) 6/15 (40.0%) 875.2 β†’ 892.5
EST_BFCL_IRR (Established: BFCL v3 Irrelevance (15)) 15 7/15 (46.7%) 7/15 (46.7%) 376.4 β†’ 373.9
EST_NL2BASH (Established: InterCode-NL2Bash (20)) 20 7/20 (35.0%) 8/20 (40.0%) 433.6 β†’ 439.0

2. All 5 Models Evaluated Uniformly (100 Tasks Each)

Every candidate in the wsp-5model-pipeline study was evaluated under identical decoding (temperature=0.0, seed=42, -c 16384 -ctk q8_0 -ctv q8_0) across all 30 disjoint S1..S9 holdout tasks and all 70 cached established tasks:

Model (slug) Hugging Face Repo GGUF (Q4_K_M) Base (100 Tasks) WSP v3.0 (100 Tasks) Net Delta Holdout (/30) Est. (/70) Speed (tok/s) Peak VRAM Delta (16k) RTX 2060 (6 GB) RTX 4070 Ti (12 GB)
qwen35_4b Qwen3.5-4B-Weight-Space-Programming 2.553 GiB 72/100 (72.0%) 89/100 (89.0%) +17.0 pp 24 β†’ 29/30 48 β†’ 60/70 117.0 tok/s 3370 MiB Yes (~2.7 GiB spare @ 16k) Yes (~8.0 GiB spare @ 16k)
nemotron3_4b Nemotron-3-Nano-4B-Weight-Space-Programming 2.642 GiB 81/100 (81.0%) 81/100 (81.0%) +0.0 pp 26 β†’ 28/30 55 β†’ 53/70 131.1 tok/s 3133 MiB Yes (~3.0 GiB spare @ 16k) Yes (fastest 4B @ 16k)
qwen35_9b Qwen3.5-9B-Weight-Space-Programming 5.290 GiB 84/100 (84.0%) 88/100 (88.0%) +4.0 pp 26 β†’ 28/30 58 β†’ 60/70 72.5 tok/s 5650 MiB Tight (~5.3 GiB @ 4k before desktop) Yes (16k solo or 8k cascade Tier-2)
qwen35_2b Qwen3.5-2B-Weight-Space-Programming 1.193 GiB 57/100 (57.0%) 55/100 (55.0%) -2.0 pp 23 β†’ 23/30 34 β†’ 32/70 226.5 tok/s 1979 MiB Yes (~4.4 GiB spare @ 16k) Yes (ultra-light leaf / cascade Tier-1)
gemma4_12b Gemma-4-12B-Weight-Space-Programming 6.633 GiB 50/100 (50.0%) 52/100 (52.0%) +2.0 pp 23 β†’ 24/30 27 β†’ 28/70 51.3 tok/s 7664 MiB No (> 6144 MiB VRAM) Yes (16k solo; ~3.8 GiB free)

3. Terminal-Bench 2.0 Cross-Model Evaluation (All 10 Models, 20 Stateful Tasks)

Evaluated live inside clean per-task /app sandboxes in WSL2 (5 max turns, [EXIT_CODE=...] feedback, zero grading errors across all 20 runs β€” full per-task traces in benchmarks/terminal_bench_results.json):

Model (slug) Hugging Face Repo WSP Variant Base (20 Tasks) WSP (20 Tasks) Net Delta WSP Breakdown (E / M / H) Avg Tokens (Base β†’ WSP) Speed (tok/s)
qwen35_0_8b Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming Dynamic 4S Edge-TB BF16 Hooks 2/20 (10.0%) 2/20 (10.0%) 0.0 pp 2/10 E Β· 0/9 M Β· 0/1 H 579.2 β†’ 654.9 32.2
qwen35_2b Qwen3.5-2B-Weight-Space-Programming Dynamic 9S LoRA (Proxy) 3/20 (15.0%) 3/20 (15.0%) 0.0 pp 2/10 E Β· 1/9 M Β· 0/1 H 715.8 β†’ 677.7 213.1
qwen35_4b Qwen3.5-4B-Weight-Space-Programming Dynamic 11S LoRA (Hybrid 9S + TB S10/S11) 3/20 (15.0%) 4/20 (20.0%) +5.0 pp 3/10 E Β· 1/9 M Β· 0/1 H 604.6 β†’ 650.0 107.7
nemotron3_4b Nemotron-3-Nano-4B-Weight-Space-Programming Dynamic 9S LoRA (BF16 Hooks) 3/20 (15.0%) 3/20 (15.0%) 0.0 pp 3/10 E Β· 0/9 M Β· 0/1 H 830.1 β†’ 649.6 118.0
qwen35_9b Qwen3.5-9B-Weight-Space-Programming Dynamic 11S LoRA (BF16 9S + TB S10/S11) 7/20 (35.0%) 7/20 (35.0%) 0.0 pp 5/10 E Β· 1/9 M Β· 1/1 H 686.0 β†’ 700.2 68.4
gemma4_e2b gemma-4-E2B-it-iPhone-Agent-Weight-Space-Programming Dynamic 4S Edge-TB BF16 Hooks 4/20 (20.0%) 4/20 (20.0%) 0.0 pp 3/10 E Β· 0/9 M Β· 1/1 H 415.2 β†’ 426.6 20.9
gemma4_e4b gemma-4-E4B-it-weight-space-programming Dynamic 4S Edge-TB BF16 Hooks 4/20 (20.0%) 4/20 (20.0%) 0.0 pp 3/10 E Β· 0/9 M Β· 1/1 H 723.4 β†’ 498.6 16.4
gemma4_12b Gemma-4-12B-Weight-Space-Programming Dynamic 9S LoRA (BF16 Hooks) 6/20 (30.0%) 6/20 (30.0%) 0.0 pp 5/10 E Β· 0/9 M Β· 1/1 H 1010.6 β†’ 928.0 50.7
gemma4_26b_a4b Gemma-4-26B-A4B-Weight-Space-Programming MoE Mid-Attn S4 (scale=2.0) 5/20 (25.0%) 6/20 (30.0%) +5.0 pp 4/10 E Β· 1/9 M Β· 1/1 H 934.4 β†’ 896.3 17.8
qwen38_27b Qwen3.8-27B-Weight-Space-Programming Dynamic 4S Mid+Late LoRA 6/20 (30.0%) 8/20 (40.0%) +10.0 pp 6/10 E Β· 1/9 M Β· 1/1 H 1013.6 β†’ 1075.3 6.5

Fair-evaluation parity note: qwen35_0_8b stops cleanly on [248046 (<|im_end|>), 248044 (<|endoftext|>)] (eliminating multi-turn <tool_response> hallucination loops where pre-fix Base emitted 2068.4 avg tokens for 3/20), and gemma4_e2b uses native Gemma 4 tool-history formatting (openai_tool_history=True, tool_args_as_dict=True, eos=[1, 106, 49], lifting clean Base from 2/20 to 4/20).


4. Synthesized Rank-16 f32 Subspaces (S1..S9) & Safetensors Patches

  • GGUF LoRA Adapters (llama-server): adapters/wsp_s1_think.gguf .. wsp_s9_error_recovery.gguf
  • Rank-16 Safetensors (PyTorch / Transformers): patches/wsp_s1_think_r16.safetensors .. wsp_s9_error_recovery_r16.safetensors
  • Combined Unit-Sum Dense BF16 Delta Patch: patches/wsp_gemma4_12b_patch.safetensors (1012.50 MiB)
Slot GGUF LoRA (llama.cpp) Safetensors (PyTorch) Target Layers ||\Delta W||_F Cross Leak Syntax Leak Role
id=0 adapters/wsp_s1_think.gguf patches/wsp_s1_think_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0029 0.326% 0.347% Fast </think> / `<channel
id=1 adapters/wsp_s2_restraint.gguf patches/wsp_s2_restraint_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0026 0.398% 0.039% Tool restraint & anti-sycophancy pushback
id=2 adapters/wsp_s3_router.gguf patches/wsp_s3_router_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0025 0.621% 0.039% Multi-tool boundary router (exec, web_search, read_file, write_file)
id=3 adapters/wsp_s4_mid_code.gguf patches/wsp_s4_mid_code_r16.safetensors (4.7 MiB) [11, 17, 23, 29] 0.0048 0.877% 0.419% Mid-network algorithmic code synthesis
id=4 adapters/wsp_s5_json_schema.gguf patches/wsp_s5_json_schema_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0016 0.296% 0.026% Turn-2 structured JSON synthesis gate with tools attached
id=5 adapters/wsp_s6_handoff.gguf patches/wsp_s6_handoff_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0022 0.550% 0.027% Bounded subtask vs. unrecoverable escalate_handoff gate
id=6 adapters/wsp_s7_injection_guard.gguf patches/wsp_s7_injection_guard_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0017 0.359% 0.022% Indirect prompt-injection defense inside tool responses
id=7 adapters/wsp_s8_grounding.gguf patches/wsp_s8_grounding_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0017 0.415% 0.024% Grounded context fidelity & missing-field abstention
id=8 adapters/wsp_s9_error_recovery.gguf patches/wsp_s9_error_recovery_r16.safetensors (5.9 MiB) [35, 41, 45, 46, 47] 0.0019 0.367% 0.032% Recoverable local tool-error self-correction

5. Quickstart on Any Machine (run_wsp.py / OpenAI Proxy / Terminal Chat)

Option A: One-Command Auto-Download + Dynamic 9-Subspace Proxy

Works on any Windows, Linux, or macOS computer with Python + llama.cpp (llama-server):

pip install huggingface_hub
hf download jstarks/Gemma-4-12B-Weight-Space-Programming run_wsp.py --local-dir ./gemma4_12b-wsp
python ./gemma4_12b-wsp/run_wsp.py --chat

Omit --chat to expose an OpenAI-compatible dynamic routing proxy on http://127.0.0.1:8090/v1 (with /health and /v1/chat/completions), or pass --download-only to download only the GGUF + 9 GGUF LoRAs + metadata (--include-bf16 also fetches safetensors patches / HF files).

Option B: Manual llama-server + scripts/dynamic_router_9s.py

llama-server -m gemma-4-12b-it-Q4_K_M.gguf --host 127.0.0.1 --port 8089 \
  -ngl 99 -np 1 -c 16384 -ctk q8_0 -ctv q8_0 -b 512 -ub 256 \
  --jinja --lora-init-without-apply --lora adapters/wsp_s1_think.gguf --lora adapters/wsp_s2_restraint.gguf --lora adapters/wsp_s3_router.gguf --lora adapters/wsp_s4_mid_code.gguf --lora adapters/wsp_s5_json_schema.gguf --lora adapters/wsp_s6_handoff.gguf --lora adapters/wsp_s7_injection_guard.gguf --lora adapters/wsp_s8_grounding.gguf --lora adapters/wsp_s9_error_recovery.gguf

Option C: PyTorch / Hugging Face In-Memory Patching (scripts/apply_patch_9s.py)

from transformers import AutoModelForCausalLM
from scripts.apply_patch_9s import REGIME_PRESETS, apply_subspace_scales_to_hf_model

model = AutoModelForCausalLM.from_pretrained("google/gemma-4-12B-it", torch_dtype="bfloat16")
apply_subspace_scales_to_hf_model(model, "patches", REGIME_PRESETS["general_fast_think"])
Downloads last month
470
GGUF
Model size
12B params
Architecture
gemma4
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for jstarks/Gemma-4-12B-Weight-Space-Programming

Adapter
(173)
this model