--- license: apache-2.0 license_link: https://ai.google.dev/gemma/apache_2 base_model: - google/gemma-4-E2B-it base_model_relation: quantized library_name: gguf pipeline_tag: text-generation language: - en tags: - gguf - gemma4 - pruning - shell --- # Bonsai ShellAI A compressed, text-only GGUF derivative of Google DeepMind's [Gemma 4 E2B instruction-tuned model](https://huggingface.co/google/gemma-4-E2B-it/tree/3e22461f65e89153144f8adb70e3b8c2cc9845a7), prepared for ShellAI command suggestions. The model generates text containing shell-command suggestions; the GGUF itself does not execute commands. ## Artifact `bonsai-shellai-Q4_K_S-MLP10-PLE_Q2_K-VOCAB_Q2_K.gguf` - Size: **1,926,811,424 bytes** (1.927 GB; 1.794 GiB). - SHA256: `d10317a1da0965d365a232500603863761f7ff76e1e04924f691ee080203049b`. - Parent revision: `3e22461f65e89153144f8adb70e3b8c2cc9845a7`. ## Download ```sh hf download micrictor/bonsai-shellai bonsai-shellai-Q4_K_S-MLP10-PLE_Q2_K-VOCAB_Q2_K.gguf --local-dir ./bonsai-shellai ``` The GGUF contains the tokenizer and model weights. Use a Gemma 4-compatible llama.cpp runtime; the tested runtime revision is recorded below. Configure the application to accept a single JSON Markdown fence as described in the evaluation. ## Modifications and calibration The nominal 10% MLP-channel cut is **8.333% after alignment**: base FFN width 6144 becomes 5632, while the final 20 shared-KV layers retain twice that width. Depth, residual width, attention/KV sharing and vocabulary are preserved. The separate per-layer embedding (PLE) table and the tied main input/output vocabulary table are Q2_K. The body uses a calibrated Q4_K_S mixed-precision recipe, including eight Q5_K matrices and floating-point exceptions; this is not an all-Q2 model or a uniform four-bit model. Activation-guided selection uses 95 verified training contexts. A fresh importance matrix was collected on the pruned BF16 model using those training contexts, processing 16,384 tokens in eight chunks. All 275 quantized body matrices passed coverage and dimension checks. Main and PLE lookup tables have no importance-matrix entries and use unweighted quantization; no lookup statistics were fabricated. Export starts from BF16, without requantizing a quantized model. No recovery training, SFT, RL or distillation was performed. Conversion/quantization used llama.cpp revision `5f55650a78f92aff4d48d671423e888fac0469ff`. Use a compatible Gemma 4 GGUF runtime and the native Gemma chat template. Runtime versions differ in architecture support; the artifact has been tested with that pinned implementation. ## Measured behavior and limitations On a **reused, inspected 24-case development panel** of executable shell-command fixtures, this model passed **14/24** when an unchanged normalization rule trims outer whitespace and unwraps one complete JSON Markdown fence. Those are exactly the same 14 passing cases as the full calibrated Q4 control. This is a narrow matched comparison, not a held-out benchmark or proof of general equivalence. Under the strict JSON interface this model passed **5/24**, versus **14/24** for the control. Fence-tolerant results do not represent strict-parser performance; the normalizer does not repair command content or extract commands from prose. Generated commands were checked in isolated fixtures during evaluation. That evaluation does not make arbitrary suggested commands safe to execute. CPU evaluation used two threads, context 2048, at most 64 generated tokens, no repacking and swap allowed. Five timing prompts had median response wall time **6.579 s**, and median prompt plus decode **5.216 s**. Sampled peak benchmark RSS was **1.971 GiB (2.117 GB)** with zero sampled process swap. RSS samples are lower bounds affected by paging and background load; they are not a physical-fit guarantee or GPU VRAM measurements. The model does **not** demonstrate 1.5 GB/GiB inference memory. No Raspberry Pi performance or GPU VRAM threshold has been validated. Review suggestions before execution and validate the actual parser and fresh acceptance cases before use. ## License and attribution The pinned upstream model card identifies Gemma 4 as Apache-2.0 and links to [Google's official Apache 2.0 license page](https://ai.google.dev/gemma/apache_2). This distribution includes [LICENSE](LICENSE), [NOTICE](NOTICE) and a [public artifact manifest](manifest.json). This is a modified derivative, not an official Google DeepMind release. Google/Google DeepMind attribution identifies the upstream source and does not imply endorsement.