gemma4-e4b-biohub-mlx

Gemma 4 E4B, LoRA fine-tuned for tool calling inside biohub — a Python package that lets researchers run bioinformatics pipelines on a SLURM cluster (ARC, University of Calgary) by describing what they want in plain English.

Quantized to 4-bit MLX for local inference on Apple Silicon. The point is that the model runs on the researcher's own laptop: no API key, no hosted service, no data leaving the user's own SSH session.

Intended use

This model is a conversational shell for a fixed tool surface, not a general assistant. It fills in parameters and narrates results. The biohub package owns every decision that matters:

• Pipeline routing is a deterministic file-glob in Python. The model never classifies inputs.
• SLURM batch scripts come from tested templates. The model never writes sbatch text.
• Tool calls are schema-validated before dispatch; malformed calls are rejected and returned to the model with a reason.

Used outside that harness, it will underperform the base model on general tasks. Use google/gemma-4-e4b-it for those.

Tool-call output format — read this before integrating

This model does not emit the standard <tool_call>{...}</tool_call> wrapper. It emits:

<|tool_call>call:TOOL_NAME{{"arg": "value"}}<tool_call|>

Note the two differences: the delimiters are <|tool_call> … <tool_call|>, and the JSON argument object carries one redundant enclosing brace pair.

This is a property of the fine-tune, not a bug. Parse it as-is rather than trying to correct it. biohub's parser (biohub/llm.py::_parse_tool_call) handles this form and keeps the standard form as a fallback.

Any prose the model emits alongside a call should be preserved — it is often the only explanation the user gets for why a tool ran.

Tools it was trained against

inspect_inputs · propose_pipeline · submit_pipeline · job_status · fetch_log · diagnose · check_outputs · cancel · arc · email_notify

The pipeline these tools drive is Snyder-Institute/Griffin-Pipeline — Nanopore basecalling, demultiplexing and QC, executed on the cluster via SLURM. The model submits it; it does not run inside the model's process.

Arguments are addressed either by an explicit path (input_dir, output_dir, directory) or by a run name (run_id). The training data uses the path form throughout; the package accepts both.

Training

Base model	google/gemma-4-e4b-it
Method	LoRA (mlx_lm.lora)
Rank / dropout / scale	16 / 0.05 / 20.0
Layers adapted	16
Iterations	1000
Learning rate	1e-4
Batch size	4
Max sequence length	2048
Gradient checkpointing	on
Hardware	Mac Studio M3 Ultra

Training data was 1000 synthetic multi-turn conversations (900 train / 100 validation / 100 test), generated from curated seed examples covering happy-path submission, missing-parameter clarification, ambiguous input, and failure narration.

After training, the adapter was fused into the base model (mlx_lm.fuse) and the result quantized with mlx_lm.convert -q --q-bits 4.

Quantization

4-bit affine, group size 64. 3.9 GB on disk.

Usage

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("TheBioHub/gemma4-e4b-biohub-mlx")

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "basecall run42, kit SQK-RBK114-96"}],
    tools=TOOL_SCHEMAS,          # OpenAI-style function schemas
    tokenize=False,
    add_generation_prompt=True,
)
out = generate(model, tokenizer, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=0.1))

mlx_lm.generate() has no temp keyword — pass a sampler, as above.

Requires Apple Silicon. There is no GGUF build: llama.cpp does not yet support Gemma4ForConditionalGeneration.

Limitations

• Apple Silicon only.
• Trained for one bundled pipeline (Griffin-Pipeline, Oxford Nanopore basecalling and QC) and one cluster's conventions. Other pipelines are recognised by the router but not bundled.
• Cluster paths, partition names and SLURM behaviour in the training data are specific to ARC at the University of Calgary.
• Quantization to 4 bits costs some accuracy relative to the merged model.
• It is a 4B-class model doing structured slot-filling. It is reliable because the surrounding package constrains it, not on its own.

License

Apache 2.0, inherited from google/gemma-4-e4b-it. See the Gemma 4 license.