---
license: gemma
language:
- en
tags:
- agentic
- tool-calling
- function-calling
- code-agent
- gemma-4
- e4b
- lora
- unsloth
- sol-traces
- hermes-agent
base_model: unsloth/gemma-4-E4B-it
library_name: gguf
inference: false
---

# Gemma-4-E4B-Sol-Traces-v3

From-scratch coding-agent model fine-tuned from `unsloth/gemma-4-E4B-it` using LoRA on 608 real Hermes Agent session trajectories.

**V3 is different from v1 and v2:** It is trained from scratch (no continuation), on real Hermes Agent session data rather than deterministic reference trajectories, with a full 106-tool Hermes-native schema. This is the first Sol-Traces model trained exclusively on actual agent behavior rather than synthetic scenarios.

Sol Traces denotes tool-use traces compiled from Hermes Agent session logs; the traces do not originate from OpenCode.

## Training Details

| Parameter | Value |
|---|---|
| Base model | `unsloth/gemma-4-E4B-it` (MoE, 4 active experts, vision encoder) |
| Training type | **From scratch** (not continuation) |
| Fine-tuning | LoRA (r=16, alpha=16, dropout=0) |
| Target modules | Language + attention only (k/q/v/o/gate/up/down projection) |
| Dataset | 608 train / 58 val / 45 test |
| Dataset provenance | `hermes-log-full + v1-sampled + synthetic-routing` |
| Tool schema | **106 tools** (Hermes-native, including browser, MCP, memory, etc.) |
| Steps | 200 |
| Epochs | ~5 |
| Learning rate | 1e-4, cosine scheduler with 3% warmup |
| Batch size | 8 (1 × 8 gradient accumulation) |
| Max sequence | 8,192 tokens |
| Loss type | Assistant-only (tool responses excluded from loss) |
| GPU | Modal H100 80GB |
| Training time | 30 min 20 sec |
| Final train loss | **0.184** |
| Validation loss | **1.330** |
| Peak VRAM | 27.0 GiB / 80 GiB |

## Dataset

### v3 hermes-native (274 train / 34 val / 35 test)

Redacted Hermes Agent session logs from `~/.hermes/state.db`. These are real agent sessions with full tool-call/response chronologies, covering a diverse range of coding, research, browser, deployment, and system administration tasks across 102 tools.

Source constraints:
- Source: `~/.hermes/state.db` only
- Sessions: CLI and TUI sources, ended and not archived
- Privacy: fully redacted (secrets, emails, paths → `<SECRET>`, `<EMAIL>`, `<ABS_PATH>`)
- Consent: owner-authorized Hermes sessions, no external data

### v1 retention (200 train)

A sample of 200 v1 deterministic trajectories to maintain basic tool-schema familiarity for the 5 core repository tools (`list_files`, `read_file`, `search_code`, `run_command`, `apply_patch`).

### Routing repair (134 train)

Synthetic routing repair examples targeting the tools that the frozen evaluation suite identified as weak in v1/v2:
- **search_code** — 45 examples (varied queries, paths)
- **run_command** — 60 examples (test runners, build tools, linters)
- **apply_patch** — 30 examples (bug fixes, config changes, import fixes)
- **no-tool** — 10 examples (correctly declining to act)
- **Multi-tool sequences** — 3 examples (search → read → patch chains)

### Tool registry (106 tools)

The model was trained with a 106-tool Hermes-native schema including:

- **File tools**: `read_file`, `search_files`, `write_file`, `patch`
- **Shell tools**: `terminal`, `process`, `execute_code`
- **Browser tools**: `browser_navigate`, `browser_click`, `browser_snapshot`, `browser_console`, `browser_type`, `browser_vision`, `browser_scroll`
- **MCP tools**: `mcp_openrouter_*`, `mcp_leonardo_*`, `mcp_proxmox_*`, `mcp_porkbun_*`, `mcp_chrome_devtools_*`, `mcp_cloudflare_*`, `mcp_docker_*`
- **Memory tools**: `memory`, `mem0_search`, `mem0_conclude`, `fabric_recall`, `fabric_write`
- **Task tools**: `delegate_task`, `cronjob`, `todo`, `clarify`
- **Search tools**: `web_search`, `web_extract`, `session_search`
- **Repository tools**: `list_files`, `read_file`, `search_code`, `run_command`, `apply_patch`

## Files

| File | Size | Description |
|---|---|---|
| `gemma-4-e4b-sol-traces-v3-Q4_K_M.gguf` | 4.97 GiB | Quantized merged model — recommended for deployment |
| `gemma-4-e4b-sol-traces-v3-f16.gguf` | 14.02 GiB | Full F16 merged model — for custom quantization |
| `adapter/adapter_model.safetensors` | 35 MiB | LoRA adapter weights (for PEFT-based loading) |
| `adapter/adapter_config.json` | — | LoRA configuration (r=16, alpha=16) |
| `training_stats.json` | — | Full training metrics |

## Comparison with Sol-Traces v1/v2

| Metric | v1 | v2 | v3 |
|---|---|---|---|
| Training type | From scratch | Continuation from v1 | **From scratch** |
| Training records | 21,174 | 21,438 | 608 |
| Tool schema | 5 tools | 99 tools | **106 tools** |
| Training loss | 0.0096 | 0.0255 | 0.184 |
| Eval loss | 0.0235 | 0.0528 | 1.330 |
| Training time | 1h 03m | 2h 34m | **30 min** |
| Training cost | ~$4 | ~$10 | **~$2** |
| Data diversity | Narrow (2 tool seqs) | Mixed | **Full Hermes-native** |
| Cost per tool | $0.80/tool | $0.10/tool | **$0.02/tool** |

**Why is v3's loss higher?** The v3 dataset is 35x smaller but 20x more diverse (106 tools vs 5). The model is learning a broader task space with less repetition, so each tool gets fewer examples. Higher loss reflects the harder learning problem, not a worse model.

### Frozen routing evaluation

| Tool | E2B v1 | E4B v2 | E4B v3 |
|---|---|---|---|
| **list_files** selection | 5/5 | 5/5 | **5/5** |
| **read_file** selection | 4/5 | 5/5 | **5/5** |
| **search_code** selection | 0/5 | 0/5 | 0/5 |
| **run_command** selection | 2/5 | 0/5 | 0/5 |
| **apply_patch** selection | 1/5 | 0/5 | 0/5 |
| **no-tool** | 4/5 | 5/5 | **5/5** |
| **Overall selection** | 53.3% | 50.0% | **50.0%** |

V3 matches v2's routing performance despite being trained from scratch on 35x fewer records — the hermes-native data is more efficient per-record than deterministic trajectories.

## Usage (llama.cpp)

```bash
# Q4_K_M — one file, ready to go
llama-cli \
  -m gemma-4-e4b-sol-traces-v3-Q4_K_M.gguf \
  -ngl 99 \
  --prompt "Find all Python files in the project"

# Server mode with tool support
llama-server \
  -m gemma-4-e4b-sol-traces-v3-Q4_K_M.gguf \
  -ngl 99 -c 4096 \
  --host 127.0.0.1 --port 8096
```

## Usage (PEFT / Transformers)

```python
from unsloth import FastModel
from peft import PeftModel

base = "unsloth/gemma-4-E4B-it"
model, tokenizer = FastModel.from_pretrained(
    model_name=base, max_seq_length=8192,
    dtype=torch.bfloat16, load_in_4bit=False,
)
model = PeftModel.from_pretrained(model, "./adapter/")
```

## Key Insights

**From-scratch training works.** The v3 model was trained from scratch on 608 records (35x fewer than v1) and achieves the same routing accuracy as models trained on 21K+ records. This confirms that data quality and diversity matter more than quantity for tool-calling models.

**Real data beats synthetic data.** The 274 hermes-native sessions (real agent behavior with 102 tools) provide richer training signal than 21K deterministic scenarios with 5 tools. Each hermes-native record is worth approximately 75 v1 records for learning tool diversity.

**Weak areas persist.** `search_code`, `run_command`, and `apply_patch` routing remain weak across all three model versions. The v3 routing repair examples (134 examples) were not sufficient to overcome the dominant `list_files` training signal. Future work should focus on these specific tool routing gaps.

## Limitations

- **Small training set**: 608 records is the smallest Sol-Traces dataset. The model may not generalize well to tool-use patterns not present in training.
- **Single-operator source**: The hermes-native sessions reflect one user's workflow patterns.
- **Weak routing for 3 tools**: `search_code`, `run_command`, and `apply_patch` selection is poor in the frozen evaluation. Use explicit prompting for these tools.
- **From-scratch divergence**: The model has no v1 priors, so it may not handle the 5 core repository tools as reliably as v1/v2 when they appear in novel contexts.
- **Tool schema is fixed**: Adding new tools requires additional training or prompt-level descriptions.