GGUF
Safetensors
English
agentic
tool-calling
function-calling
code-agent
gemma-4
e4b
lora
unsloth
sol-traces
hermes-agent
conversational
Instructions to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M # Run inference directly in the terminal: llama cli -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M # Run inference directly in the terminal: llama cli -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Use Docker
docker model run hf.co/RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with Ollama:
ollama run hf.co/RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
- Unsloth Studio
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 to start chatting
- Pi
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with Docker Model Runner:
docker model run hf.co/RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
- Lemonade
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Run and chat with the model
lemonade run user.Gemma-4-E4B-Sol-Traces-v3-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use RedTeamLab/Gemma-4-E4B-Sol-Traces-v3 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default RedTeamLab/Gemma-4-E4B-Sol-Traces-v3:Q4_K_M
Run Hermes
hermes
- Atomic Chat
| license: gemma | |
| language: | |
| - en | |
| tags: | |
| - agentic | |
| - tool-calling | |
| - function-calling | |
| - code-agent | |
| - gemma-4 | |
| - e4b | |
| - lora | |
| - unsloth | |
| - sol-traces | |
| - hermes-agent | |
| base_model: unsloth/gemma-4-E4B-it | |
| library_name: gguf | |
| inference: false | |
| # Gemma-4-E4B-Sol-Traces-v3 | |
| From-scratch coding-agent model fine-tuned from `unsloth/gemma-4-E4B-it` using LoRA on 608 real Hermes Agent session trajectories. | |
| **V3 is different from v1 and v2:** It is trained from scratch (no continuation), on real Hermes Agent session data rather than deterministic reference trajectories, with a full 106-tool Hermes-native schema. This is the first Sol-Traces model trained exclusively on actual agent behavior rather than synthetic scenarios. | |
| Sol Traces denotes tool-use traces compiled from Hermes Agent session logs; the traces do not originate from OpenCode. | |
| ## Training Details | |
| | Parameter | Value | | |
| |---|---| | |
| | Base model | `unsloth/gemma-4-E4B-it` (MoE, 4 active experts, vision encoder) | | |
| | Training type | **From scratch** (not continuation) | | |
| | Fine-tuning | LoRA (r=16, alpha=16, dropout=0) | | |
| | Target modules | Language + attention only (k/q/v/o/gate/up/down projection) | | |
| | Dataset | 608 train / 58 val / 45 test | | |
| | Dataset provenance | `hermes-log-full + v1-sampled + synthetic-routing` | | |
| | Tool schema | **106 tools** (Hermes-native, including browser, MCP, memory, etc.) | | |
| | Steps | 200 | | |
| | Epochs | ~5 | | |
| | Learning rate | 1e-4, cosine scheduler with 3% warmup | | |
| | Batch size | 8 (1 Γ 8 gradient accumulation) | | |
| | Max sequence | 8,192 tokens | | |
| | Loss type | Assistant-only (tool responses excluded from loss) | | |
| | GPU | Modal H100 80GB | | |
| | Training time | 30 min 20 sec | | |
| | Final train loss | **0.184** | | |
| | Validation loss | **1.330** | | |
| | Peak VRAM | 27.0 GiB / 80 GiB | | |
| ## Dataset | |
| ### v3 hermes-native (274 train / 34 val / 35 test) | |
| Redacted Hermes Agent session logs from `~/.hermes/state.db`. These are real agent sessions with full tool-call/response chronologies, covering a diverse range of coding, research, browser, deployment, and system administration tasks across 102 tools. | |
| Source constraints: | |
| - Source: `~/.hermes/state.db` only | |
| - Sessions: CLI and TUI sources, ended and not archived | |
| - Privacy: fully redacted (secrets, emails, paths β `<SECRET>`, `<EMAIL>`, `<ABS_PATH>`) | |
| - Consent: owner-authorized Hermes sessions, no external data | |
| ### v1 retention (200 train) | |
| A sample of 200 v1 deterministic trajectories to maintain basic tool-schema familiarity for the 5 core repository tools (`list_files`, `read_file`, `search_code`, `run_command`, `apply_patch`). | |
| ### Routing repair (134 train) | |
| Synthetic routing repair examples targeting the tools that the frozen evaluation suite identified as weak in v1/v2: | |
| - **search_code** β 45 examples (varied queries, paths) | |
| - **run_command** β 60 examples (test runners, build tools, linters) | |
| - **apply_patch** β 30 examples (bug fixes, config changes, import fixes) | |
| - **no-tool** β 10 examples (correctly declining to act) | |
| - **Multi-tool sequences** β 3 examples (search β read β patch chains) | |
| ### Tool registry (106 tools) | |
| The model was trained with a 106-tool Hermes-native schema including: | |
| - **File tools**: `read_file`, `search_files`, `write_file`, `patch` | |
| - **Shell tools**: `terminal`, `process`, `execute_code` | |
| - **Browser tools**: `browser_navigate`, `browser_click`, `browser_snapshot`, `browser_console`, `browser_type`, `browser_vision`, `browser_scroll` | |
| - **MCP tools**: `mcp_openrouter_*`, `mcp_leonardo_*`, `mcp_proxmox_*`, `mcp_porkbun_*`, `mcp_chrome_devtools_*`, `mcp_cloudflare_*`, `mcp_docker_*` | |
| - **Memory tools**: `memory`, `mem0_search`, `mem0_conclude`, `fabric_recall`, `fabric_write` | |
| - **Task tools**: `delegate_task`, `cronjob`, `todo`, `clarify` | |
| - **Search tools**: `web_search`, `web_extract`, `session_search` | |
| - **Repository tools**: `list_files`, `read_file`, `search_code`, `run_command`, `apply_patch` | |
| ## Files | |
| | File | Size | Description | | |
| |---|---|---| | |
| | `gemma-4-e4b-sol-traces-v3-Q4_K_M.gguf` | 4.97 GiB | Quantized merged model β recommended for deployment | | |
| | `gemma-4-e4b-sol-traces-v3-f16.gguf` | 14.02 GiB | Full F16 merged model β for custom quantization | | |
| | `adapter/adapter_model.safetensors` | 35 MiB | LoRA adapter weights (for PEFT-based loading) | | |
| | `adapter/adapter_config.json` | β | LoRA configuration (r=16, alpha=16) | | |
| | `training_stats.json` | β | Full training metrics | | |
| ## Comparison with Sol-Traces v1/v2 | |
| | Metric | v1 | v2 | v3 | | |
| |---|---|---|---| | |
| | Training type | From scratch | Continuation from v1 | **From scratch** | | |
| | Training records | 21,174 | 21,438 | 608 | | |
| | Tool schema | 5 tools | 99 tools | **106 tools** | | |
| | Training loss | 0.0096 | 0.0255 | 0.184 | | |
| | Eval loss | 0.0235 | 0.0528 | 1.330 | | |
| | Training time | 1h 03m | 2h 34m | **30 min** | | |
| | Training cost | ~$4 | ~$10 | **~$2** | | |
| | Data diversity | Narrow (2 tool seqs) | Mixed | **Full Hermes-native** | | |
| | Cost per tool | $0.80/tool | $0.10/tool | **$0.02/tool** | | |
| **Why is v3's loss higher?** The v3 dataset is 35x smaller but 20x more diverse (106 tools vs 5). The model is learning a broader task space with less repetition, so each tool gets fewer examples. Higher loss reflects the harder learning problem, not a worse model. | |
| ### Frozen routing evaluation | |
| | Tool | E2B v1 | E4B v2 | E4B v3 | | |
| |---|---|---|---| | |
| | **list_files** selection | 5/5 | 5/5 | **5/5** | | |
| | **read_file** selection | 4/5 | 5/5 | **5/5** | | |
| | **search_code** selection | 0/5 | 0/5 | 0/5 | | |
| | **run_command** selection | 2/5 | 0/5 | 0/5 | | |
| | **apply_patch** selection | 1/5 | 0/5 | 0/5 | | |
| | **no-tool** | 4/5 | 5/5 | **5/5** | | |
| | **Overall selection** | 53.3% | 50.0% | **50.0%** | | |
| V3 matches v2's routing performance despite being trained from scratch on 35x fewer records β the hermes-native data is more efficient per-record than deterministic trajectories. | |
| ## Usage (llama.cpp) | |
| ```bash | |
| # Q4_K_M β one file, ready to go | |
| llama-cli \ | |
| -m gemma-4-e4b-sol-traces-v3-Q4_K_M.gguf \ | |
| -ngl 99 \ | |
| --prompt "Find all Python files in the project" | |
| # Server mode with tool support | |
| llama-server \ | |
| -m gemma-4-e4b-sol-traces-v3-Q4_K_M.gguf \ | |
| -ngl 99 -c 4096 \ | |
| --host 127.0.0.1 --port 8096 | |
| ``` | |
| ## Usage (PEFT / Transformers) | |
| ```python | |
| from unsloth import FastModel | |
| from peft import PeftModel | |
| base = "unsloth/gemma-4-E4B-it" | |
| model, tokenizer = FastModel.from_pretrained( | |
| model_name=base, max_seq_length=8192, | |
| dtype=torch.bfloat16, load_in_4bit=False, | |
| ) | |
| model = PeftModel.from_pretrained(model, "./adapter/") | |
| ``` | |
| ## Key Insights | |
| **From-scratch training works.** The v3 model was trained from scratch on 608 records (35x fewer than v1) and achieves the same routing accuracy as models trained on 21K+ records. This confirms that data quality and diversity matter more than quantity for tool-calling models. | |
| **Real data beats synthetic data.** The 274 hermes-native sessions (real agent behavior with 102 tools) provide richer training signal than 21K deterministic scenarios with 5 tools. Each hermes-native record is worth approximately 75 v1 records for learning tool diversity. | |
| **Weak areas persist.** `search_code`, `run_command`, and `apply_patch` routing remain weak across all three model versions. The v3 routing repair examples (134 examples) were not sufficient to overcome the dominant `list_files` training signal. Future work should focus on these specific tool routing gaps. | |
| ## Limitations | |
| - **Small training set**: 608 records is the smallest Sol-Traces dataset. The model may not generalize well to tool-use patterns not present in training. | |
| - **Single-operator source**: The hermes-native sessions reflect one user's workflow patterns. | |
| - **Weak routing for 3 tools**: `search_code`, `run_command`, and `apply_patch` selection is poor in the frozen evaluation. Use explicit prompting for these tools. | |
| - **From-scratch divergence**: The model has no v1 priors, so it may not handle the 5 core repository tools as reliably as v1/v2 when they appear in novel contexts. | |
| - **Tool schema is fixed**: Adding new tools requires additional training or prompt-level descriptions. | |