Gemma-4-E4B-Sol-Traces-v1

Agentic tool-use LLM fine-tuned from unsloth/gemma-4-E4B-it using LoRA.
Trained on 25,000 verified coding-agent trajectories — real tool-calling sequences with function definitions, file operations, shell commands, and multi-step recovery patterns.

The E4B is the mid-range model in the Gemma 4 E-family (4 active experts from a larger MoE pool), offering the best quality-to-size trade-off with strong tool-use capability and efficient inference.

Sol Traces captures full coding-agent sessions: how an agent reads code, searches for patterns, runs commands, applies patches, and verifies results. This is not just instruction-following — it's learned tool-use decision making.

Training Details

Parameter	Value
Base model	unsloth/gemma-4-E4B-it (MoE, 4 active experts)
Fine-tuning	LoRA (r=16, alpha=16, dropout=0)
Target modules	Language + attention (k/q/v/o/gate/up/down projection)
Dataset	21,174 train / 1,324 val (gemma-4-native-tools format)
Dataset provenance	original-synthetic — 25,000 verified trajectories from 32,560 attempts
Epochs	1
Learning rate	1e-4, cosine scheduler with 3% warmup
Batch size	8 (1 × 8 gradient accumulation)
Max sequence	8,192 tokens
Loss type	Assistant-only (tool responses excluded from loss)
GPU	Modal H100 80GB
Training time	~1h 03min (pilot 3min + full 1h)
Final train loss	0.00960
Validation loss	0.02347
Peak VRAM	27.0 GiB / 80 GiB
Throughput	6,518 tok/s

Dataset

The training dataset consists of 25,000 executable agent trajectories generated through a deterministic scenario generator using 100 seeds with repository-family-balanced split strategy:

• 21,174 training records
• 1,324 validation records
• 2,502 test records

Each trajectory is a full agent session containing:

• System instruction: Repository coding agent with tool-use guidelines
• User task: A well-scoped coding task across 85+ repository families
• Assistant tool calls: Multi-step function-calling sequences using 5 tools:
  • list_files — glob-based file discovery
  • read_file — line-range file reading
  • search_code — regex code search
  • run_command — allowlisted shell execution
  • apply_patch — unified diff application
• Tool responses: Output, exit codes, truncation markers
• Verification: Post-task validation commands with pass/fail outcomes

Task Types

Type	Description
fix_bug	Fix a known bug with specific line/signature targets
add_feature	Add a new function/module/endpoint
refactor	Restructure code without changing external behavior
optimize	Improve performance, reduce memory, add caching
test	Add or fix tests for existing functionality
security	Fix a vulnerability, add input validation

Languages: Python, TypeScript, Rust, Go, Java, C++, and more.

Data Generation with GPT-5.6-Sol

The training trajectories were generated by GPT-5.6-Sol (OpenAI's flagship coding agent model, May 2026) running as a repository coding agent through the OpenCode CLI agent framework. This "Sol Traces" methodology captures authentic tool-use behavior:

1. Scenario generation: A deterministic seed-based generator creates structured coding tasks with clear success criteria, repository templates, and verification commands
2. Agent execution: GPT-5.6-Sol autonomously works through each task using the full tool set — reading files, searching code, running commands, applying patches, and verifying results
3. Trajectory capture: Every tool call, response, and decision is recorded into a canonical event stream
4. Verification gate: Only trajectories that pass post-task validation (tests pass, bugs fixed, features work) are accepted
5. Format compilation: Verified trajectories are compiled into gemma-4-native-tools format with assistant-only loss masking

Each trajectory represents the execution trace of a state-of-the-art coding agent solving a real repository task — what it searched for, what it read, what command it ran, what patch it applied, and whether it passed verification. The model learns not just what to do but how and when to use each tool, including recovery from errors and dead ends.

Key Statistics

Metric	Value
Generator model	GPT-5.6-Sol (OpenAI)
Attempted seeds	32,560
Accepted trajectories	25,000 (76.8% acceptance rate)
Rejections	5,872 structural duplicates + 316 verification failures
Provenance	original-synthetic
Repository families	85+ across 6+ languages

Files

File	Size	Description
gemma-4-e4b-sol-traces-v1-Q4_K_M.gguf	4.9 GB	Quantized merged model (Q4_K_M) — recommended for deployment
gemma-4-e4b-sol-traces-v1-f16.gguf	13.9 GB	Full bf16 merged model — for custom quantization
training_stats.json	—	Full training and pilot metrics

Note: The Q4_K_M file is the recommended deployment format. The F16 is provided for downstream quantization experiments.

Usage (llama.cpp)

# Q4_K_M — one file, ready to go
llama-cli \
  -m gemma-4-e4b-sol-traces-v1-Q4_K_M.gguf \
  -ngl 99 \
  --prompt "List the files in the repository matching *.py"

# With conversation template
llama-cli \
  -m gemma-4-e4b-sol-traces-v1-Q4_K_M.gguf \
  -ngl 99 \
  --temp 0.2 \
  --chat-template gemma \
  -p "Find all TODO comments in the codebase"

Capabilities

The model excels at:

• Function calling: Selecting and populating the right tool from natural language
• Code navigation: Searching, reading, and listing files to understand codebases
• Shell execution: Running commands with proper flags and paths
• Patch application: Making small, correct code changes via unified diffs
• Multi-step recovery: Handling errors, retrying with different approaches
• Verification: Running tests and validating changes

Comparison with Other Sol-Traces Models

Model	Active Params	Q4 Size	Training Loss	Speed	Best For
E2B	~5B	3.2 GB	0.0229	Fastest	Edge, CPU+GPU hybrid, low-resource
12B Unified	12B	6.8 GB	0.0800	Fast	Balanced performance
E4B (this)	~8B	4.9 GB	0.0096	Fast	Best quality-size trade-off
26B-A4B	~8B*	15.6 GB	0.0113	Moderate	Maximum capability

*E4B and 26B-A4B both activate 4 experts but have different base architectures (dedicated encoder vs unified).

Why E4B?

The E4B achieved the lowest training loss (0.0096) of all four sol-traces models while maintaining a compact 4.9 GB Q4_K_M footprint. It offers:

• ~30% smaller than the 12B model (4.9 GB vs 6.8 GB Q4)
• ~3x smaller than the 26B model (4.9 GB vs 15.6 GB Q4)
• Highest throughput at 6,518 tok/s (1.4x faster than E2B)
• Best loss at 0.0096 (lowest across all four models)

This makes E4B the recommended default for most sol-traces deployments.

Limitations

• Fine-tuned for repository coding agent scenarios — general chat or creative writing may not benefit
• Single-turn trajectories only — no conversational memory across separate turns
• Tool schemas are fixed to the 5 tools in the training set
• Trained on synthetic trajectories — real-world coding patterns may differ

Training Stats

{
  "training_loss": 0.00960,
  "eval_loss": 0.02347,
  "steps": 377,
  "train_tokens": 24,704,714,
  "peak_vram_gib": 27.0,
  "throughput_tok_s": 6518,
  "runtime": "1h 03m"
}

Disclaimer

Use at your own risk. This model is fine-tuned for coding-agent scenarios. The model owner accepts no liability for any damages or losses arising from its use. Users are responsible for compliance with applicable laws and regulations.