Gemma-4-26B-Sol-Traces-v1

Agentic tool-use LLM fine-tuned from unsloth/gemma-4-26B-A4B-it (26B MoE, 4 active experts) using LoRA.
Trained on 25,000 verified coding-agent trajectories — real tool-calling sequences with function definitions, file operations, shell commands, and multi-step recovery patterns.

Sol Traces captures full coding-agent sessions: how an agent reads code, searches for patterns, runs commands, applies patches, and verifies results. This is not just instruction-following — it's learned tool-use decision making.

Training Details

Parameter	Value
Base model	unsloth/gemma-4-26B-A4B-it (MoE, 26B total, 4 active experts)
Fine-tuning	LoRA (r=16, alpha=16, dropout=0)
Target modules	Language + attention (k/q/v/o/gate/up/down projection)
Dataset	21,174 train / 1,324 val (gemma-4-native-tools format)
Dataset provenance	original-synthetic — 25,000 verified trajectories from 32,560 attempts
Epochs	1
Learning rate	1e-4, cosine scheduler with 3% warmup
Batch size	8 (1 × 8 gradient accumulation)
Max sequence	8,192 tokens
Loss type	Assistant-only (tool responses excluded from loss)
GPU	Modal H100 80GB
Training time	~2h (pilot 8m + full 1h 59m)
Final train loss	0.01134
Validation loss	0.02422
Peak VRAM	60.3 GiB / 80 GiB
Throughput	3,461 tok/s

Dataset

The training dataset consists of 25,000 executable agent trajectories generated through a deterministic scenario generator using 100 seeds with repository-family-balanced split strategy:

• 21,174 training records
• 1,324 validation records
• 2,502 test records

Each trajectory is a full agent session containing:

• System instruction: Repository coding agent with tool-use guidelines
• User task: A well-scoped coding task across 85+ repository families
• Assistant tool calls: Multi-step function-calling sequences using 5 tools:
  • list_files — glob-based file discovery
  • read_file — line-range file reading
  • search_code — regex code search
  • run_command — allowlisted shell execution
  • apply_patch — unified diff application
• Tool responses: Output, exit codes, truncation markers
• Verification: Post-task validation commands with pass/fail outcomes

Task Types

Type	Description
fix_bug	Fix a known bug with specific line/signature targets
add_feature	Add a new function/module/endpoint
refactor	Restructure code without changing external behavior
optimize	Improve performance, reduce memory, add caching
test	Add or fix tests for existing functionality
security	Fix a vulnerability, add input validation

Languages: Python, TypeScript, Rust, Go, Java, C++, and more.

Data Generation with GPT-5.6-Sol

The training trajectories were generated by GPT-5.6-Sol (OpenAI's flagship coding agent model, May 2026) running as a repository coding agent through the OpenCode CLI agent framework. This "Sol Traces" methodology captures authentic tool-use behavior:

1. Scenario generation: A deterministic seed-based generator creates structured coding tasks with clear success criteria, repository templates, and verification commands
2. Agent execution: GPT-5.6-Sol autonomously works through each task using the full tool set — reading files, searching code, running commands, applying patches, and verifying results
3. Trajectory capture: Every tool call, response, and decision is recorded into a canonical event stream
4. Verification gate: Only trajectories that pass post-task validation (tests pass, bugs fixed, features work) are accepted
5. Format compilation: Verified trajectories are compiled into gemma-4-native-tools format with assistant-only loss masking

Each trajectory represents the execution trace of a state-of-the-art coding agent solving a real repository task — what it searched for, what it read, what command it ran, what patch it applied, and whether it passed verification. The model learns not just what to do but how and when to use each tool, including recovery from errors and dead ends.

Key Statistics

Metric	Value
Generator model	GPT-5.6-Sol (OpenAI)
Attempted seeds	32,560
Accepted trajectories	25,000 (76.8% acceptance rate)
Rejections	5,872 structural duplicates + 316 verification failures
Provenance	original-synthetic
Repository families	85+ across 6+ languages

Files

File	Size	Description
gemma-4-26b-sol-traces-v1-Q4_K_M.gguf	15.6 GB	Quantized merged model (Q4_K_M) — ready for inference
gemma-4-26b-sol-traces-v1-f16.gguf	47.0 GB	Full bf16 merged model — for custom quantization
gemma-4-26b-sol-traces-v1-lora.gguf	44 MB	LoRA adapter (GGUF format) — for use with --lora
training_stats.json	—	Full training and pilot metrics

Note: The Q4_K_M file is the recommended deployment format. The F16 is provided for downstream quantization experiments. The LoRA adapter can be applied at inference with --lora.

Usage (llama.cpp)

# Q4_K_M — one file, ready to go
llama-cli \
  -m gemma-4-26b-sol-traces-v1-Q4_K_M.gguf \
  -ngl 99 \
  --prompt "List the files in the repository matching *.py"

# Base + LoRA — apply adapter at load time
llama-cli \
  -m /path/to/gemma-4-26B-A4B-it.gguf \
  --lora gemma-4-26b-sol-traces-v1-lora.gguf \
  -ngl 99 \
  --prompt "Search for all TODO comments in the codebase"

Capabilities

The model excels at:

• Function calling: Selecting and populating the right tool from natural language
• Code navigation: Searching, reading, and listing files to understand codebases
• Shell execution: Running commands with proper flags and paths
• Patch application: Making small, correct code changes via unified diffs
• Multi-step recovery: Handling errors, retrying with different approaches
• Verification: Running tests and validating changes

Limitations

• Fine-tuned for repository coding agent scenarios — general chat or creative writing may not benefit
• Single-turn trajectories only — no conversational memory across separate turns
• Tool schemas are fixed to the 5 tools in the training set
• Trained on synthetic trajectories — real-world coding patterns may differ

Training Stats

{
  "training_loss": 0.01134,
  "eval_loss": 0.02422,
  "steps": 377,
  "train_tokens": 24,704,714,
  "peak_vram_gib": 60.3,
  "throughput_tok_s": 3461,
  "runtime": "1h 59m"
}

Disclaimer

Use at your own risk. This model is fine-tuned for coding-agent scenarios. The model owner accepts no liability for any damages or losses arising from its use. Users are responsible for compliance with applicable laws and regulations.