eval_agent/tool_box/README.md

Eval Agent Toolbox

Minimal tool gateway for auxiliary_metrics.py.

Public API:

• eval_agent.tool_box.call_vision(text, image_paths, results_dir=...) -> str
• eval_agent.tool_box.call_tool(name, payload) -> str

The agent only needs the public API. Provider details are hidden in:

• eval_agent/tool_box/_internal/

Vision usage controls

Environment variables:

• EVAL_TOOLBOX_VISION_MAX_CALLS (default 2)
• EVAL_TOOLBOX_VISION_MAX_IMAGES (default 2)
• EVAL_TOOLBOX_VISION_MODEL (default gemini-2.5-flash)
• EVAL_TOOLBOX_GCP_PROJECT (default research-01-268019)
• EVAL_TOOLBOX_GCP_LOCATION (default global)
• EVAL_TOOLBOX_VISION_SYSTEM (optional system instruction)

Usage is logged to:

• <experiment_root>/eval_agent_memory/tool_usage.json

Template

Reference example:

• eval_agent/templates/auxiliary_metrics_template.py