File size: 865 Bytes
3f6526a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | # Eval Agent Toolbox
Minimal tool gateway for `auxiliary_metrics.py`.
Public API:
- `eval_agent.tool_box.call_vision(text, image_paths, results_dir=...) -> str`
- `eval_agent.tool_box.call_tool(name, payload) -> str`
The agent only needs the public API. Provider details are hidden in:
- `eval_agent/tool_box/_internal/`
## Vision usage controls
Environment variables:
- `EVAL_TOOLBOX_VISION_MAX_CALLS` (default `2`)
- `EVAL_TOOLBOX_VISION_MAX_IMAGES` (default `2`)
- `EVAL_TOOLBOX_VISION_MODEL` (default `gemini-2.5-flash`)
- `EVAL_TOOLBOX_GCP_PROJECT` (default `research-01-268019`)
- `EVAL_TOOLBOX_GCP_LOCATION` (default `global`)
- `EVAL_TOOLBOX_VISION_SYSTEM` (optional system instruction)
Usage is logged to:
- `<experiment_root>/eval_agent_memory/tool_usage.json`
## Template
Reference example:
- `eval_agent/templates/auxiliary_metrics_template.py`
|