metadata
tags:
- ml-intern
ArcGIS Test Script Generator Agent
An AI agent that reads ArcGIS tool documentation and generates test scripts following your team's template and conventions.
Quick Start
# 1. Install
pip install -e .
# 2. Set credentials
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
export AZURE_OPENAI_KEY="your-key"
# 3. Set up your directories
export DOCS_DIR="./docs" # Tool documentation (.md files)
export TEMPLATE_DIR="./templates" # Your test template + validators.json
export EXAMPLES_DIR="./examples" # Example test scripts by category
export TEST_DATA_ROOT="./test_data" # With manifest.json
# 4. Generate a test
python -m arcgis_test_agent generate Buffer
How It Works
User: "Generate test for Buffer"
│
▼
┌─────────────────────────────────────────────┐
│ PHASE 1: PLANNING │
│ • read_tool_docs("Buffer") │
│ • search_data_index("polygon shapefile") │
│ • search_data_index("polyline features") │
│ • read_output_validators("feature_class") │
│ • read_example_tests("analysis") │
├─────────────────────────────────────────────┤
│ PHASE 2: CODING │
│ • read_test_template() │
│ • [LLM writes complete test script] │
├─────────────────────────────────────────────┤
│ PHASE 3: REVIEW │
│ • validate_script(code) │
│ • [Fix issues if any, retry up to 3x] │
└─────────────────────────────────────────────┘
│
▼
generated_tests/test_buffer.py
Directory Setup
You need to provide 4 things:
1. docs/ — Tool Documentation
One markdown file per tool:
docs/
├── Buffer.md
├── Clip.md
├── ReconstructSurface.md
└── ...
2. templates/test_template.py — Your Test Template
The exact structure your generated tests must follow.
3. templates/validators.json — Output Validators Registry
{
"feature_class": [
{"name": "assertFeatureClassExists", "signature": "self.assertFeatureClassExists(path)", "description": "..."}
],
"raster": [...],
"json": [...]
}
4. test_data/manifest.json — Available Test Data Index
[
{
"path": "vector/polygon/parcels.shp",
"data_type": "vector",
"geometry_type": "polygon",
"description": "Polygon shapefile with 500 parcel features in NAD83",
"spatial_reference": "NAD83 UTM Zone 11N",
"feature_count": 500
}
]
CLI Usage
python -m arcgis_test_agent generate Buffer
python -m arcgis_test_agent generate Buffer --output tests/test_buffer.py
python -m arcgis_test_agent generate Buffer --context "focus on 3D polygon inputs"
python -m arcgis_test_agent generate Buffer --verbose
python -m arcgis_test_agent info
Python API
from arcgis_test_agent import TestGeneratorAgent, AzureConfig, AgentConfig
agent = TestGeneratorAgent()
result = agent.generate("Buffer")
if result.success:
print(result.script)
Evaluation
from arcgis_test_agent.eval import evaluate_structural
with open("generated_tests/test_buffer.py") as f:
code = f.read()
score = evaluate_structural(code)
print(f"Composite score: {score.composite:.3f}")
Demo (No Azure Credentials Required)
python arcgis_test_agent/demo/run_demo.py
Project Structure
arcgis_test_agent/
├── __init__.py # Package entry point
├── __main__.py # CLI
├── agent.py # Core agent loop (tool-calling)
├── config.py # Configuration management
├── eval.py # Evaluation framework
├── tools/
│ ├── schemas.py # OpenAI function schemas (6 tools)
│ └── handlers.py # Tool implementations
├── prompts/
│ └── system.py # System prompt
└── demo/
├── run_demo.py # End-to-end demo
├── docs/ # Sample tool docs
├── templates/ # Sample template + validators
├── examples/ # Sample test scripts
└── test_data/ # Sample manifest
Generated by ML Intern
This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.
- Try ML Intern: https://smolagents-ml-intern.hf.space
- Source code: https://github.com/huggingface/ml-intern
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "dduseja/arcgis-test-agent"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
For non-causal architectures, replace AutoModelForCausalLM with the appropriate AutoModel class.