---
tags:
- ml-intern
---
# ArcGIS Test Script Generator Agent

An AI agent that reads ArcGIS tool documentation and generates test scripts following your team's template and conventions.

## Quick Start

```bash
# 1. Install
pip install -e .

# 2. Set credentials
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
export AZURE_OPENAI_KEY="your-key"

# 3. Set up your directories
export DOCS_DIR="./docs"           # Tool documentation (.md files)
export TEMPLATE_DIR="./templates"  # Your test template + validators.json
export EXAMPLES_DIR="./examples"   # Example test scripts by category
export TEST_DATA_ROOT="./test_data" # With manifest.json

# 4. Generate a test
python -m arcgis_test_agent generate Buffer
```

## How It Works

```
User: "Generate test for Buffer"
         │
         ▼
┌─────────────────────────────────────────────┐
│  PHASE 1: PLANNING                          │
│  • read_tool_docs("Buffer")                 │
│  • search_data_index("polygon shapefile")   │
│  • search_data_index("polyline features")   │
│  • read_output_validators("feature_class")  │
│  • read_example_tests("analysis")           │
├─────────────────────────────────────────────┤
│  PHASE 2: CODING                            │
│  • read_test_template()                     │
│  • [LLM writes complete test script]        │
├─────────────────────────────────────────────┤
│  PHASE 3: REVIEW                            │
│  • validate_script(code)                    │
│  • [Fix issues if any, retry up to 3x]     │
└─────────────────────────────────────────────┘
         │
         ▼
  generated_tests/test_buffer.py
```

## Directory Setup

You need to provide 4 things:

### 1. `docs/` — Tool Documentation

One markdown file per tool:
```
docs/
├── Buffer.md
├── Clip.md
├── ReconstructSurface.md
└── ...
```

### 2. `templates/test_template.py` — Your Test Template

The exact structure your generated tests must follow.

### 3. `templates/validators.json` — Output Validators Registry

```json
{
    "feature_class": [
        {"name": "assertFeatureClassExists", "signature": "self.assertFeatureClassExists(path)", "description": "..."}
    ],
    "raster": [...],
    "json": [...]
}
```

### 4. `test_data/manifest.json` — Available Test Data Index

```json
[
    {
        "path": "vector/polygon/parcels.shp",
        "data_type": "vector",
        "geometry_type": "polygon",
        "description": "Polygon shapefile with 500 parcel features in NAD83",
        "spatial_reference": "NAD83 UTM Zone 11N",
        "feature_count": 500
    }
]
```

## CLI Usage

```bash
python -m arcgis_test_agent generate Buffer
python -m arcgis_test_agent generate Buffer --output tests/test_buffer.py
python -m arcgis_test_agent generate Buffer --context "focus on 3D polygon inputs"
python -m arcgis_test_agent generate Buffer --verbose
python -m arcgis_test_agent info
```

## Python API

```python
from arcgis_test_agent import TestGeneratorAgent, AzureConfig, AgentConfig

agent = TestGeneratorAgent()
result = agent.generate("Buffer")

if result.success:
    print(result.script)
```

## Evaluation

```python
from arcgis_test_agent.eval import evaluate_structural

with open("generated_tests/test_buffer.py") as f:
    code = f.read()

score = evaluate_structural(code)
print(f"Composite score: {score.composite:.3f}")
```

## Demo (No Azure Credentials Required)

```bash
python arcgis_test_agent/demo/run_demo.py
```

## Project Structure

```
arcgis_test_agent/
├── __init__.py          # Package entry point
├── __main__.py          # CLI
├── agent.py             # Core agent loop (tool-calling)
├── config.py            # Configuration management
├── eval.py              # Evaluation framework
├── tools/
│   ├── schemas.py       # OpenAI function schemas (6 tools)
│   └── handlers.py      # Tool implementations
├── prompts/
│   └── system.py        # System prompt
└── demo/
    ├── run_demo.py      # End-to-end demo
    ├── docs/            # Sample tool docs
    ├── templates/       # Sample template + validators
    ├── examples/        # Sample test scripts
    └── test_data/       # Sample manifest
```

<!-- ml-intern-provenance -->
## Generated by ML Intern

This model repository was generated by [ML Intern](https://github.com/huggingface/ml-intern), an agent for machine learning research and development on the Hugging Face Hub.

- Try ML Intern: https://smolagents-ml-intern.hf.space
- Source code: https://github.com/huggingface/ml-intern

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "dduseja/arcgis-test-agent"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
```

For non-causal architectures, replace `AutoModelForCausalLM` with the appropriate `AutoModel` class.