README.md

---
title: API Triage Agent
emoji: 🔧
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
---

# API Triage Agent

An OpenEnv-compliant reinforcement learning environment that trains AI agents to diagnose and resolve real-world API integration failures. The agent inspects logs, identifies error patterns, and applies corrective actions — mirroring the workflow of an on-call support engineer.

## Why This Matters

API failures cost companies millions in downtime and lost revenue. Authentication expiry, malformed payloads, rate limiting, endpoint misconfiguration, and server errors are the top five categories that SRE teams deal with daily. This environment lets an LLM agent learn the diagnostic playbook through trial and error, receiving shaped rewards that encourage methodical investigation before action.

## Project Structure

```
api-triage-agent/
├── app.py                          # FastAPI server (main entry point)
├── inference.py                    # Baseline agent script (OpenAI client)
├── openenv.yaml                    # OpenEnv manifest (tasks + graders)
├── Dockerfile                      # Container definition for HF Spaces
├── requirements.txt
├── pyproject.toml
├── environment/
│   ├── api_triage_env.py           # Core environment (reset/step/state)
│   ├── incident_generator.py       # Procedural incident generation
│   ├── action_space.py             # Valid action definitions
│   └── reward.py                   # 5-factor reward function
├── tasks/
│   ├── grading_helper.py           # Shared grading utilities
│   ├── auth_error/grader.py        # Task 1 grader
│   ├── missing_fields/grader.py    # Task 2 grader
│   ├── rate_limit/grader.py        # Task 3 grader
│   ├── timeout/grader.py           # Task 4 grader
│   ├── wrong_endpoint/grader.py    # Task 5 grader
│   └── server_error/grader.py      # Task 6 grader
└── tests/
    ├── test_env.py                 # Environment unit tests
    └── test_graders.py             # Grader validation tests
```

## Action Space

The agent chooses from 8 discrete actions per step:

| Action | Description | When to Use |
|--------|-------------|-------------|
| `inspect_logs` | Read error logs for diagnostic clues | First step in any incident |
| `inspect_request` | Examine the failed HTTP request | Gather additional context |
| `refresh_token` | Regenerate expired API credentials | 401 Unauthorized errors |
| `add_field` | Add missing required payload fields | 400 Bad Request errors |
| `wait_retry` | Back off and retry the request | 429 Rate Limit / 408 Timeout |
| `change_endpoint` | Switch to the correct API path | 404 Not Found errors |
| `escalate` | Escalate to a human operator | 500 Internal Server errors |
| `resolve` | End the episode (must apply fix first) | After successful remediation |

## Observation Space

Each step returns a structured observation:

| Field | Type | Description |
|-------|------|-------------|
| `step` | `int` | Current step number (1-indexed) |
| `max_steps` | `int` | Maximum allowed steps (default: 10) |
| `incident_summary` | `str` | Human-readable problem description |
| `logs` | `list[str]` | Simulated error log entries |
| `response_code` | `int` | HTTP status code (401, 400, 429, 408, 404, 500) |
| `fix_applied` | `bool` | Whether the correct fix action has been taken |
| `is_resolved` | `bool` | Whether the episode has terminated |

## Tasks (6 tasks, Easy → Hard)

Each task targets a specific API failure pattern. Graders evaluate whether the agent follows the correct diagnostic-then-fix workflow.

| # | Task ID | Difficulty | HTTP Code | Correct Fix | Grader |
|---|---------|-----------|-----------|-------------|--------|
| 1 | `auth_error` | Easy | 401 | `refresh_token` → `resolve` | `tasks.auth_error.grader:grade` |
| 2 | `missing_fields` | Easy | 400 | `add_field` → `resolve` | `tasks.missing_fields.grader:grade` |
| 3 | `rate_limit` | Medium | 429 | `wait_retry` → `resolve` | `tasks.rate_limit.grader:grade` |
| 4 | `timeout` | Medium | 408 | `wait_retry` → `resolve` | `tasks.timeout.grader:grade` |
| 5 | `wrong_endpoint` | Medium | 404 | `change_endpoint` → `resolve` | `tasks.wrong_endpoint.grader:grade` |
| 6 | `server_error` | Hard | 500 | `escalate` → `resolve` | `tasks.server_error.grader:grade` |

## Reward Function

A 5-factor shaped reward system provides partial-credit signals at every step, guiding the agent toward the optimal diagnostic workflow:

| Factor | Reward | Rationale |
|--------|--------|-----------|
| Correct fix action | **+5.0** | Directly addresses the root cause |
| Diagnostic action (`inspect_logs`, `inspect_request`) | **+0.5** | Encourages investigation before action |
| Successful resolution (`resolve` after fix) | **+15.0** | Large bonus for completing the episode correctly |
| Premature resolution (`resolve` without fix) | **-10.0** | Prevents the agent from "lying" about fixing the issue |
| Wrong action | **-2.0** | Mild penalty to discourage random exploration |
| Max steps reached | **-5.0** | Time pressure to act efficiently |

**Reward range:** `[-20.0, +20.5]`  
**Optimal 3-step episode:** `inspect_logs (+0.5)` → correct fix (`+5.0`) → `resolve (+15.0)` = **+20.5**

## API Endpoints

The FastAPI server exposes the following endpoints:

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/` | Root status check |
| `GET` | `/health` | Health check |
| `POST` | `/reset` | Reset environment, get initial observation |
| `POST` | `/step` | Execute an action, receive observation + reward |
| `GET` | `/state` | Get current environment state |
| `GET` | `/tasks` | List all tasks with grader references |
| `POST` | `/grade/{task_id}` | Run a specific task's grader |
| `GET` | `/docs` | Swagger UI documentation |

## Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `API_BASE_URL` | LLM API endpoint | `https://router.huggingface.co/v1` |
| `MODEL_NAME` | Model identifier for inference | `Qwen/Qwen2.5-72B-Instruct` |
| `HF_TOKEN` | Hugging Face API key | *(required)* |

## Setup & Run

### Local Development

```bash
# Clone the repo
git clone https://huggingface.co/spaces/Kavya988/API_DEBUG_SOLVER
cd API_DEBUG_SOLVER

# Create virtual environment
python -m venv venv
source venv/bin/activate    # Linux/Mac
venv\Scripts\activate       # Windows

# Install dependencies
pip install -r requirements.txt

# Run the server
python app.py
# Server starts at http://localhost:7860

# Run tests
pytest tests/test_env.py -v
python tests/test_graders.py
```

### Docker

```bash
docker build -t api-triage-agent .
docker run -p 7860:7860 -e HF_TOKEN=your_token api-triage-agent
```

### Run Inference

```bash
export HF_TOKEN=your_token
export API_BASE_URL=https://router.huggingface.co/v1
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct

python inference.py
```

The inference script runs all 6 tasks sequentially, emitting structured `[START]`/`[STEP]`/`[END]` logs for each task. Scores are computed from actual episode rewards, clamped to `[0.0, 1.0]`.

## OpenEnv Compliance

This environment implements the full OpenEnv specification:

- **Typed models:** Pydantic `BaseModel` for observations, actions, and state
- **Standard endpoints:** `reset()`, `step()`, `state()` via FastAPI
- **Task discovery:** `GET /tasks` returns all tasks with grader module references
- **Agent graders:** 6 grader functions (one per task), each returning scores in `[0.0, 1.0]`
- **Manifest:** `openenv.yaml` defines environment metadata, action/observation spaces, and task-grader mappings
- **Containerized:** Dockerfile builds and runs on HF Spaces (port 7860)

## Deployment

Live at: **[https://kavya988-api-debug-solver.hf.space](https://kavya988-api-debug-solver.hf.space)**