ARCHITECTURE.md

# Backend Architecture

## Project Structure

```
MLops-Openenvhack/
├── app.py                 # FastAPI server - main entry point
├── inference.py           # Baseline LLM agent for evaluation
├── models.py              # Pydantic models (Action, Observation, State)
├── mlops_environment.py   # Core environment logic
├── artifact_generator.py  # Procedural bug/artifact generation
├── client.py              # Python client library
├── openenv.yaml           # OpenEnv specification
├── Dockerfile             # Container configuration
├── requirements.txt       # Python dependencies
└── README.md             # Documentation
```

## How It Works

### 1. Server (app.py)
- Runs FastAPI on port 7860
- Provides REST endpoints:
  - `GET /health` - Health check
  - `POST /reset` - Initialize new task
  - `POST /step` - Execute action
  - `GET /state` - Get current state
  - `GET /tasks` - List available tasks
  - `GET /openenv/state` - OpenEnv state

### 2. Environment (mlops_environment.py)
- Manages task state
- Processes actions through `_handle_*` methods
- Generates rewards based on agent behavior
- Tracks artifacts read and sanity checks

### 3. Artifact Generator (artifact_generator.py)
- Procedurally generates training artifacts with planted bugs
- Creates realistic: logs, configs, preprocessing code, eval results
- Supports 9 bug types across 3 difficulty levels

### 4. Inference Agent (inference.py)
- LLM-powered agent using OpenAI API
- Reads artifacts, runs sanity checks
- Submits diagnosis with confidence scoring
- Implements rate limiting and fallback

## API Flow

```
Client -> app.py (FastAPI)
           |
           +-> mlops_environment.py (core logic)
                    |
                    +-> artifact_generator.py (bug generation)
                    |
                    +-> models.py (data validation)
                    |
                    +-> Returns Observation, Reward, Done, Info
```

## Task Flow

```
1. Client POST /reset with task_id (easy/medium/hard)
2. Environment generates artifacts with planted bug
3. Client POST /step with action
4. Environment processes action, returns observation
5. Agent investigates until diagnosis submitted
6. Grader scores against planted bug (0.0 - 1.0)
```

## Data Models

### Action Types
- read_config, read_logs, check_dataset_stats
- inspect_preprocessing, read_eval_results
- run_sanity_check, query_artifact
- submit_diagnosis

### Reward Structure
- +0.02 per new artifact read
- -0.02 per duplicate read
- +0.01 per new sanity check
- Terminal: +0.15 category + 0.25 file + 0.30 field + 0.30 fix