---
title: Contract Validation Environment Server
emoji: 📝
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
app_port: 8000
base_path: /web
tags:
  - openenv
---

# Contract Validation Environment

The **Contract Validation Environment** is an OpenEnv-compliant RL and LLM benchmark designed to test an agent's ability to act as a precise legal assistant. The agent must review various contract clauses, identify specific legal risks (e.g., liability, termination, payment), and correctly flag them without generating false positives on standard, safe clauses.

## 🚀 Motivation
Legal contract review is a massive industry, but off-the-shelf LLMs often struggle with "alert fatigue"—flagging everything as a risk. This environment challenges agents to precisely isolate genuine liabilities across varying difficulty levels while explicitly rewarding speed and accuracy.

---

## 🎯 Tasks & Difficulty

The environment features 3 deterministic tasks with increasing complexity:

1. **Easy:** 1 clause. Contains a single, explicit liability risk. Tests basic risk identification.
2. **Medium:** 3 clauses. Requires identifying payment and termination risks while actively ignoring a safe governing-law distractor clause.
3. **Hard:** 5 clauses. A complex mix of confidentiality, liability, and compliance risks interspersed with dense, safe, standard boilerplate clauses. Challenges frontier models to avoid false positives.

---

## 📊 Environment Details

### Action Space (`ContractValidationAction`)
- `clause_id` (int): The ID of the clause being reviewed (Set to 0 if submitting final).
- `risk_type` (str): The identified risk (e.g., 'liability', 'payment', 'termination', 'confidentiality', 'compliance', or 'none').
- `submit_final` (bool): Set to `True` when the agent has finished flagging risks to end the episode and receive a final score.
- `explanation` (str): The agent's chain-of-thought or reasoning for the decision.

### Observation Space (`ContractValidationObservation`)
- `task_level` (str): Difficulty level of the current task ("easy", "medium", "hard").
- `contract_clauses` (list): List of dictionaries containing the `id` and `text` of the contract clauses to review.
- `flagged_risks` (dict): A dictionary mapping clause IDs to the risks currently flagged by the agent.
- `step_count` (int): Number of steps taken in the current episode.
- `reward` (float): The reward delta granted for the most recent action.
- `done` (bool): Whether the episode has concluded.
- `info` (dict): Additional environment info, including the current internal `score` from the grader.

### Reward Function & Grader
The environment utilizes a **trajectory-based reward system**. The grader calculates a score between `0.0` and `1.0` based on precision and recall.
* **Positive Reward:** Granted for newly correct flags.
* **Negative Penalty:** Applied for flagging safe clauses or assigning the wrong risk type.
* **Step Penalty:** A `-0.02` penalty is applied per step to encourage the agent to evaluate the contract efficiently. Rewards are clamped to `max(0.0)` to ensure compatibility with OpenEnv graders.
* **Completion Bonus:** A `+0.5` bonus is awarded if the agent submits the contract with a perfect 1.0 grader score.

---

## Project Structure

```
contract_validation/
├── .dockerignore          # Docker build exclusions
├── .env                   # Local environment variables (API keys - DO NOT COMMIT)
├── .gitignore             # Git tracking exclusions (ignores .env, caches, etc.)
├── __init__.py            # Module exports
├── README.md              # Project documentation (with tags: - openenv)
├── openenv.yaml           # OpenEnv manifest
├── pyproject.toml         # Project metadata and dependencies
├── uv.lock                # Locked dependencies (generated)
├── client.py              # ContractValidationEnv client
├── inference.py           # Evaluation script for the OpenEnv grader (JSON logging)
├── models.py              # Action and Observation Pydantic models
├── Dockerfile             # Container image definition
└── server/
    ├── __init__.py        # Server module exports
    ├── contract_validation_environment.py  # Core environment logic and task data
    └── app.py             # FastAPI application (HTTP + WebSocket endpoints)
```