Blame for README.md

---
title: InformationIntegrityEnv
emoji: 🔍
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 6.11.0
app_file: app.py
pinned: false
license: mit
tags:
  - openenv
  - content-moderation
  - rl-environment
  - agent-benchmark
---

# InformationIntegrityEnv

A **deterministic, benchmarkable OpenEnv environment** for evaluating LLM agents on reasoning-based content moderation.

Agents must verify claims, weigh conflicting evidence, and make explainable moderation decisions (`allow` / `flag` / `remove`). Three difficulty tiers — easy, medium, hard — provide 15 fully static scenarios that run completely offline.

---

## Architecture

```
tasks/
  easy/scenarios.json     ← 5 static scenarios (clear-cut misinformation)
  medium/scenarios.json   ← 5 static scenarios (misleading / mixed evidence)
  hard/scenarios.json     ← 5 static scenarios (conflicting evidence, nuanced)
  graders.py              ← EasyGrader / MediumGrader / HardGrader
  registry.py             ← TASKS["easy"] / TASKS["medium"] / TASKS["hard"]
        │
        ▼
InformationIntegrityEnv         (environment.py)
  reset(difficulty, scenario_id) → Observation
  step(action)                   → (Observation, Reward, done, info)
  state()                        → full debug state
        │
  ┌─────┴──────┐
  │            │
inference.py   server.py
(benchmark     (FastAPI
 agent loop)    REST API)
```

No external API calls occur during evaluation. `DataCollector` is only used to build/refresh datasets.

---

## Quick Start

### 1. Install

```bash
pip install -r requirements.txt
```

### 2. Set API keys (for the LLM agent — not needed for just the server)

```bash
export HF_TOKEN=sk-...          # or OPENAI_API_KEY
export MODEL_NAME=gpt-4o
export API_BASE_URL=https://api.openai.com/v1
```

### 3. Run the full benchmark

```bash
python inference.py --benchmark --out results.json
```

This loops over all 15 scenarios (5 per difficulty) and prints:

```
[START] episode=E01 difficulty=easy claims=1 sources=5
[STEP]  step=1 action=check_source reward=+0.200 total=+0.200 done=False ...
[STEP]  step=2 action=verify_claim  reward=+0.200 total=+0.400 done=False ...
[STEP]  step=3 action=decide        reward=+1.000 total=+1.400 done=True  ...
[END]   episode=E01 steps=3 decision=remove total_reward=+1.4000 grader_score=0.8750

...

difficulty=easy   avg_score=0.7800  avg_reward=1.2000  n=5
difficulty=medium avg_score=0.6300  avg_reward=0.9000  n=5
difficulty=hard   avg_score=0.5200  avg_reward=0.6500  n=5
Difficulty tiers well separated: easy > medium > hard ✓
```

### 4. Single episode (local, no server)

```bash
# Random scenario from medium dataset
python inference.py --local --difficulty medium

# Specific scenario by index
python inference.py --local --difficulty hard --scenario-id 2

# Specific scenario by id
python inference.py --local --difficulty easy --scenario-id easy_001

# From a custom JSON file (DataCollector output)
python inference.py --local --scenario my_scenario.json
```

### 5. Start the API server

```bash
python server.py
# or
uvicorn server:app --reload
```

---

## Action Space

| Action | Required fields | Description |
|--------|----------------|-------------|
| `verify_claim` | `claim_id` | Investigate a claim; returns true / false / partially_true / unknown |
| `check_source` | `source_id` | Read a source; returns title, text, stance, reliability |
| `decide` | `decision` | Final call: `allow` / `flag` / `remove` — ends episode |

All actions accept an optional `reasoning` string (logged, not graded).

**Minimum evidence required before deciding** (or premature penalty applies):
- ≥ 1 claim verified
- ≥ 2 sources checked

---

## Observation Space

```json
{
  "content": "The post text under review",
  "claims": [
    {"id": "claim_001", "text": "...", "verification_status": "unverified", "confidence": 0.0}
  ],
  "available_sources": [
    {"id": "src_001", "title": "...", "text": "...", "url": "...",
     "stance": "contradicting", "reliability": 0.97, "source_type": "official"}
  ],
  "interaction_history": [],
  "status_flags": {
    "decision_made": false,
    "sufficient_evidence": false,
    "premature_decision": false,
    "redundant_actions": 0,
    "max_steps_reached": false
  },
  "step_count": 0,
  "task_difficulty": "medium",
  "episode_id": "uuid"
}
```

---

## Reward Function

| Component | Value | Trigger |
|-----------|-------|---------|
| Verify claim | +0.20 | Per unique claim verified |
| Check source | +0.20 | Per unique source checked |
| Multi-source bonus | +0.20 | When ≥ 3 unique sources checked |
| Correct decision | +1.00 | Exact ground truth match |
| Partial decision | +0.50 | One tier off (flag↔remove, flag↔allow) |
| Premature decision | −0.50 | Deciding before sufficient evidence |
| Redundant action | −0.10 | Re-checking same claim/source |
| Inefficiency | −0.05 | Per step beyond efficient threshold |
| Over-aggression | −0.30 | `remove` when ground truth is `allow`/`flag` |

Total clipped to `[-2.0, +2.0]`. All values configurable in `reward_config.yaml`.

---

## Grader Scoring

Each episode is scored `[0.0, 1.0]` across four dimensions:

| Dimension | Easy | Medium | Hard |
|-----------|------|--------|------|
| Decision correctness | 50% | 35% | 30% |
| Evidence breadth | 25% | 35% | 30% |
| Reasoning depth | 15% | 20% | 30% |
| Efficiency | 10% | 10% | 10% |

---

## Task Datasets

### Easy (5 scenarios)
Clear-cut false claims contradicted by multiple authoritative sources (WHO, CDC, NASA, FDA).
Ground truth: 3× remove, 1× flag, 1× allow.

| ID | Topic | Ground Truth |
|----|-------|-------------|
| easy_001 | Bleach cures COVID | remove |
| easy_002 | NASA fabricated climate data | remove |
| easy_003 | mRNA vaccines alter DNA | remove |
| easy_004 | Sydney is Australia's capital | flag |
| easy_005 | Webb telescope early universe | allow |

### Medium (5 scenarios)
Partially true or misleadingly framed content. Mixed evidence requires weighing.
Ground truth: 4× flag, 1× allow.

| ID | Topic | Ground Truth |
|----|-------|-------------|
| medium_001 | Sugar more addictive than cocaine | flag |
| medium_002 | Real unemployment is 20%+ | flag |
| medium_003 | AI will eliminate 85M jobs | flag |
| medium_004 | Pharma knew OxyContin was addictive | allow |
| medium_005 | Facebook hid teen mental health data | flag |

### Hard (5 scenarios)
Conflicting credible evidence, genuinely contested claims, contested policy debates.
Ground truth: 5× flag.

| ID | Topic | Ground Truth |
|----|-------|-------------|
| hard_001 | NATO provoked Ukraine invasion | flag |
| hard_002 | Lockdowns killed more than COVID | flag |
| hard_003 | SEC crackdown is banking conspiracy | flag |
| hard_004 | AI poses extinction-level risk | flag |
| hard_005 | British colonialism net beneficial | flag |

---

## API Reference

| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/health` | Liveness probe |
| GET | `/tasks` | List all available scenario datasets |
| POST | `/reset` | Start a new episode |
| POST | `/step` | Submit one action |
| GET | `/state/{session_id}` | Full internal state (debug/audit) |
| GET | `/scenario/{session_id}` | Scenario metadata (no ground truth) |
| DELETE | `/session/{session_id}` | End session and free memory |
| POST | `/collect` | Build a live scenario via DataCollector (dataset creation only) |

### Reset — evaluation path (no DataCollector needed)

```bash
# Random scenario from easy dataset
curl -X POST http://localhost:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"difficulty": "easy"}'

# Specific scenario by index
curl -X POST http://localhost:8000/reset \
  -d '{"difficulty": "medium", "scenario_id": 2}'

# Specific scenario by id
curl -X POST http://localhost:8000/reset \
  -d '{"difficulty": "hard", "scenario_id": "hard_001"}'
```

### Full episode example

```bash
SESSION=$(curl -s -X POST http://localhost:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"difficulty": "easy"}' | jq -r .session_id)

# Check a source
curl -X POST http://localhost:8000/step \
  -d "{\"session_id\": \"$SESSION\", \"action\": {\"type\": \"check_source\", \"source_id\": \"src_001\"}}"

# Verify a claim
curl -X POST http://localhost:8000/step \
  -d "{\"session_id\": \"$SESSION\", \"action\": {\"type\": \"verify_claim\", \"claim_id\": \"claim_001\"}}"

# Decide
curl -X POST http://localhost:8000/step \
  -d "{\"session_id\": \"$SESSION\", \"action\": {\"type\": \"decide\", \"decision\": \"remove\"}}"
```

---

## Docker

### Server (default)

```bash
docker build -t info-integrity-env .
docker run --env-file .env -p 8000:8000 info-integrity-env
```

### Benchmark (offline — no external services needed)

```bash
docker run --env-file .env info-integrity-env \
  python inference.py --benchmark --out /tmp/results.json
```

### Full stack with Redis (for live data collection)

```bash
docker compose up --build
```

---

## Baseline Results

| Difficulty | Avg Grader Score | Avg Reward | Decision Accuracy |
|-----------|-----------------|------------|-----------------|
| Easy | ~0.78 | ~1.20 | ~85% |
| Medium | ~0.63 | ~0.90 | ~70% |
| Hard | ~0.52 | ~0.65 | ~55% |

*Results vary by LLM. Scores above from `gpt-4o` with temperature=0.*

---

## Extending the Environment

| File | What to change |
|------|---------------|
| `tasks/{difficulty}/scenarios.json` | Add or edit scenarios (run `openenv validate` after) |
| `reward_config.yaml` | Tune reward/penalty magnitudes |
| `tasks/graders.py` | Register custom graders: `GraderFactory.register("hard_v2", MyGrader)` |
| `data_collector.py` | Add new search backends via `SearchBackend` ABC |
| `domain_reliability.yaml` | Add trusted domains with custom reliability scores |

---

## Project Structure

```
├── models.py                    Pydantic models (Observation, Action, Reward, Scenario, ...)
├── environment.py               InformationIntegrityEnv (OpenEnv core)
├── reward.py                    Configurable dense reward function
├── explainability.py            Structured reasoning + template-based explanation
├── data_collector.py            Multi-backend data collection (dataset creation only)
├── inference.py                 Benchmark agent + single-episode runner
├── server.py                    FastAPI server (HF Space / Docker)
├── tasks/
│   ├── __init__.py
│   ├── graders.py               Easy / Medium / Hard deterministic graders
│   ├── registry.py              TASKS registry — lazy-loads scenario datasets
│   ├── easy/
│   │   └── scenarios.json       5 easy scenarios
│   ├── medium/
│   │   └── scenarios.json       5 medium scenarios
│   └── hard/
│       └── scenarios.json       5 hard scenarios
├── openenv.yaml                 OpenEnv v1.1.0 specification
├── reward_config.yaml           Reward magnitudes (runtime-configurable)
├── domain_reliability.yaml      Domain trust scores for DataCollector
├── Dockerfile                   Production container
├── docker-compose.yml           Local dev stack (server + Redis)
└── requirements.txt
```