README.md

metadata

title: InformationIntegrityEnv
emoji: 🔍
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 6.11.0
app_file: app.py
pinned: false
license: mit
tags:
  - openenv
  - content-moderation
  - rl-environment
  - agent-benchmark

InformationIntegrityEnv

A deterministic, benchmarkable OpenEnv environment for evaluating LLM agents on reasoning-based content moderation.

Agents must verify claims, weigh conflicting evidence, and make explainable moderation decisions (allow / flag / remove). Three difficulty tiers — easy, medium, hard — provide 15 fully static scenarios that run completely offline.

---

Architecture

tasks/
  easy/scenarios.json     ← 5 static scenarios (clear-cut misinformation)
  medium/scenarios.json   ← 5 static scenarios (misleading / mixed evidence)
  hard/scenarios.json     ← 5 static scenarios (conflicting evidence, nuanced)
  graders.py              ← EasyGrader / MediumGrader / HardGrader
  registry.py             ← TASKS["easy"] / TASKS["medium"] / TASKS["hard"]
        │
        ▼
InformationIntegrityEnv         (environment.py)
  reset(difficulty, scenario_id) → Observation
  step(action)                   → (Observation, Reward, done, info)
  state()                        → full debug state
        │
  ┌─────┴──────┐
  │            │
inference.py   server.py
(benchmark     (FastAPI
 agent loop)    REST API)

No external API calls occur during evaluation. DataCollector is only used to build/refresh datasets.

---

Quick Start

1. Install

pip install -r requirements.txt

2. Set API keys (for the LLM agent — not needed for just the server)

export HF_TOKEN=sk-...          # or OPENAI_API_KEY
export MODEL_NAME=gpt-4o
export API_BASE_URL=https://api.openai.com/v1

3. Run the full benchmark

python inference.py --benchmark --out results.json

This loops over all 15 scenarios (5 per difficulty) and prints:

[START] episode=E01 difficulty=easy claims=1 sources=5
[STEP]  step=1 action=check_source reward=+0.200 total=+0.200 done=False ...
[STEP]  step=2 action=verify_claim  reward=+0.200 total=+0.400 done=False ...
[STEP]  step=3 action=decide        reward=+1.000 total=+1.400 done=True  ...
[END]   episode=E01 steps=3 decision=remove total_reward=+1.4000 grader_score=0.8750

...

difficulty=easy   avg_score=0.7800  avg_reward=1.2000  n=5
difficulty=medium avg_score=0.6300  avg_reward=0.9000  n=5
difficulty=hard   avg_score=0.5200  avg_reward=0.6500  n=5
Difficulty tiers well separated: easy > medium > hard ✓

4. Single episode (local, no server)

# Random scenario from medium dataset
python inference.py --local --difficulty medium

# Specific scenario by index
python inference.py --local --difficulty hard --scenario-id 2

# Specific scenario by id
python inference.py --local --difficulty easy --scenario-id easy_001

# From a custom JSON file (DataCollector output)
python inference.py --local --scenario my_scenario.json

5. Start the API server

python server.py
# or
uvicorn server:app --reload

---

Action Space

Action	Required fields	Description
verify_claim	claim_id	Investigate a claim; returns true / false / partially_true / unknown
check_source	source_id	Read a source; returns title, text, stance, reliability
decide	decision	Final call: allow / flag / remove — ends episode

All actions accept an optional reasoning string (logged, not graded).

Minimum evidence required before deciding (or premature penalty applies):

• ≥ 1 claim verified
• ≥ 2 sources checked

---

Observation Space

{
  "content": "The post text under review",
  "claims": [
    {"id": "claim_001", "text": "...", "verification_status": "unverified", "confidence": 0.0}
  ],
  "available_sources": [
    {"id": "src_001", "title": "...", "text": "...", "url": "...",
     "stance": "contradicting", "reliability": 0.97, "source_type": "official"}
  ],
  "interaction_history": [],
  "status_flags": {
    "decision_made": false,
    "sufficient_evidence": false,
    "premature_decision": false,
    "redundant_actions": 0,
    "max_steps_reached": false
  },
  "step_count": 0,
  "task_difficulty": "medium",
  "episode_id": "uuid"
}

---

Reward Function

Component	Value	Trigger
Verify claim	+0.20	Per unique claim verified
Check source	+0.20	Per unique source checked
Multi-source bonus	+0.20	When ≥ 3 unique sources checked
Correct decision	+1.00	Exact ground truth match
Partial decision	+0.50	One tier off (flag↔remove, flag↔allow)
Premature decision	−0.50	Deciding before sufficient evidence
Redundant action	−0.10	Re-checking same claim/source
Inefficiency	−0.05	Per step beyond efficient threshold
Over-aggression	−0.30	remove when ground truth is allow/flag

Total clipped to [-2.0, +2.0]. All values configurable in reward_config.yaml.

---

Grader Scoring

Each episode is scored [0.0, 1.0] across four dimensions:

Dimension	Easy	Medium	Hard
Decision correctness	50%	35%	30%
Evidence breadth	25%	35%	30%
Reasoning depth	15%	20%	30%
Efficiency	10%	10%	10%

---

Task Datasets

Easy (5 scenarios)

Clear-cut false claims contradicted by multiple authoritative sources (WHO, CDC, NASA, FDA). Ground truth: 3× remove, 1× flag, 1× allow.

ID	Topic	Ground Truth
easy_001	Bleach cures COVID	remove
easy_002	NASA fabricated climate data	remove
easy_003	mRNA vaccines alter DNA	remove
easy_004	Sydney is Australia's capital	flag
easy_005	Webb telescope early universe	allow

Medium (5 scenarios)

Partially true or misleadingly framed content. Mixed evidence requires weighing. Ground truth: 4× flag, 1× allow.

ID	Topic	Ground Truth
medium_001	Sugar more addictive than cocaine	flag
medium_002	Real unemployment is 20%+	flag
medium_003	AI will eliminate 85M jobs	flag
medium_004	Pharma knew OxyContin was addictive	allow
medium_005	Facebook hid teen mental health data	flag

Hard (5 scenarios)

Conflicting credible evidence, genuinely contested claims, contested policy debates. Ground truth: 5× flag.

ID	Topic	Ground Truth
hard_001	NATO provoked Ukraine invasion	flag
hard_002	Lockdowns killed more than COVID	flag
hard_003	SEC crackdown is banking conspiracy	flag
hard_004	AI poses extinction-level risk	flag
hard_005	British colonialism net beneficial	flag

---

API Reference

Method	Endpoint	Description
GET	/health	Liveness probe
GET	/tasks	List all available scenario datasets
POST	/reset	Start a new episode
POST	/step	Submit one action
GET	/state/{session_id}	Full internal state (debug/audit)
GET	/scenario/{session_id}	Scenario metadata (no ground truth)
DELETE	/session/{session_id}	End session and free memory
POST	/collect	Build a live scenario via DataCollector (dataset creation only)

Reset — evaluation path (no DataCollector needed)

# Random scenario from easy dataset
curl -X POST http://localhost:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"difficulty": "easy"}'

# Specific scenario by index
curl -X POST http://localhost:8000/reset \
  -d '{"difficulty": "medium", "scenario_id": 2}'

# Specific scenario by id
curl -X POST http://localhost:8000/reset \
  -d '{"difficulty": "hard", "scenario_id": "hard_001"}'

Full episode example

SESSION=$(curl -s -X POST http://localhost:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"difficulty": "easy"}' | jq -r .session_id)

# Check a source
curl -X POST http://localhost:8000/step \
  -d "{\"session_id\": \"$SESSION\", \"action\": {\"type\": \"check_source\", \"source_id\": \"src_001\"}}"

# Verify a claim
curl -X POST http://localhost:8000/step \
  -d "{\"session_id\": \"$SESSION\", \"action\": {\"type\": \"verify_claim\", \"claim_id\": \"claim_001\"}}"

# Decide
curl -X POST http://localhost:8000/step \
  -d "{\"session_id\": \"$SESSION\", \"action\": {\"type\": \"decide\", \"decision\": \"remove\"}}"

---

Docker

Server (default)

docker build -t info-integrity-env .
docker run --env-file .env -p 8000:8000 info-integrity-env

Benchmark (offline — no external services needed)

docker run --env-file .env info-integrity-env \
  python inference.py --benchmark --out /tmp/results.json

Full stack with Redis (for live data collection)

docker compose up --build

---

Baseline Results

Difficulty	Avg Grader Score	Avg Reward	Decision Accuracy
Easy	~0.78	~1.20	~85%
Medium	~0.63	~0.90	~70%
Hard	~0.52	~0.65	~55%

Results vary by LLM. Scores above from gpt-4o with temperature=0.

---

Extending the Environment

File	What to change
tasks/{difficulty}/scenarios.json	Add or edit scenarios (run openenv validate after)
reward_config.yaml	Tune reward/penalty magnitudes
tasks/graders.py	Register custom graders: GraderFactory.register("hard_v2", MyGrader)
data_collector.py	Add new search backends via SearchBackend ABC
domain_reliability.yaml	Add trusted domains with custom reliability scores

---

Project Structure

├── models.py                    Pydantic models (Observation, Action, Reward, Scenario, ...)
├── environment.py               InformationIntegrityEnv (OpenEnv core)
├── reward.py                    Configurable dense reward function
├── explainability.py            Structured reasoning + template-based explanation
├── data_collector.py            Multi-backend data collection (dataset creation only)
├── inference.py                 Benchmark agent + single-episode runner
├── server.py                    FastAPI server (HF Space / Docker)
├── tasks/
│   ├── __init__.py
│   ├── graders.py               Easy / Medium / Hard deterministic graders
│   ├── registry.py              TASKS registry — lazy-loads scenario datasets
│   ├── easy/
│   │   └── scenarios.json       5 easy scenarios
│   ├── medium/
│   │   └── scenarios.json       5 medium scenarios
│   └── hard/
│       └── scenarios.json       5 hard scenarios
├── openenv.yaml                 OpenEnv v1.1.0 specification
├── reward_config.yaml           Reward magnitudes (runtime-configurable)
├── domain_reliability.yaml      Domain trust scores for DataCollector
├── Dockerfile                   Production container
├── docker-compose.yml           Local dev stack (server + Redis)
└── requirements.txt