developer.md

Developer Guide

---

1. Environment Setup

python -m venv .venv && source .venv/bin/activate
pip install git+https://github.com/meta-pytorch/OpenEnv.git
pip install -r requirements.txt

export API_BASE_URL="https://api.openai.com/v1"
export MODEL_NAME="gpt-4o-mini"
export HF_TOKEN="hf_..."
export ENV_URL="http://localhost:7860"

---

2. Local Dev

# Start server
PYTHONPATH=. uvicorn server.app:app --host 0.0.0.0 --port 7860 --reload

# Health check
curl http://localhost:7860/health        # {"status":"healthy"}
curl http://localhost:7860/docs          # OpenAPI UI

# HTTP reset (for debugging)
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task": "easy"}'

# Run baseline
python inference.py

---

3. Docker

docker build -t sfd .
docker run -p 7860:7860 --rm sfd

# Run inference against docker
ENV_URL=http://localhost:7860 python inference.py

---

4. HF Spaces Deployment

4a. What to upload

models.py
client.py
server/
data/
openenv.yaml
Dockerfile
requirements.txt
README.md
inference.py

Do NOT upload: .venv/, __pycache__/, .git/

4b. Create Space (Docker SDK)

Go to https://huggingface.co/new-space → Select Docker SDK → Create.

Or via CLI:

pip install huggingface_hub
python -c "
from huggingface_hub import HfApi
HfApi().create_repo('YOUR_USERNAME/silent-failure-detector', repo_type='space', space_sdk='docker')
"

4c. Push

git init
git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/silent-failure-detector
git add models.py client.py server/ data/ openenv.yaml Dockerfile requirements.txt README.md inference.py
git commit -m "initial"
git push origin main

4d. Space Secrets

Settings → Variables and secrets → Add:

Name	Value
HF_TOKEN	Your HF token
API_BASE_URL	LLM provider base URL
MODEL_NAME	Model name

4e. Verify

SPACE="https://YOUR_USERNAME-silent-failure-detector.hf.space"
curl -f "$SPACE/health"

# WebSocket reset (what the SDK uses):
python3 -c "
import asyncio
from client import SilentFailureEnv
async def test():
    async with SilentFailureEnv(base_url='$SPACE') as env:
        r = await env.reset(task='easy')
        print('session_id:', r.observation.session_id)
        print('total_steps:', r.observation.total_steps)
asyncio.run(test())
"

---

5. Running Inference Against HF Space

export ENV_URL="https://your-username-silent-failure-detector.hf.space"
export API_BASE_URL="https://api.openai.com/v1"
export MODEL_NAME="gpt-4o-mini"
export HF_TOKEN="sk-..."

python inference.py

Expected runtime: under 10 minutes on vcpu=2, mem=8GB.

---

6. Pre-Submission Validation Checklist

• GET /health returns {"status":"healthy"}
• WebSocket /ws accepts reset/step messages
• POST /reset works (HTTP fallback)
• POST /step returns reward in [0.0, 1.0] and done bool
• All 3 tasks (easy/medium/hard) produce graded scores
• python inference.py runs to completion, logs [START]/[STEP]/[END]
• docker build -t sfd . && docker run -p 7860:7860 sfd works
• openenv.yaml present and valid

---

7. How the OpenEnv SDK Is Used

The server inherits from the actual OpenEnv Environment ABC:

from openenv.core.env_server.interfaces import Environment
from openenv.core.env_server import create_fastapi_app

class SilentFailureEnvironment(Environment[SFDAction, SFDObservation, SFDState]):
    def reset(self, task="easy", **kwargs) -> SFDObservation: ...
    def step(self, action: SFDAction, **kwargs) -> SFDObservation: ...
    @property
    def state(self) -> SFDState: ...

app = create_fastapi_app(SilentFailureEnvironment, SFDAction, SFDObservation)

create_fastapi_app automatically provides:

• GET /health
• POST /reset, POST /step, GET /state (HTTP)
• WS /ws (WebSocket — primary transport used by the client)
• GET /docs (OpenAPI UI)
• GET /web (interactive web UI)

The client inherits from EnvClient and uses WebSocket:

from openenv.core.env_client import EnvClient

class SilentFailureEnv(EnvClient[SFDAction, SFDObservation, SFDState]):
    def _step_payload(self, action): return {"message": action.message}
    def _parse_result(self, payload): return StepResult(...)
    def _parse_state(self, payload): return SFDState(...)

Usage is always async:

async with SilentFailureEnv(base_url="http://localhost:7860") as env:
    result = await env.reset(task="medium")
    result = await env.step(SFDAction(message='{"flag":true,"confidence":0.9}'))

---

8. Extending the Dataset

Edit scripts/generate_dataset.py, add items to ITEMS:

{
    "id": "h_wc_013",           # unique: {difficulty}_{label_abbrev}_{num}
    "domain": "finance",        # medicine|law|finance|coding|science|geography|history
    "difficulty": "hard",       # easy|medium|hard
    "label": "wrong_confident", # wrong_confident|correct|wrong_uncertain|correct_misleading
    "question": "...",
    "ai_response": "...",
}

Minimum pool sizes needed: easy ≥10, medium ≥20, hard ≥30.

Target label balance per tier: ~40% wrong_confident, ~35% correct, ~15% wrong_uncertain, ~10% correct_misleading.

Then: python scripts/generate_dataset.py

---

9. Training an RL Agent with GRPO (TRL)

Install TRL:

pip install git+https://github.com/huggingface/trl.git

The environment is stateless per session_id and supports concurrent sessions (SUPPORTS_CONCURRENT_SESSIONS = True).

Recommended curriculum:

1. Warm up on easy until recall > 0.7
2. Train primarily on medium
3. Use hard as evaluation benchmark only

Reward is sparse (terminal only). For faster GRPO convergence, consider shaping: give +0.1 for each correctly classified item using the known labels as a secondary signal during training rollouts.

---

10. Grader Verification

from server.grader import compute_reward

labels = ["wrong_confident", "correct", "wrong_confident", "wrong_uncertain"]
flags  = [True, False, False, False]
confs  = [0.9, 0.1, 0.3, 0.2]

reward = compute_reward(labels, flags, confs)
print(reward)
# TP=1, FN=1, TN=2, FP=0
# recall=0.5, specificity=1.0, base=0.5
# calibration_bonus~0.1, reward~0.55
assert 0.0 <= reward <= 1.0