developer.md

# Developer Guide

---

## 1. Environment Setup

```bash
python -m venv .venv && source .venv/bin/activate
pip install git+https://github.com/meta-pytorch/OpenEnv.git
pip install -r requirements.txt

export API_BASE_URL="https://api.openai.com/v1"
export MODEL_NAME="gpt-4o-mini"
export HF_TOKEN="hf_..."
export ENV_URL="http://localhost:7860"
```

---

## 2. Local Dev

```bash
# Start server
PYTHONPATH=. uvicorn server.app:app --host 0.0.0.0 --port 7860 --reload

# Health check
curl http://localhost:7860/health        # {"status":"healthy"}
curl http://localhost:7860/docs          # OpenAPI UI

# HTTP reset (for debugging)
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task": "easy"}'

# Run baseline
python inference.py
```

---

## 3. Docker

```bash
docker build -t sfd .
docker run -p 7860:7860 --rm sfd

# Run inference against docker
ENV_URL=http://localhost:7860 python inference.py
```

---

## 4. HF Spaces Deployment

### 4a. What to upload

```
models.py
client.py
server/
data/
openenv.yaml
Dockerfile
requirements.txt
README.md
inference.py
```

Do NOT upload: `.venv/`, `__pycache__/`, `.git/`

### 4b. Create Space (Docker SDK)

Go to https://huggingface.co/new-space → Select **Docker** SDK → Create.

Or via CLI:
```bash
pip install huggingface_hub
python -c "
from huggingface_hub import HfApi
HfApi().create_repo('YOUR_USERNAME/silent-failure-detector', repo_type='space', space_sdk='docker')
"
```

### 4c. Push

```bash
git init
git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/silent-failure-detector
git add models.py client.py server/ data/ openenv.yaml Dockerfile requirements.txt README.md inference.py
git commit -m "initial"
git push origin main
```

### 4d. Space Secrets

Settings → Variables and secrets → Add:

| Name | Value |
|---|---|
| `HF_TOKEN` | Your HF token |
| `API_BASE_URL` | LLM provider base URL |
| `MODEL_NAME` | Model name |

### 4e. Verify

```bash
SPACE="https://YOUR_USERNAME-silent-failure-detector.hf.space"
curl -f "$SPACE/health"

# WebSocket reset (what the SDK uses):
python3 -c "
import asyncio
from client import SilentFailureEnv
async def test():
    async with SilentFailureEnv(base_url='$SPACE') as env:
        r = await env.reset(task='easy')
        print('session_id:', r.observation.session_id)
        print('total_steps:', r.observation.total_steps)
asyncio.run(test())
"
```

---

## 5. Running Inference Against HF Space

```bash
export ENV_URL="https://your-username-silent-failure-detector.hf.space"
export API_BASE_URL="https://api.openai.com/v1"
export MODEL_NAME="gpt-4o-mini"
export HF_TOKEN="sk-..."

python inference.py
```

Expected runtime: under 10 minutes on vcpu=2, mem=8GB.

---

## 6. Pre-Submission Validation Checklist

- [ ] `GET /health` returns `{"status":"healthy"}`
- [ ] WebSocket `/ws` accepts reset/step messages
- [ ] `POST /reset` works (HTTP fallback)
- [ ] `POST /step` returns reward in [0.0, 1.0] and done bool
- [ ] All 3 tasks (easy/medium/hard) produce graded scores
- [ ] `python inference.py` runs to completion, logs [START]/[STEP]/[END]
- [ ] `docker build -t sfd . && docker run -p 7860:7860 sfd` works
- [ ] `openenv.yaml` present and valid

---

## 7. How the OpenEnv SDK Is Used

The server inherits from the actual OpenEnv `Environment` ABC:

```python
from openenv.core.env_server.interfaces import Environment
from openenv.core.env_server import create_fastapi_app

class SilentFailureEnvironment(Environment[SFDAction, SFDObservation, SFDState]):
    def reset(self, task="easy", **kwargs) -> SFDObservation: ...
    def step(self, action: SFDAction, **kwargs) -> SFDObservation: ...
    @property
    def state(self) -> SFDState: ...

app = create_fastapi_app(SilentFailureEnvironment, SFDAction, SFDObservation)
```

`create_fastapi_app` automatically provides:
- `GET /health`
- `POST /reset`, `POST /step`, `GET /state` (HTTP)
- `WS /ws` (WebSocket — primary transport used by the client)
- `GET /docs` (OpenAPI UI)
- `GET /web` (interactive web UI)

The client inherits from `EnvClient` and uses WebSocket:

```python
from openenv.core.env_client import EnvClient

class SilentFailureEnv(EnvClient[SFDAction, SFDObservation, SFDState]):
    def _step_payload(self, action): return {"message": action.message}
    def _parse_result(self, payload): return StepResult(...)
    def _parse_state(self, payload): return SFDState(...)
```

Usage is always async:
```python
async with SilentFailureEnv(base_url="http://localhost:7860") as env:
    result = await env.reset(task="medium")
    result = await env.step(SFDAction(message='{"flag":true,"confidence":0.9}'))
```

---

## 8. Extending the Dataset

Edit `scripts/generate_dataset.py`, add items to `ITEMS`:

```python
{
    "id": "h_wc_013",           # unique: {difficulty}_{label_abbrev}_{num}
    "domain": "finance",        # medicine|law|finance|coding|science|geography|history
    "difficulty": "hard",       # easy|medium|hard
    "label": "wrong_confident", # wrong_confident|correct|wrong_uncertain|correct_misleading
    "question": "...",
    "ai_response": "...",
}
```

Minimum pool sizes needed: easy ≥10, medium ≥20, hard ≥30.

Target label balance per tier: ~40% wrong_confident, ~35% correct, ~15% wrong_uncertain, ~10% correct_misleading.

Then: `python scripts/generate_dataset.py`

---

## 9. Training an RL Agent with GRPO (TRL)

Install TRL:
```bash
pip install git+https://github.com/huggingface/trl.git
```

The environment is stateless per `session_id` and supports concurrent sessions (`SUPPORTS_CONCURRENT_SESSIONS = True`).

Recommended curriculum:
1. Warm up on `easy` until recall > 0.7
2. Train primarily on `medium`
3. Use `hard` as evaluation benchmark only

Reward is sparse (terminal only). For faster GRPO convergence, consider shaping: give +0.1 for each correctly classified item using the known labels as a secondary signal during training rollouts.

---

## 10. Grader Verification

```python
from server.grader import compute_reward

labels = ["wrong_confident", "correct", "wrong_confident", "wrong_uncertain"]
flags  = [True, False, False, False]
confs  = [0.9, 0.1, 0.3, 0.2]

reward = compute_reward(labels, flags, confs)
print(reward)
# TP=1, FN=1, TN=2, FP=0
# recall=0.5, specificity=1.0, base=0.5
# calibration_bonus~0.1, reward~0.55
assert 0.0 <= reward <= 1.0
```