Add usage guide with demo walkthrough and API reference

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

USAGE_GUIDE.md

@@ -0,0 +1,322 @@
+# HCM:21 Usage Guide
+
+How to run, explore, and demo the HCM:21 environment.
+
+---
+
+## 1. Live HuggingFace Space
+
+The environment is deployed and running at:
+
+**https://paretooptimal-hcm21.hf.space**
+
+No setup required — open a browser and start interacting immediately.
+
+| Endpoint | URL | Purpose |
+|----------|-----|---------|
+| Web Interface | [/web](https://paretooptimal-hcm21.hf.space/web) | Interactive dashboard for manual play |
+| API Docs | [/docs](https://paretooptimal-hcm21.hf.space/docs) | Swagger UI with all endpoints |
+| Health Check | [/health](https://paretooptimal-hcm21.hf.space/health) | Returns `{"status":"healthy"}` |
+| Schema | [/schema](https://paretooptimal-hcm21.hf.space/schema) | Action/Observation/State JSON schemas |
+| Metadata | [/metadata](https://paretooptimal-hcm21.hf.space/metadata) | Environment name, version, description |
+
+> **Note:** If the Space has been idle, it may take 30-60 seconds to wake up on first request.
+
+---
+
+## 2. Web Interface — Best Demo Experience
+
+Open **https://paretooptimal-hcm21.hf.space/web** for the interactive dashboard.
+
+### Layout
+
+The interface is a split-pane design:
+
+- **Left pane** — Action controls: buttons to Reset/Step/Get State, plus a form to build actions
+- **Right pane** — Observations: current state, JSON observation display, and a scrollable action history log
+
+### Recommended Demo Walkthrough
+
+Follow this sequence to showcase all four HCM:21 phases in a single quarter:
+
+#### Step 1: Reset the Environment
+
+Click **Reset Environment**. You'll see the initial observation with company data — 300 employees across 5 departments, baseline HCVA/HCROI metrics, and available scanning actions.
+
+#### Step 2: Scanning Phase (gather intelligence)
+
+Submit these actions using the form (set **Action Type** dropdown, fill relevant fields, click **Step**):
+
+| # | Action Type | Department | Metric Name | What You'll See |
+|---|------------|------------|-------------|-----------------|
+| 1 | `query_department` | Engineering | — | Headcount, avg salary, avg performance, flight risk |
+| 2 | `query_department` | Sales | — | Same metrics for Sales |
+| 3 | `calculate_metric` | — | hcva | Human Capital Value Added across the company |
+| 4 | `review_financials` | — | — | Revenue, employment costs, HR budget, P&L |
+
+> After 2+ scanning actions, `advance_phase` becomes available.
+
+#### Step 3: Advance to Planning
+
+| # | Action Type | Notes |
+|---|------------|-------|
+| 5 | `advance_phase` | Moves from Scanning → Planning |
+
+#### Step 4: Planning Phase (set strategy)
+
+| # | Action Type | Department | Amount/Count | What It Does |
+|---|------------|------------|--------------|--------------|
+| 6 | `set_hiring_target` | Engineering | count: 3 | Plan to hire 3 engineers |
+| 7 | `set_training_budget` | Engineering | amount: 50000 | Allocate $50K for engineering L&D |
+| 8 | `set_compensation_policy` | Sales | amount: 3.0 | 3% salary adjustment for Sales |
+| 9 | `set_retention_program` | Engineering | amount: 20000 | $20K retention initiative |
+
+#### Step 5: Advance to Producing
+
+| # | Action Type |
+|---|------------|
+| 10 | `advance_phase` |
+
+#### Step 6: Producing Phase (execute decisions)
+
+| # | Action Type | Department | Amount/Count |
+|---|------------|------------|--------------|
+| 11 | `execute_hiring` | Engineering | count: 3 |
+| 12 | `execute_training` | Engineering | amount: 20 (hours) |
+
+#### Step 7: Advance to Controlling
+
+| # | Action Type |
+|---|------------|
+| 13 | `advance_phase` |
+
+#### Step 8: Controlling Phase (measure & advance)
+
+| # | Action Type | What Happens |
+|---|------------|--------------|
+| 14 | `submit_report` | Quarterly HR report with metric changes |
+| 15 | `advance_quarter` | Quarter ends, reward computed, stochastic events may fire |
+
+**Watch the right pane** — the observation after `advance_quarter` shows:
+- The quarterly reward (e.g., 0.2374)
+- Any stochastic events that fired (market downturn, competitor poaching, etc.)
+- Updated metrics and available actions for the next quarter
+
+Repeat for all 6 quarters to complete a full episode. The final `advance_quarter` in Q6 returns the episode score (0-1).
+
+### What to Point Out During a Demo
+
+- **Phase enforcement**: Try submitting `execute_hiring` during Scanning — it will be rejected. The HCM:21 cycle (Scan → Plan → Produce → Control) is enforced.
+- **Sparse rewards**: Rewards only appear at quarter boundaries via `advance_quarter`. No intermediate feedback.
+- **Stochastic events**: Each quarter, 0-2 random events fire (market crash, exec departure, competitor poaching). Watch how they cascade.
+- **Growing state**: Click **Get State** to see the full state object grow with metric history, active initiatives, and event logs across quarters.
+- **Score calibration**: Random agents score ~0.15-0.25. A thoughtful human playing through the web UI should reach 0.40-0.60.
+
+---
+
+## 3. Running Locally
+
+### Option A: Python (development)
+
+```bash
+# Clone and install
+git clone https://github.com/n8mauer/HCM-21.git
+cd HCM-21
+pip install -e ".[server]"
+
+# Start the server
+uvicorn hr_env.server.app:app --host 0.0.0.0 --port 8000
+
+# Open the web interface
+# → http://localhost:8000/web
+```
+
+### Option B: Docker
+
+```bash
+docker build -t hcm21 .
+docker run -p 8000:8000 hcm21
+
+# → http://localhost:8000/web
+```
+
+### Option C: OpenEnv CLI
+
+```bash
+pip install openenv-core
+openenv validate    # Checks compliance
+openenv push        # Deploys to HF Spaces
+```
+
+---
+
+## 4. CLI Agent (Autonomous Play)
+
+Run the sample Claude-based agent for a full autonomous 6-quarter episode:
+
+```bash
+# Install agent dependencies
+pip install -e ".[agent]"
+
+# Set your API key
+export ANTHROPIC_API_KEY=sk-ant-...
+
+# Run against local server
+python -m agent.run --url ws://localhost:8000 --model claude-sonnet-4-6 --seed 42
+
+# Or run against the live HF Space
+python -m agent.run --url wss://paretooptimal-hcm21.hf.space --model claude-sonnet-4-6 --seed 42
+```
+
+The agent will:
+1. Scan all departments and calculate baseline metrics
+2. Formulate a multi-quarter strategy
+3. Execute hiring, training, compensation, and retention decisions
+4. Adapt to stochastic events (market crashes, competitor poaching)
+5. Summarize each quarter to manage context window limits
+6. Print step-by-step actions, rewards, and the final episode score
+
+Example output:
+```
+Q1 Scanning | query_department Engineering | headcount: 62, avg_perf: 3.4, flight_risk: 0.18
+Q1 Planning | set_hiring_target Engineering count=5
+Q1 Producing | execute_hiring Engineering count=5
+Q1 Controlling | advance_quarter | Reward: 0.2374 | Event: competitor_poaching (Sales)
+Q2 Scanning | query_department Sales | flight_risk: 0.31 (↑)
+...
+Final Score: 0.6847 (strong strategic agent range)
+```
+
+---
+
+## 5. Training Notebook (Google Colab)
+
+Train a small LLM to play as CHCO using reinforcement learning.
+
+### Setup
+
+1. Open `notebooks/hcm21_trl_training.ipynb` in Google Colab
+2. Select **Runtime → Change runtime type → T4 GPU** (free tier works)
+3. Run all cells
+
+### What It Does
+
+| Component | Detail |
+|-----------|--------|
+| Model | Qwen3-0.6B via Unsloth (4-bit QLoRA) |
+| Algorithm | GRPO (Group Relative Policy Optimization) from HuggingFace TRL |
+| Environment | Connects to the live HF Space via WebSocket |
+| Dataset | 20 episode prompts with different seeds |
+| Training | batch_size=2, gradient_accumulation=4, ~30 min on T4 |
+
+### Key Cells
+
+1. **Install Dependencies** — `pip install unsloth trl openenv-core`
+2. **Load Model** — Qwen3-0.6B with LoRA (r=16) on all attention + MLP layers
+3. **Environment Wrapper** — `HCM21Env` class wraps `GenericEnvClient` (WebSocket) for stateful sessions
+4. **Training Dataset** — 20 prompts describing the CHCO role with different episode seeds
+5. **GRPO Training** — TRL's `GRPOTrainer` with `environment_factory=HCM21Env`
+6. **Evaluation** — Runs a full 6-quarter episode with the trained model
+
+### Output
+
+The trained LoRA adapter is saved to `./hcm21-agent-lora/` and can be pushed to HuggingFace Hub.
+
+---
+
+## 6. API Reference (Custom Integrations)
+
+### REST Endpoints (Stateless)
+
+> **Important:** REST endpoints are stateless — each `/step` call creates a fresh environment. Use WebSocket for multi-step episodes.
+
+```bash
+# Health check
+curl https://paretooptimal-hcm21.hf.space/health
+
+# Get action/observation schemas
+curl https://paretooptimal-hcm21.hf.space/schema
+
+# Reset (stateless — returns initial observation)
+curl -X POST https://paretooptimal-hcm21.hf.space/reset \
+  -H "Content-Type: application/json" \
+  -d '{"seed": 42}'
+
+# Step (stateless — wraps action in {"action": {...}})
+curl -X POST https://paretooptimal-hcm21.hf.space/step \
+  -H "Content-Type: application/json" \
+  -d '{"action": {"action_type": "query_department", "department": "Engineering"}}'
+```
+
+### WebSocket (Stateful Sessions)
+
+For multi-step episodes, use the WebSocket endpoint which maintains state across calls:
+
+```python
+from openenv.core import GenericEnvClient, GenericAction
+
+client = GenericEnvClient(base_url="https://paretooptimal-hcm21.hf.space")
+client.connect()
+
+# Reset
+result = client.reset(seed=42)
+print(result.observation)
+
+# Take actions in sequence (state persists)
+action = GenericAction(action_type="query_department", department="Engineering")
+result = client.step(action)
+print(result.observation)
+
+# Continue stepping through phases and quarters...
+client.disconnect()
+```
+
+### Python Direct (No Server)
+
+For local testing without a server:
+
+```python
+from hr_env.server.environment import HRProductivityEnvironment
+from hr_env.models import HRAction
+
+env = HRProductivityEnvironment()
+obs = env.reset(seed=42)
+print(obs.message)
+
+# Take actions directly
+obs = env.step(HRAction(action_type="query_department", department="Engineering"))
+print(obs.data)
+```
+
+---
+
+## 7. Scenario Variants
+
+Four pre-built scenarios test different strategic challenges:
+
+| Scenario | Seed | Challenge | Target Score |
+|----------|------|-----------|-------------|
+| High Turnover | 42 | Engineering hemorrhaging talent | 0.65 |
+| Budget Cuts | 123 | HR budget slashed 50% | 0.60 |
+| Rapid Growth | 456 | Scale workforce while maintaining quality | 0.65 |
+| Balanced Optimization | 789 | Below-average metrics across the board | 0.60 |
+
+Use the seed when resetting to reproduce a specific scenario:
+
+```python
+obs = env.reset(seed=42)   # High turnover scenario
+obs = env.reset(seed=123)  # Budget cuts scenario
+```
+
+---
+
+## Quick Reference
+
+| What You Want | How To Do It |
+|---------------|--------------|
+| Try it now (no setup) | Open [/web](https://paretooptimal-hcm21.hf.space/web) in your browser |
+| Run an AI agent | `python -m agent.run --url ws://localhost:8000` |
+| Train a model | Open `notebooks/hcm21_trl_training.ipynb` in Colab |
+| Build a custom agent | Use `GenericEnvClient` (WebSocket) — see Section 6 |
+| Run locally | `pip install -e ".[server]" && uvicorn hr_env.server.app:app` |
+| View API docs | Open [/docs](https://paretooptimal-hcm21.hf.space/docs) |