USAGE_GUIDE.md

# HCM:21 Usage Guide

How to run, explore, and demo the HCM:21 environment.

---

## 1. Live HuggingFace Space

The environment is deployed and running at:

**https://paretooptimal-hcm21.hf.space**

No setup required — open a browser and start interacting immediately.

| Endpoint | URL | Purpose |
|----------|-----|---------|
| Web Interface | [/web](https://paretooptimal-hcm21.hf.space/web) | Interactive dashboard for manual play |
| API Docs | [/docs](https://paretooptimal-hcm21.hf.space/docs) | Swagger UI with all endpoints |
| Health Check | [/health](https://paretooptimal-hcm21.hf.space/health) | Returns `{"status":"healthy"}` |
| Schema | [/schema](https://paretooptimal-hcm21.hf.space/schema) | Action/Observation/State JSON schemas |
| Metadata | [/metadata](https://paretooptimal-hcm21.hf.space/metadata) | Environment name, version, description |

> **Note:** If the Space has been idle, it may take 30-60 seconds to wake up on first request.

---

## 2. Web Interface — Best Demo Experience

Open **https://paretooptimal-hcm21.hf.space/web** for the interactive dashboard.

### Layout

The interface is a split-pane design:

- **Left pane** — Action controls: buttons to Reset/Step/Get State, plus a form to build actions
- **Right pane** — Observations: current state, JSON observation display, and a scrollable action history log

### Recommended Demo Walkthrough

Follow this sequence to showcase all four HCM:21 phases in a single quarter:

#### Step 1: Reset the Environment

Click **Reset Environment**. You'll see the initial observation with company data — 300 employees across 5 departments, baseline HCVA/HCROI metrics, and available scanning actions.

#### Step 2: Scanning Phase (gather intelligence)

Submit these actions using the form (set **Action Type** dropdown, fill relevant fields, click **Step**):

| # | Action Type | Department | Metric Name | What You'll See |
|---|------------|------------|-------------|-----------------|
| 1 | `query_department` | Engineering | — | Headcount, avg salary, avg performance, flight risk |
| 2 | `query_department` | Sales | — | Same metrics for Sales |
| 3 | `calculate_metric` | — | hcva | Human Capital Value Added across the company |
| 4 | `review_financials` | — | — | Revenue, employment costs, HR budget, P&L |

> After 2+ scanning actions, `advance_phase` becomes available.

#### Step 3: Advance to Planning

| # | Action Type | Notes |
|---|------------|-------|
| 5 | `advance_phase` | Moves from Scanning → Planning |

#### Step 4: Planning Phase (set strategy)

| # | Action Type | Department | Amount/Count | What It Does |
|---|------------|------------|--------------|--------------|
| 6 | `set_hiring_target` | Engineering | count: 3 | Plan to hire 3 engineers |
| 7 | `set_training_budget` | Engineering | amount: 50000 | Allocate $50K for engineering L&D |
| 8 | `set_compensation_policy` | Sales | amount: 3.0 | 3% salary adjustment for Sales |
| 9 | `set_retention_program` | Engineering | amount: 20000 | $20K retention initiative |

#### Step 5: Advance to Producing

| # | Action Type |
|---|------------|
| 10 | `advance_phase` |

#### Step 6: Producing Phase (execute decisions)

| # | Action Type | Department | Amount/Count |
|---|------------|------------|--------------|
| 11 | `execute_hiring` | Engineering | count: 3 |
| 12 | `execute_training` | Engineering | amount: 20 (hours) |

#### Step 7: Advance to Controlling

| # | Action Type |
|---|------------|
| 13 | `advance_phase` |

#### Step 8: Controlling Phase (measure & advance)

| # | Action Type | What Happens |
|---|------------|--------------|
| 14 | `submit_report` | Quarterly HR report with metric changes |
| 15 | `advance_quarter` | Quarter ends, reward computed, stochastic events may fire |

**Watch the right pane** — the observation after `advance_quarter` shows:
- The quarterly reward (e.g., 0.2374)
- Any stochastic events that fired (market downturn, competitor poaching, etc.)
- Updated metrics and available actions for the next quarter

Repeat for all 6 quarters to complete a full episode. The final `advance_quarter` in Q6 returns the episode score (0-1).

### What to Point Out During a Demo

- **Phase enforcement**: Try submitting `execute_hiring` during Scanning — it will be rejected. The HCM:21 cycle (Scan → Plan → Produce → Control) is enforced.
- **Sparse rewards**: Rewards only appear at quarter boundaries via `advance_quarter`. No intermediate feedback.
- **Stochastic events**: Each quarter, 0-2 random events fire (market crash, exec departure, competitor poaching). Watch how they cascade.
- **Growing state**: Click **Get State** to see the full state object grow with metric history, active initiatives, and event logs across quarters.
- **Score calibration**: Random agents score ~0.15-0.25. A thoughtful human playing through the web UI should reach 0.40-0.60.

---

## 3. Running Locally

### Option A: Python (development)

```bash
# Clone and install
git clone https://github.com/n8mauer/HCM-21.git
cd HCM-21
pip install -e ".[server]"

# Start the server
uvicorn hr_env.server.app:app --host 0.0.0.0 --port 8000

# Open the web interface
# → http://localhost:8000/web
```

### Option B: Docker

```bash
docker build -t hcm21 .
docker run -p 8000:8000 hcm21

# → http://localhost:8000/web
```

### Option C: OpenEnv CLI

```bash
pip install openenv-core
openenv validate    # Checks compliance
openenv push        # Deploys to HF Spaces
```

---

## 4. CLI Agent (Autonomous Play)

Run the sample Claude-based agent for a full autonomous 6-quarter episode:

```bash
# Install agent dependencies
pip install -e ".[agent]"

# Set your API key
export ANTHROPIC_API_KEY=sk-ant-...

# Run against local server
python -m agent.run --url ws://localhost:8000 --model claude-sonnet-4-6 --seed 42

# Or run against the live HF Space
python -m agent.run --url wss://paretooptimal-hcm21.hf.space --model claude-sonnet-4-6 --seed 42
```

The agent will:
1. Scan all departments and calculate baseline metrics
2. Formulate a multi-quarter strategy
3. Execute hiring, training, compensation, and retention decisions
4. Adapt to stochastic events (market crashes, competitor poaching)
5. Summarize each quarter to manage context window limits
6. Print step-by-step actions, rewards, and the final episode score

Example output:
```
Q1 Scanning | query_department Engineering | headcount: 62, avg_perf: 3.4, flight_risk: 0.18
Q1 Planning | set_hiring_target Engineering count=5
Q1 Producing | execute_hiring Engineering count=5
Q1 Controlling | advance_quarter | Reward: 0.2374 | Event: competitor_poaching (Sales)
Q2 Scanning | query_department Sales | flight_risk: 0.31 (↑)
...
Final Score: 0.6847 (strong strategic agent range)
```

---

## 5. Training Notebook (Google Colab)

Train a small LLM to play as CHCO using reinforcement learning.

### Setup

1. Open `notebooks/hcm21_trl_training.ipynb` in Google Colab
2. Select **Runtime → Change runtime type → T4 GPU** (free tier works)
3. Run all cells

### What It Does

| Component | Detail |
|-----------|--------|
| Model | Qwen3-0.6B via Unsloth (4-bit QLoRA) |
| Algorithm | GRPO (Group Relative Policy Optimization) from HuggingFace TRL |
| Environment | Connects to the live HF Space via WebSocket |
| Dataset | 20 episode prompts with different seeds |
| Training | batch_size=2, gradient_accumulation=4, ~30 min on T4 |

### Key Cells

1. **Install Dependencies** — `pip install unsloth trl openenv-core`
2. **Load Model** — Qwen3-0.6B with LoRA (r=16) on all attention + MLP layers
3. **Environment Wrapper** — `HCM21Env` class wraps `GenericEnvClient` (WebSocket) for stateful sessions
4. **Training Dataset** — 20 prompts describing the CHCO role with different episode seeds
5. **GRPO Training** — TRL's `GRPOTrainer` with `environment_factory=HCM21Env`
6. **Evaluation** — Runs a full 6-quarter episode with the trained model

### Output

The trained LoRA adapter is saved to `./hcm21-agent-lora/` and can be pushed to HuggingFace Hub.

---

## 6. API Reference (Custom Integrations)

### REST Endpoints (Stateless)

> **Important:** REST endpoints are stateless — each `/step` call creates a fresh environment. Use WebSocket for multi-step episodes.

```bash
# Health check
curl https://paretooptimal-hcm21.hf.space/health

# Get action/observation schemas
curl https://paretooptimal-hcm21.hf.space/schema

# Reset (stateless — returns initial observation)
curl -X POST https://paretooptimal-hcm21.hf.space/reset \
  -H "Content-Type: application/json" \
  -d '{"seed": 42}'

# Step (stateless — wraps action in {"action": {...}})
curl -X POST https://paretooptimal-hcm21.hf.space/step \
  -H "Content-Type: application/json" \
  -d '{"action": {"action_type": "query_department", "department": "Engineering"}}'
```

### WebSocket (Stateful Sessions)

For multi-step episodes, use the WebSocket endpoint which maintains state across calls:

```python
from openenv.core import GenericEnvClient, GenericAction

client = GenericEnvClient(base_url="https://paretooptimal-hcm21.hf.space")
client.connect()

# Reset
result = client.reset(seed=42)
print(result.observation)

# Take actions in sequence (state persists)
action = GenericAction(action_type="query_department", department="Engineering")
result = client.step(action)
print(result.observation)

# Continue stepping through phases and quarters...
client.disconnect()
```

### Python Direct (No Server)

For local testing without a server:

```python
from hr_env.server.environment import HRProductivityEnvironment
from hr_env.models import HRAction

env = HRProductivityEnvironment()
obs = env.reset(seed=42)
print(obs.message)

# Take actions directly
obs = env.step(HRAction(action_type="query_department", department="Engineering"))
print(obs.data)
```

---

## 7. Scenario Variants

Four pre-built scenarios test different strategic challenges:

| Scenario | Seed | Challenge | Target Score |
|----------|------|-----------|-------------|
| High Turnover | 42 | Engineering hemorrhaging talent | 0.65 |
| Budget Cuts | 123 | HR budget slashed 50% | 0.60 |
| Rapid Growth | 456 | Scale workforce while maintaining quality | 0.65 |
| Balanced Optimization | 789 | Below-average metrics across the board | 0.60 |

Use the seed when resetting to reproduce a specific scenario:

```python
obs = env.reset(seed=42)   # High turnover scenario
obs = env.reset(seed=123)  # Budget cuts scenario
```

---

## Quick Reference

| What You Want | How To Do It |
|---------------|--------------|
| Try it now (no setup) | Open [/web](https://paretooptimal-hcm21.hf.space/web) in your browser |
| Run an AI agent | `python -m agent.run --url ws://localhost:8000` |
| Train a model | Open `notebooks/hcm21_trl_training.ipynb` in Colab |
| Build a custom agent | Use `GenericEnvClient` (WebSocket) — see Section 6 |
| Run locally | `pip install -e ".[server]" && uvicorn hr_env.server.app:app` |
| View API docs | Open [/docs](https://paretooptimal-hcm21.hf.space/docs) |