USAGE_GUIDE.md

HCM:21 Usage Guide

How to run, explore, and demo the HCM:21 environment.

---

1. Live HuggingFace Space

The environment is deployed and running at:

https://paretooptimal-hcm21.hf.space

No setup required — open a browser and start interacting immediately.

Endpoint	URL	Purpose
Web Interface	/web	Interactive dashboard for manual play
API Docs	/docs	Swagger UI with all endpoints
Health Check	/health	Returns {"status":"healthy"}
Schema	/schema	Action/Observation/State JSON schemas
Metadata	/metadata	Environment name, version, description

Note: If the Space has been idle, it may take 30-60 seconds to wake up on first request.

---

2. Web Interface — Best Demo Experience

Open https://paretooptimal-hcm21.hf.space/web for the interactive dashboard.

Layout

The interface is a split-pane design:

• Left pane — Action controls: buttons to Reset/Step/Get State, plus a form to build actions
• Right pane — Observations: current state, JSON observation display, and a scrollable action history log

Recommended Demo Walkthrough

Follow this sequence to showcase all four HCM:21 phases in a single quarter:

Step 1: Reset the Environment

Click Reset Environment. You'll see the initial observation with company data — 300 employees across 5 departments, baseline HCVA/HCROI metrics, and available scanning actions.

Step 2: Scanning Phase (gather intelligence)

Submit these actions using the form (set Action Type dropdown, fill relevant fields, click Step):

#	Action Type	Department	Metric Name	What You'll See
1	query_department	Engineering	—	Headcount, avg salary, avg performance, flight risk
2	query_department	Sales	—	Same metrics for Sales
3	calculate_metric	—	hcva	Human Capital Value Added across the company
4	review_financials	—	—	Revenue, employment costs, HR budget, P&L

After 2+ scanning actions, advance_phase becomes available.

Step 3: Advance to Planning

#	Action Type	Notes
5	advance_phase	Moves from Scanning → Planning

Step 4: Planning Phase (set strategy)

#	Action Type	Department	Amount/Count	What It Does
6	set_hiring_target	Engineering	count: 3	Plan to hire 3 engineers
7	set_training_budget	Engineering	amount: 50000	Allocate $50K for engineering L&D
8	set_compensation_policy	Sales	amount: 3.0	3% salary adjustment for Sales
9	set_retention_program	Engineering	amount: 20000	$20K retention initiative

Step 5: Advance to Producing

#	Action Type
10	advance_phase

Step 6: Producing Phase (execute decisions)

#	Action Type	Department	Amount/Count
11	execute_hiring	Engineering	count: 3
12	execute_training	Engineering	amount: 20 (hours)

Step 7: Advance to Controlling

#	Action Type
13	advance_phase

Step 8: Controlling Phase (measure & advance)

#	Action Type	What Happens
14	submit_report	Quarterly HR report with metric changes
15	advance_quarter	Quarter ends, reward computed, stochastic events may fire

Watch the right pane — the observation after advance_quarter shows:

• The quarterly reward (e.g., 0.2374)
• Any stochastic events that fired (market downturn, competitor poaching, etc.)
• Updated metrics and available actions for the next quarter

Repeat for all 6 quarters to complete a full episode. The final advance_quarter in Q6 returns the episode score (0-1).

What to Point Out During a Demo

• Phase enforcement: Try submitting execute_hiring during Scanning — it will be rejected. The HCM:21 cycle (Scan → Plan → Produce → Control) is enforced.
• Sparse rewards: Rewards only appear at quarter boundaries via advance_quarter. No intermediate feedback.
• Stochastic events: Each quarter, 0-2 random events fire (market crash, exec departure, competitor poaching). Watch how they cascade.
• Growing state: Click Get State to see the full state object grow with metric history, active initiatives, and event logs across quarters.
• Score calibration: Random agents score ~0.15-0.25. A thoughtful human playing through the web UI should reach 0.40-0.60.

---

3. Running Locally

Option A: Python (development)

# Clone and install
git clone https://github.com/n8mauer/HCM-21.git
cd HCM-21
pip install -e ".[server]"

# Start the server
uvicorn hr_env.server.app:app --host 0.0.0.0 --port 8000

# Open the web interface
# → http://localhost:8000/web

Option B: Docker

docker build -t hcm21 .
docker run -p 8000:8000 hcm21

# → http://localhost:8000/web

Option C: OpenEnv CLI

pip install openenv-core
openenv validate    # Checks compliance
openenv push        # Deploys to HF Spaces

---

4. CLI Agent (Autonomous Play)

Run the sample Claude-based agent for a full autonomous 6-quarter episode:

# Install agent dependencies
pip install -e ".[agent]"

# Set your API key
export ANTHROPIC_API_KEY=sk-ant-...

# Run against local server
python -m agent.run --url ws://localhost:8000 --model claude-sonnet-4-6 --seed 42

# Or run against the live HF Space
python -m agent.run --url wss://paretooptimal-hcm21.hf.space --model claude-sonnet-4-6 --seed 42

The agent will:

1. Scan all departments and calculate baseline metrics
2. Formulate a multi-quarter strategy
3. Execute hiring, training, compensation, and retention decisions
4. Adapt to stochastic events (market crashes, competitor poaching)
5. Summarize each quarter to manage context window limits
6. Print step-by-step actions, rewards, and the final episode score

Example output:

Q1 Scanning | query_department Engineering | headcount: 62, avg_perf: 3.4, flight_risk: 0.18
Q1 Planning | set_hiring_target Engineering count=5
Q1 Producing | execute_hiring Engineering count=5
Q1 Controlling | advance_quarter | Reward: 0.2374 | Event: competitor_poaching (Sales)
Q2 Scanning | query_department Sales | flight_risk: 0.31 (↑)
...
Final Score: 0.6847 (strong strategic agent range)

---

5. Training Notebook (Google Colab)

Train a small LLM to play as CHCO using reinforcement learning.

Setup

1. Open notebooks/hcm21_trl_training.ipynb in Google Colab
2. Select Runtime → Change runtime type → T4 GPU (free tier works)
3. Run all cells

What It Does

Component	Detail
Model	Qwen3-0.6B via Unsloth (4-bit QLoRA)
Algorithm	GRPO (Group Relative Policy Optimization) from HuggingFace TRL
Environment	Connects to the live HF Space via WebSocket
Dataset	20 episode prompts with different seeds
Training	batch_size=2, gradient_accumulation=4, ~30 min on T4

Key Cells

1. Install Dependencies — pip install unsloth trl openenv-core
2. Load Model — Qwen3-0.6B with LoRA (r=16) on all attention + MLP layers
3. Environment Wrapper — HCM21Env class wraps GenericEnvClient (WebSocket) for stateful sessions
4. Training Dataset — 20 prompts describing the CHCO role with different episode seeds
5. GRPO Training — TRL's GRPOTrainer with environment_factory=HCM21Env
6. Evaluation — Runs a full 6-quarter episode with the trained model

Output

The trained LoRA adapter is saved to ./hcm21-agent-lora/ and can be pushed to HuggingFace Hub.

---

6. API Reference (Custom Integrations)

REST Endpoints (Stateless)

Important: REST endpoints are stateless — each /step call creates a fresh environment. Use WebSocket for multi-step episodes.

# Health check
curl https://paretooptimal-hcm21.hf.space/health

# Get action/observation schemas
curl https://paretooptimal-hcm21.hf.space/schema

# Reset (stateless — returns initial observation)
curl -X POST https://paretooptimal-hcm21.hf.space/reset \
  -H "Content-Type: application/json" \
  -d '{"seed": 42}'

# Step (stateless — wraps action in {"action": {...}})
curl -X POST https://paretooptimal-hcm21.hf.space/step \
  -H "Content-Type: application/json" \
  -d '{"action": {"action_type": "query_department", "department": "Engineering"}}'

WebSocket (Stateful Sessions)

For multi-step episodes, use the WebSocket endpoint which maintains state across calls:

from openenv.core import GenericEnvClient, GenericAction

client = GenericEnvClient(base_url="https://paretooptimal-hcm21.hf.space")
client.connect()

# Reset
result = client.reset(seed=42)
print(result.observation)

# Take actions in sequence (state persists)
action = GenericAction(action_type="query_department", department="Engineering")
result = client.step(action)
print(result.observation)

# Continue stepping through phases and quarters...
client.disconnect()

Python Direct (No Server)

For local testing without a server:

from hr_env.server.environment import HRProductivityEnvironment
from hr_env.models import HRAction

env = HRProductivityEnvironment()
obs = env.reset(seed=42)
print(obs.message)

# Take actions directly
obs = env.step(HRAction(action_type="query_department", department="Engineering"))
print(obs.data)

---

7. Scenario Variants

Four pre-built scenarios test different strategic challenges:

Scenario	Seed	Challenge	Target Score
High Turnover	42	Engineering hemorrhaging talent	0.65
Budget Cuts	123	HR budget slashed 50%	0.60
Rapid Growth	456	Scale workforce while maintaining quality	0.65
Balanced Optimization	789	Below-average metrics across the board	0.60

Use the seed when resetting to reproduce a specific scenario:

obs = env.reset(seed=42)   # High turnover scenario
obs = env.reset(seed=123)  # Budget cuts scenario

---

Quick Reference

What You Want	How To Do It
Try it now (no setup)	Open /web in your browser
Run an AI agent	python -m agent.run --url ws://localhost:8000
Train a model	Open notebooks/hcm21_trl_training.ipynb in Colab
Build a custom agent	Use GenericEnvClient (WebSocket) — see Section 6
Run locally	pip install -e ".[server]" && uvicorn hr_env.server.app:app
View API docs	Open /docs