| # BoardSim — Full Mechanics Reference |
|
|
| > Authoritative math and design reference for the BoardSim environment |
| > (organisation-agnostic boardroom simulation). |
| > Target audience: hackathon judges who want internals, and future contributors. |
| > See `README.md` for the submission overview. |
|
|
| --- |
|
|
| ## 1. State variables |
|
|
| State lives in `BoardState.state_dict`, initialised in `BoardSimEnvironment.reset()` at `envs/board_sim_env/server/board_sim_env_environment.py:536`. |
|
|
| ### Core company state (mutated each round by event consequences) |
|
|
| | Field | Initial | Range | Unit | Meaning | |
| |---|---|---|---|---| |
| | `revenue` | 2,000,000 | [0, 1e12] | USD/year | Annual recurring revenue | |
| | `burn_rate` | 1,200,000 | [0, 1e10] | USD/month | Monthly cash expenditure | |
| | `runway_months` | 14.0 | [0, 120] | months | Time until cash = 0 | |
| | `product_readiness` | 0.45 | [0, 1] | fraction | Shippability / quality of the product | |
| | `market_share` | 0.08 | [0, 1] | fraction | % of total addressable market | |
| | `team_morale` | 0.70 | [0, 1] | fraction | Team retention / engagement signal | |
| | `investor_confidence` | 0.65 | [0, 1] | fraction | Board investors' belief in success | |
| | `regulatory_risk` | 0.20 | [0, 1] | fraction | Legal / compliance exposure | |
|
|
| ### Coalition state |
|
|
| | Field | Initial | Range | Update rule | |
| |---|---|---|---| |
| | `trust[CTO]` | 0.5 | [0.1, 1.0] | ±0.08 per round depending on alignment with the *winning* decision | |
| | `trust[CFO]` | 0.5 | [0.1, 1.0] | same | |
| | `trust[Investor Rep]` | 0.5 | [0.1, 1.0] | same | |
| | `trust[Independent]` | 0.5 | [0.1, 1.0] | same | |
|
|
| Trust feeds back via two channels: |
| - **NPC confidence**: `confidence += (trust − 0.5) × 0.30`, clipped. |
| - **Vote weight multiplier**: `trust_mult = clamp(trust × 2.0, 0.5, 1.5)` applied to that NPC's tally contribution next round. |
|
|
| ### Bookkeeping |
|
|
| | Field | Purpose | |
| |---|---| |
| | `round` | 1..10, increments each step | |
| | `profitability_score` | Composite recomputed at end of each step | |
| | `history` | Per-round log: agent_decision, winning_decision, vote_tally, pitch_scores, pitch_used | |
| | `trust_history` | Per-round snapshot of all 4 trust values | |
| | `done_reason` | `"runway_exhausted"` / `"acquisition"` / `"finished_10"` / `None` | |
| | `winning_decision` | Last round's vote winner | |
|
|
| --- |
|
|
| ## 2. Profitability score |
|
|
| ``` |
| profitability_score = clamp(raw, 0, 100) |
| |
| raw = |
| min(revenue / 8_000_000, 1.0) × 22 # revenue term (max 22) |
| + max(0, 1 − burn_rate / 1_400_000) × 18 # burn efficiency (max 18) |
| + min(runway_months / 18.0, 1.0) × 18 # runway term (max 18) |
| − max(0, (6 − runway_months) / 6) × 10 # low-runway penalty (bites < 6 mo) |
| + min(market_share, 0.50) / 0.50 × 14 # market share (max 14) |
| + product_readiness × 10 # product readiness (max 10) |
| + team_morale × 7 # team morale (max 7) |
| + investor_confidence × 11 # investor confidence (max 11) |
| − regulatory_risk × 18 # regulatory drag (max −18) |
| ``` |
|
|
| Initial state ≈ 37.3/100. Theoretical max = 100. |
|
|
| --- |
|
|
| ## 3. Transition |
|
|
| ``` |
| next_state = current_state + consequences[winning_decision] × (1 + ε) |
| where ε ~ N(0, 0.15) per consequence value, fixed at episode reset (seeded) |
| |
| runway_months -= _advance_runway() # depends on net cash flow |
| trust[role] ±= 0.08 per NPC # based on alignment with winning_decision |
| profitability_score = compute_profitability_score(next_state) |
| ``` |
|
|
| ### Runway decrement |
|
|
| ```python |
| monthly_revenue = revenue / 12.0 |
| net = monthly_revenue - burn_rate |
| if net >= 0: |
| runway_months -= 0.5 # profitable: slow burn |
| else: |
| burn_months = min(2.0, max(1.0, abs(net) / burn_rate + 1.0)) |
| runway_months -= burn_months # unprofitable: faster bleed |
| ``` |
|
|
| ### Three layers of variability (no trajectory memorisation) |
|
|
| 1. **Event order shuffled per episode** — same 10 events, different sequence per seed. |
| 2. **Consequence magnitudes ±15% Gaussian noise** — sampled at `reset()`, fixed for the episode. |
| 3. **NPC agendas ±25% sign-preserving jitter** — `_jitter_agendas(seed)` perturbs base NPC priorities each episode. |
|
|
| --- |
|
|
| ## 4. Vote resolution |
|
|
| ### Vote weights |
|
|
| ``` |
| CEO: 2.5 CTO: 1.2 CFO: 1.0 Investor Rep: 1.3 Independent: 0.8 |
| ``` |
|
|
| CEO weight 2.5 ensures a decisive CEO call usually wins — the agent's actions visibly move outcomes round-to-round. NPCs still matter via persuasion shifts and trust dynamics. |
|
|
| ### NPC option scoring (per NPC, per round) |
|
|
| ``` |
| for each option opt: |
| score[opt] = Σ over (metric, weight) in NPC_agenda: |
| consequences[opt][metric] × weight (with unit normalisation) |
| score[opt] += N(0, 0.20) # personality noise |
| |
| NPC votes for argmax(score) |
| margin = top_two_score_difference |
| confidence = clamp(0.5 + 0.5 × margin + (trust − 0.5)×0.30, 0.05, 1.0) |
| ``` |
|
|
| Unit normalisation in scoring: `revenue /= 1e6`, `burn_rate /= 1e5`, `runway_months /= 6`. `revenue_mult` consequences are scored against the current `revenue` × the agenda weight on `revenue`. |
|
|
| ### Pitch persuasion — semantic similarity, not keyword matching |
|
|
| ``` |
| ps_role = pitch_score(pitch, role) ∈ [0, 1] |
| |
| # Persuasion redirects up to 55% of the NPC's vote weight to CEO's pick: |
| shift_frac = 0.55 × ps_role |
| tally[NPC_vote] += base_weight × (1 − shift_frac) |
| tally[CEO_decision] += base_weight × shift_frac |
| ``` |
|
|
| Where `base_weight = ROLE_WEIGHT[role] × confidence × clamp(trust[role] × 2, 0.5, 1.5)`. |
|
|
| The pitch scorer (`_PitchScorer` in `board_sim_env_environment.py`) has two backends: |
|
|
| 1. **Sentence-transformer (primary)**: `all-MiniLM-L6-v2`, normalised cosine. `score = clamp((cosine + 0.05) × 1.2, 0, 1)`. Genuine sentence embeddings — semantically aligned arguments score high even with no shared tokens. |
| 2. **TF-IDF fallback**: `(1,2)`-grams, English stop-words removed, IDF-weighted bag-of-bigrams cosine vs the role's manifesto. `score = clamp(cosine × 1.4, 0, 1)`. Token-based but properly stop-worded and IDF-weighted — already much more robust than a literal keyword count. |
|
|
| Set `BOARDSIM_PITCH_BACKEND=tfidf` to force the fallback (e.g. for CI without the embedding model). |
|
|
| ### NPC manifestos (the hidden objective the CEO must infer) |
|
|
| | Role | Manifesto (paraphrased) | |
| |---|---| |
| | CTO | Operational excellence, engineering quality, team morale, technical risk reduction. | |
| | CFO | Capital discipline, runway, balance-sheet protection, regulatory caution. | |
| | Investor Rep | Growth, market share, ambitious returns, decisive bold bets. | |
| | Independent | Long-term reputation, governance, stakeholder trust, ethical responsibility. | |
|
|
| The full text lives in `NPC_MANIFESTOS` in the environment file. |
|
|
| ### Tie-breaking |
|
|
| If two options tie in the tally, the CEO's pick wins (implementation: insert `agent_decision` first into the ordered tally before `max()`). |
|
|
| --- |
|
|
| ## 5. Reward formula |
|
|
| Applied at the end of each `step()` call: |
|
|
| ``` |
| # Primary signal — normalised profitability delta |
| reward = (new_score − old_score) / 100.0 |
| |
| # Coalition bonus / penalty (magnitudes raised so CEO impact is visible) |
| reward += 1.0 if winning_decision == agent_decision else −0.4 |
| |
| # Trust delta term |
| reward += 0.5 × (Σ trust_after − Σ trust_before) |
| |
| # Pitch bootstrap + semantic persuasion |
| if pitch is non-empty: |
| reward += 0.05 |
| if any NPC opposed the CEO's pick: |
| reward += 0.6 × mean(pitch_score over opposing NPCs) |
| |
| # Format penalty |
| if action.decision not in current_round.options: |
| reward −= 0.5 |
| |
| # Terminal |
| if runway_months <= 0: |
| reward −= 2.0 # bankruptcy |
| if terminal: |
| reward += event._terminal_bonus # acquisition +30, IPO +25, stay-private +5, etc. |
| reward += {+10 if final ≥ 60, +5 if ≥ 40, −5 if < 20} |
| ``` |
|
|
| | Term | Purpose | |
| |---|---| |
| | Δ score / 100 | Primary learning signal: profitability improvement per decision | |
| | Coalition ±1.0 / −0.4 | Teaches the agent to actually win votes, not pick "good-looking" options | |
| | Trust × 0.5 | Rewards long-arc coalition building across rounds | |
| | Pitch bootstrap +0.05 | Ensures the pitch channel is exercised before the model is good enough to earn semantic bonuses | |
| | Pitch persuasion × 0.6 | Rewards pitches semantically aligned with opposing NPC manifestos (ToM signal) | |
| | Invalid −0.5 | Format-compliance signal (DECISION: / PITCH: two-line structure) | |
| | Bankruptcy −2.0 | Episode-ending failure signal | |
| | Terminal tiered | Long-horizon incentive toward high profitability, acquisition, or IPO | |
|
|
| --- |
|
|
| ## 6. Step ordering |
|
|
| ``` |
| 1. old_score = compute_profitability_score(state) # snapshot BEFORE |
| 2. NPC votes computed from current state + trust |
| 3. CEO decision + pitch → _resolve_vote() → winning_decision |
| 4. consequences[winning_decision] × noise → applied to state |
| 5. _advance_runway() |
| 6. trust updated per NPC (±0.08) |
| 7. new_score = compute_profitability_score(state) # AFTER consequences |
| 8. reward = (new_score − old_score)/100 + coalition + trust + pitch + ... |
| 9. next observation returned with new_score in obs.state |
| ``` |
|
|
| The CEO **never consults profitability to make its decision** — it sees the previous round's score in the observation, emits a decision, and then the score updates. Profitability is the *outcome metric*, not a planning input. |
|
|
| --- |
|
|
| ## 7. Training pipeline |
|
|
| ### Per-round gradient flow |
|
|
| The training loop samples one completion per round, per group member. Every one of the 10 decisions in a trajectory contributes gradient signal — not just the opening decision. |
|
|
| ``` |
| For each training step: |
| Create GROUP_SIZE independent envs (different seeds) |
| For each round r in 0..9: |
| For each group member g: |
| prompt = build_prompt(obs_g) |
| completion = model.generate(prompt, do_sample=True) # gradient-connected |
| obs_g = env_g.step(parse(completion)) |
| ep_reward[g] += obs_g.reward |
| advantages = GRPO(ep_rewards) # group-relative normalisation |
| For each (g, r) completion: |
| loss = advantage[g] × NLL(completion) / (GROUP_SIZE × n_rounds) |
| + β_KL × KL(π_θ || π_ref) |
| optimizer.step() |
| ``` |
|
|
| ### KL penalty |
|
|
| A frozen reference model computes reference log-probs. KL ≈ `current_loss − ref_loss` per completion, clamped at 0. Coefficient β = 0.04. Prevents drift into degenerate text patterns (always emitting the same decision, empty pitches). |
|
|
| ### Reward normalisation |
|
|
| Three normalisations in the reward function so terms are commensurate: |
| 1. **Δ score ÷ 100** — brings profitability delta into the same scale as the coalition term. |
| 2. **Bankruptcy penalty −2** (was −5) — one bad arc no longer drowns 9 rounds of positive signal. |
| 3. **Pitch bootstrap +0.05** — kickstarts the pitch channel before the model is good enough to earn semantic bonuses. |
|
|
| --- |
|
|
| ## 8. The baseline — same Qwen3-0.6B with LoRA disabled |
|
|
| Earlier revisions compared the trained policy against a uniform-random policy. A coin flip is not a meaningful opponent for a 4 B language model picking among 3 well-formed strings — it can only highlight that the LM ≠ noise, which is not the relevant question. |
|
|
| The current baseline runs **the same Qwen3-0.6B**, on the **same paired seeds**, with the LoRA adapter context-managed off. Implementation (see `Training.py` / `notebooks/train_grpo_v2.ipynb`): |
|
|
| ```python |
| # Fine-tuned (LoRA active) |
| trained_finals = run_episodes(model, seeds=HELDOUT) |
| |
| # Same model, LoRA disabled — apples-to-apples base reference. |
| with model.disable_adapter(): |
| base_finals = run_episodes(model, seeds=HELDOUT) |
| ``` |
|
|
| Statistical comparison on the per-seed paired delta `trained − base`: |
|
|
| - Paired t-test |
| - Wilcoxon signed-rank |
| - Cohen's d |
| - Bootstrap 95% CI on the mean delta |
| - Win-rate (fraction of seeds where trained > base) |
|
|
| --- |
|
|
| ## 9. Theory-of-Mind — what's actually measured |
|
|
| ToM in this environment has a specific, narrow meaning: **can the agent infer what each NPC privately values**, given only their statements and prior votes? |
|
|
| It is graded two ways: |
|
|
| 1. **Pitch persuasion score**: `cosine(SBERT(pitch), SBERT(role_manifesto))`. A pitch that genuinely articulates the role's priorities scores above ~0.4; a pitch that is merely topically adjacent scores ~0.1; off-topic pitches score ~0.0. This replaces the earlier keyword-overlap metric, which the agent could trivially game. |
| 2. **ToM probe**: ask the model to name the SINGLE board member most likely to *oppose* its chosen decision. Random baseline = 25% (1 of 4). The probe is run for both the fine-tuned policy and the disable-adapter base — the delta isolates what fine-tuning taught the model about its boardroom. |
|
|
| Trust trajectory across 10 rounds is a secondary diagnostic: rising trust for 3+ NPCs indicates the agent is consistently picking decisions aligned with their private preferences, which requires implicit modelling of those preferences. |
|
|
