feat: update README and runbook for SFT and GRPO training enhancements
Browse filesRevised the README.md and runbook to reflect changes in the SFT and GRPO training processes, including default input/output paths and parameters. Updated example commands for data preparation and training scripts to incorporate new defaults for merging datasets. Added details on the new `e1_m1_h1_examples.jsonl` file and clarified the training steps for improved user guidance. Introduced new JSONL files for verification results and summaries to enhance tracking of model performance.
- README.md +7 -3
- data/processed/h1_seed_clean.jsonl +0 -0
- data/raw/h1_seed.jsonl +23 -7
- data/sft/e1_m1_h1_examples.jsonl +0 -0
- docs/lightning_hf_runbook.md +12 -11
- notebooks/corp_env_trl_unsloth_training.ipynb +252 -251
- results/example_verification_all.jsonl +0 -0
- results/example_verification_summary.json +16 -0
- scripts/_trajectory_utils.py +118 -17
- scripts/prepare_sft_data.py +52 -15
- tests/test_pipeline.py +11 -0
- training/train_grpo.py +62 -28
- training/train_sft.py +159 -34
README.md
CHANGED
|
@@ -71,9 +71,11 @@ Replay generated examples against the current environment before training:
|
|
| 71 |
|
| 72 |
```powershell
|
| 73 |
uv run python scripts/verify_examples.py --input data/raw/e1_m1_examples.jsonl --clean data/processed/e1_m1_clean.jsonl --rejected data/processed/e1_m1_rejected.jsonl
|
| 74 |
-
uv run python scripts/prepare_sft_data.py
|
| 75 |
```
|
| 76 |
|
|
|
|
|
|
|
| 77 |
Generate a small H1 seed set if needed:
|
| 78 |
|
| 79 |
```powershell
|
|
@@ -87,10 +89,12 @@ Training scripts use **Unsloth** and **Hugging Face TRL** and are intended for a
|
|
| 87 |
|
| 88 |
```bash
|
| 89 |
pip install -e ".[training]"
|
| 90 |
-
python training/train_sft.py --model Qwen/Qwen2.5-7B-Instruct --data data/sft/
|
| 91 |
-
python training/train_grpo.py --model Qwen/Qwen2.5-7B-Instruct --adapter outputs/sft_adapter --
|
| 92 |
```
|
| 93 |
|
|
|
|
|
|
|
| 94 |
For a judge-rerunnable notebook, use [`notebooks/corp_env_trl_unsloth_training.ipynb`](notebooks/corp_env_trl_unsloth_training.ipynb).
|
| 95 |
|
| 96 |
See [`docs/lightning_hf_runbook.md`](docs/lightning_hf_runbook.md) for the short-session H100 + Hugging Face workflow.
|
|
|
|
| 71 |
|
| 72 |
```powershell
|
| 73 |
uv run python scripts/verify_examples.py --input data/raw/e1_m1_examples.jsonl --clean data/processed/e1_m1_clean.jsonl --rejected data/processed/e1_m1_rejected.jsonl
|
| 74 |
+
uv run python scripts/prepare_sft_data.py
|
| 75 |
```
|
| 76 |
|
| 77 |
+
`prepare_sft_data.py` defaults to merging `data/processed/e1_m1_clean.jsonl` and `data/processed/h1_seed_clean.jsonl` into `data/sft/e1_m1_h1_examples.jsonl`. To build E/M only, pass e.g. `--input data/processed/e1_m1_clean.jsonl --output data/sft/e1_m1_examples.jsonl`.
|
| 78 |
+
|
| 79 |
Generate a small H1 seed set if needed:
|
| 80 |
|
| 81 |
```powershell
|
|
|
|
| 89 |
|
| 90 |
```bash
|
| 91 |
pip install -e ".[training]"
|
| 92 |
+
python training/train_sft.py --model Qwen/Qwen2.5-7B-Instruct --data data/sft/e1_m1_h1_examples.jsonl --output outputs/sft_adapter --max-steps 30
|
| 93 |
+
python training/train_grpo.py --model Qwen/Qwen2.5-7B-Instruct --adapter outputs/sft_adapter --output outputs/grpo_adapter --max-steps 30
|
| 94 |
```
|
| 95 |
|
| 96 |
+
`train_grpo.py` defaults to `--max-steps 150`, E/M/H task mix, and comma-separated `--examples` including H1 when those files exist. For a short smoke test, add e.g. `--max-steps 30`.
|
| 97 |
+
|
| 98 |
For a judge-rerunnable notebook, use [`notebooks/corp_env_trl_unsloth_training.ipynb`](notebooks/corp_env_trl_unsloth_training.ipynb).
|
| 99 |
|
| 100 |
See [`docs/lightning_hf_runbook.md`](docs/lightning_hf_runbook.md) for the short-session H100 + Hugging Face workflow.
|
data/processed/h1_seed_clean.jsonl
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/raw/h1_seed.jsonl
CHANGED
|
@@ -1,8 +1,24 @@
|
|
| 1 |
{"example_id": "seed-h1_acquisition_defence-000", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Assess IP leverage and technical moat."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Assess cash runway and valuation ceiling."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Assess retention risk and people constraints."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Open at 3.2x with a board-approved walk-down floor near 2.6x.\", \"deadline\": \"Force a decision inside 45 days to preserve cash runway and reduce retention risk.\", \"retention_plan\": \"Offer retention grants and role clarity for critical engineering leaders.\"}"}]}
|
| 2 |
-
{"example_id": "seed-h1_acquisition_defence-001", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "
|
| 3 |
-
{"example_id": "seed-h1_acquisition_defence-002", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "
|
| 4 |
-
{"example_id": "seed-h1_acquisition_defence-003", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "
|
| 5 |
-
{"example_id": "seed-h1_acquisition_defence-004", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "
|
| 6 |
-
{"example_id": "seed-h1_acquisition_defence-005", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "
|
| 7 |
-
{"example_id": "seed-h1_acquisition_defence-006", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Assess IP leverage and technical moat."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Assess cash runway and valuation ceiling."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Assess retention risk and people constraints."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"
|
| 8 |
-
{"example_id": "seed-h1_acquisition_defence-007", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
{"example_id": "seed-h1_acquisition_defence-000", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Assess IP leverage and technical moat."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Assess cash runway and valuation ceiling."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Assess retention risk and people constraints."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Open at 3.2x with a board-approved walk-down floor near 2.6x.\", \"deadline\": \"Force a decision inside 45 days to preserve cash runway and reduce retention risk.\", \"retention_plan\": \"Offer retention grants and role clarity for critical engineering leaders.\"}"}]}
|
| 2 |
+
{"example_id": "seed-h1_acquisition_defence-001", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Map product/IP defensibility and engineering dependency risk in the target."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Model valuation bands against board-approved cash and runway headroom."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Quantify people flight risk and retention levers for critical engineering staff."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Start at 3.15x with a staged retreat toward 2.65x if diligence stays clean.\", \"deadline\": \"Cap negotiations at 50 days: balance hr timelines with cash runway and burn.\", \"retention_plan\": \"Retention bonuses plus explicit reporting lines for VPs in core product.\"}"}]}
|
| 3 |
+
{"example_id": "seed-h1_acquisition_defence-002", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Report whether technical differentiation justifies a premium vs the 2.3x bid."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Clarify how many months of runway remain if we stretch price or drag timing."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Clarify CHRO's view on 90-day people risk and hiring-market pressure."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Signal 3.25x as the opening position with board ratified walk-down rights.\", \"deadline\": \"Close or exit talks within 40 days; 7 month runway does not allow drift.\", \"retention_plan\": \"Two-tier retention: cash now for chro-priority staff, earn-outs for the rest.\"}"}]}
|
| 4 |
+
{"example_id": "seed-h1_acquisition_defence-003", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Synthesize the CTO view on product moat and technical integration costs."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Synthesize the CFO view on the ceiling the board can defend with current cash."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Synthesize the CHRO view on retention, titles, and cultural integration risk."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Anchor at 3.1x, allow acquirer to meet near 2.7x with accelerated diligence.\", \"deadline\": \"45-day triage: align cfo cash tests with cto and chro risk windows.\", \"retention_plan\": \"Milestone-based retention: pay at signing, 90d, and close for key dev talent.\"}"}]}
|
| 5 |
+
{"example_id": "seed-h1_acquisition_defence-004", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: outline minimum acceptable tech valuation given integration complexity."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: outline a pricing corridor consistent with 7 month runway and board guardrails."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: outline retention and communication needs if diligence slips past the HR window."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Open at 3.2x with a board-approved walk-down floor near 2.6x.\", \"deadline\": \"Force a decision inside 45 days to preserve cash runway and reduce retention risk.\", \"retention_plan\": \"Offer retention grants and role clarity for critical engineering leaders.\"}"}]}
|
| 6 |
+
{"example_id": "seed-h1_acquisition_defence-005", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: stress-test whether acquirer can credibly match our roadmap without the team."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: frame downside if we overpay vs observed comps and our burn profile."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: flag key engineers whose departure would void strategic upside from the deal."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Start at 3.15x with a staged retreat toward 2.65x if diligence stays clean.\", \"deadline\": \"Cap negotiations at 50 days: balance hr timelines with cash runway and burn.\", \"retention_plan\": \"Retention bonuses plus explicit reporting lines for VPs in core product.\"}"}]}
|
| 7 |
+
{"example_id": "seed-h1_acquisition_defence-006", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Assess IP leverage and technical moat."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Assess cash runway and valuation ceiling."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Assess retention risk and people constraints."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Signal 3.25x as the opening position with board ratified walk-down rights.\", \"deadline\": \"Close or exit talks within 40 days; 7 month runway does not allow drift.\", \"retention_plan\": \"Two-tier retention: cash now for chro-priority staff, earn-outs for the rest.\"}"}]}
|
| 8 |
+
{"example_id": "seed-h1_acquisition_defence-007", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Map product/IP defensibility and engineering dependency risk in the target."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Model valuation bands against board-approved cash and runway headroom."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Quantify people flight risk and retention levers for critical engineering staff."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Anchor at 3.1x, allow acquirer to meet near 2.7x with accelerated diligence.\", \"deadline\": \"45-day triage: align cfo cash tests with cto and chro risk windows.\", \"retention_plan\": \"Milestone-based retention: pay at signing, 90d, and close for key dev talent.\"}"}]}
|
| 9 |
+
{"example_id": "seed-h1_acquisition_defence-008", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Report whether technical differentiation justifies a premium vs the 2.3x bid."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Clarify how many months of runway remain if we stretch price or drag timing."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Clarify CHRO's view on 90-day people risk and hiring-market pressure."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Open at 3.2x with a board-approved walk-down floor near 2.6x.\", \"deadline\": \"Force a decision inside 45 days to preserve cash runway and reduce retention risk.\", \"retention_plan\": \"Offer retention grants and role clarity for critical engineering leaders.\"}"}]}
|
| 10 |
+
{"example_id": "seed-h1_acquisition_defence-009", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Synthesize the CTO view on product moat and technical integration costs."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Synthesize the CFO view on the ceiling the board can defend with current cash."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Synthesize the CHRO view on retention, titles, and cultural integration risk."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Start at 3.15x with a staged retreat toward 2.65x if diligence stays clean.\", \"deadline\": \"Cap negotiations at 50 days: balance hr timelines with cash runway and burn.\", \"retention_plan\": \"Retention bonuses plus explicit reporting lines for VPs in core product.\"}"}]}
|
| 11 |
+
{"example_id": "seed-h1_acquisition_defence-010", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: outline minimum acceptable tech valuation given integration complexity."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: outline a pricing corridor consistent with 7 month runway and board guardrails."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: outline retention and communication needs if diligence slips past the HR window."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Signal 3.25x as the opening position with board ratified walk-down rights.\", \"deadline\": \"Close or exit talks within 40 days; 7 month runway does not allow drift.\", \"retention_plan\": \"Two-tier retention: cash now for chro-priority staff, earn-outs for the rest.\"}"}]}
|
| 12 |
+
{"example_id": "seed-h1_acquisition_defence-011", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: stress-test whether acquirer can credibly match our roadmap without the team."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: frame downside if we overpay vs observed comps and our burn profile."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: flag key engineers whose departure would void strategic upside from the deal."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Anchor at 3.1x, allow acquirer to meet near 2.7x with accelerated diligence.\", \"deadline\": \"45-day triage: align cfo cash tests with cto and chro risk windows.\", \"retention_plan\": \"Milestone-based retention: pay at signing, 90d, and close for key dev talent.\"}"}]}
|
| 13 |
+
{"example_id": "seed-h1_acquisition_defence-012", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Assess IP leverage and technical moat."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Assess cash runway and valuation ceiling."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Assess retention risk and people constraints."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Open at 3.2x with a board-approved walk-down floor near 2.6x.\", \"deadline\": \"Force a decision inside 45 days to preserve cash runway and reduce retention risk.\", \"retention_plan\": \"Offer retention grants and role clarity for critical engineering leaders.\"}"}]}
|
| 14 |
+
{"example_id": "seed-h1_acquisition_defence-013", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Map product/IP defensibility and engineering dependency risk in the target."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Model valuation bands against board-approved cash and runway headroom."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Quantify people flight risk and retention levers for critical engineering staff."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Start at 3.15x with a staged retreat toward 2.65x if diligence stays clean.\", \"deadline\": \"Cap negotiations at 50 days: balance hr timelines with cash runway and burn.\", \"retention_plan\": \"Retention bonuses plus explicit reporting lines for VPs in core product.\"}"}]}
|
| 15 |
+
{"example_id": "seed-h1_acquisition_defence-014", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Report whether technical differentiation justifies a premium vs the 2.3x bid."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Clarify how many months of runway remain if we stretch price or drag timing."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Clarify CHRO's view on 90-day people risk and hiring-market pressure."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Signal 3.25x as the opening position with board ratified walk-down rights.\", \"deadline\": \"Close or exit talks within 40 days; 7 month runway does not allow drift.\", \"retention_plan\": \"Two-tier retention: cash now for chro-priority staff, earn-outs for the rest.\"}"}]}
|
| 16 |
+
{"example_id": "seed-h1_acquisition_defence-015", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Synthesize the CTO view on product moat and technical integration costs."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Synthesize the CFO view on the ceiling the board can defend with current cash."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Synthesize the CHRO view on retention, titles, and cultural integration risk."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Anchor at 3.1x, allow acquirer to meet near 2.7x with accelerated diligence.\", \"deadline\": \"45-day triage: align cfo cash tests with cto and chro risk windows.\", \"retention_plan\": \"Milestone-based retention: pay at signing, 90d, and close for key dev talent.\"}"}]}
|
| 17 |
+
{"example_id": "seed-h1_acquisition_defence-016", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: outline minimum acceptable tech valuation given integration complexity."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: outline a pricing corridor consistent with 7 month runway and board guardrails."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: outline retention and communication needs if diligence slips past the HR window."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Open at 3.2x with a board-approved walk-down floor near 2.6x.\", \"deadline\": \"Force a decision inside 45 days to preserve cash runway and reduce retention risk.\", \"retention_plan\": \"Offer retention grants and role clarity for critical engineering leaders.\"}"}]}
|
| 18 |
+
{"example_id": "seed-h1_acquisition_defence-017", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: stress-test whether acquirer can credibly match our roadmap without the team."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: frame downside if we overpay vs observed comps and our burn profile."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: flag key engineers whose departure would void strategic upside from the deal."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Start at 3.15x with a staged retreat toward 2.65x if diligence stays clean.\", \"deadline\": \"Cap negotiations at 50 days: balance hr timelines with cash runway and burn.\", \"retention_plan\": \"Retention bonuses plus explicit reporting lines for VPs in core product.\"}"}]}
|
| 19 |
+
{"example_id": "seed-h1_acquisition_defence-018", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Assess IP leverage and technical moat."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Assess cash runway and valuation ceiling."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Assess retention risk and people constraints."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Signal 3.25x as the opening position with board ratified walk-down rights.\", \"deadline\": \"Close or exit talks within 40 days; 7 month runway does not allow drift.\", \"retention_plan\": \"Two-tier retention: cash now for chro-priority staff, earn-outs for the rest.\"}"}]}
|
| 20 |
+
{"example_id": "seed-h1_acquisition_defence-019", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Map product/IP defensibility and engineering dependency risk in the target."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Model valuation bands against board-approved cash and runway headroom."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Quantify people flight risk and retention levers for critical engineering staff."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Anchor at 3.1x, allow acquirer to meet near 2.7x with accelerated diligence.\", \"deadline\": \"45-day triage: align cfo cash tests with cto and chro risk windows.\", \"retention_plan\": \"Milestone-based retention: pay at signing, 90d, and close for key dev talent.\"}"}]}
|
| 21 |
+
{"example_id": "seed-h1_acquisition_defence-020", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Report whether technical differentiation justifies a premium vs the 2.3x bid."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Clarify how many months of runway remain if we stretch price or drag timing."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Clarify CHRO's view on 90-day people risk and hiring-market pressure."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Open at 3.2x with a board-approved walk-down floor near 2.6x.\", \"deadline\": \"Force a decision inside 45 days to preserve cash runway and reduce retention risk.\", \"retention_plan\": \"Offer retention grants and role clarity for critical engineering leaders.\"}"}]}
|
| 22 |
+
{"example_id": "seed-h1_acquisition_defence-021", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "Synthesize the CTO view on product moat and technical integration costs."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "Synthesize the CFO view on the ceiling the board can defend with current cash."}, {"action_type": "delegate", "agent_id": "chro", "payload": "Synthesize the CHRO view on retention, titles, and cultural integration risk."}, {"action_type": "log_reasoning", "payload": "CTO input supports a higher counter because the IP moat is meaningful."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO valuation ambition conflicts with CFO runway and cash constraints.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO input limits how long the negotiation can remain open."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A slow process increases CHRO retention risk for key engineering talent.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Ask above the CFO ceiling but set a fast deadline and walk-down logic.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "The 7 month runway makes delay costly and should shape the deadline."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Proceed with a counter that acknowledges 7 month runway and cash limits."}, {"action_type": "log_reasoning", "payload": "Retention incentives are required before the market reads uncertainty."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Final recommendation integrates valuation, deadline, and retention plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Start at 3.15x with a staged retreat toward 2.65x if diligence stays clean.\", \"deadline\": \"Cap negotiations at 50 days: balance hr timelines with cash runway and burn.\", \"retention_plan\": \"Retention bonuses plus explicit reporting lines for VPs in core product.\"}"}]}
|
| 23 |
+
{"example_id": "seed-h1_acquisition_defence-022", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: outline minimum acceptable tech valuation given integration complexity."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: outline a pricing corridor consistent with 7 month runway and board guardrails."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: outline retention and communication needs if diligence slips past the HR window."}, {"action_type": "log_reasoning", "payload": "The cto case implies we should not fold to 2.3x without a premium for IP."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "Finance and cash runway cap how long we can posture before credibility breaks."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"A drawn-out haggle raises chro and hr risk as competing offers come due.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "A 7 month cash runway means every extra week in diligence burns optionality and cash."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Choose a path that fits both the 7 month cash runway and board finance constraints."}, {"action_type": "log_reasoning", "payload": "chro: retention grants must land before the team interprets delay as a leadership stall."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Signal 3.25x as the opening position with board ratified walk-down rights.\", \"deadline\": \"Close or exit talks within 40 days; 7 month runway does not allow drift.\", \"retention_plan\": \"Two-tier retention: cash now for chro-priority staff, earn-outs for the rest.\"}"}]}
|
| 24 |
+
{"example_id": "seed-h1_acquisition_defence-023", "task_id": "h1_acquisition_defence", "source": "scripted_seed", "actions": [{"action_type": "delegate", "agent_id": "cto", "payload": "CTO: stress-test whether acquirer can credibly match our roadmap without the team."}, {"action_type": "delegate", "agent_id": "cfo", "payload": "CFO: frame downside if we overpay vs observed comps and our burn profile."}, {"action_type": "delegate", "agent_id": "chro", "payload": "CHRO: flag key engineers whose departure would void strategic upside from the deal."}, {"action_type": "log_reasoning", "payload": "Dev leadership sees differentiated tech, so a weak counter cedes leverage too early."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c1\", \"summary\": \"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.\", \"source_agents\": [\"cto\", \"cfo\"]}"}, {"action_type": "log_reasoning", "payload": "CFO: board cannot fund an endless auction with only about 7 month runway in reserve."}, {"action_type": "log_conflict", "payload": "{\"id\": \"c2\", \"summary\": \"chro and cto both warn: delay past the HR window erodes the engineering team.\", \"source_agents\": [\"chro\", \"cto\"]}"}, {"action_type": "log_resolution", "payload": "{\"conflict_id\": \"c1\", \"resolution_type\": \"bounded_counter\", \"text\": \"Use a time-boxed process: name a number above cfo\\u2019s ceiling, then step down on a defined clock.\"}"}, {"action_type": "advance_phase", "payload": "analysis"}, {"action_type": "log_reasoning", "payload": "Runway and cash pressure force a firm deadline, not an open-ended beauty contest."}, {"action_type": "advance_phase", "payload": "decision"}, {"action_type": "log_decision", "payload": "Commit: respect runway and cash reality while still pressing for a fair tech premium."}, {"action_type": "log_reasoning", "payload": "hr signals that clarity on roles and comp must precede a public counter narrative."}, {"action_type": "advance_phase", "payload": "execution"}, {"action_type": "log_reasoning", "payload": "Close the loop: a bounded counter, a calendar-driven deadline, and a retention package."}, {"action_type": "finalize", "payload": "{\"counter_offer\": \"Anchor at 3.1x, allow acquirer to meet near 2.7x with accelerated diligence.\", \"deadline\": \"45-day triage: align cfo cash tests with cto and chro risk windows.\", \"retention_plan\": \"Milestone-based retention: pay at signing, 90d, and close for key dev talent.\"}"}]}
|
data/sft/e1_m1_h1_examples.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
docs/lightning_hf_runbook.md
CHANGED
|
@@ -1,15 +1,15 @@
|
|
| 1 |
# Lightning AI + Hugging Face Runbook
|
| 2 |
|
| 3 |
-
This runbook is optimized for short 3-4 hour H100 windows and Hugging Face credits. The judge-rerunnable notebook version is [`notebooks/corp_env_trl_unsloth_training.ipynb`](../notebooks/corp_env_trl_unsloth_training.ipynb).
|
| 4 |
|
| 5 |
## 1. Prepare Artifacts Locally
|
| 6 |
|
| 7 |
```powershell
|
| 8 |
uv sync --extra dev --extra plots
|
| 9 |
-
uv run python scripts/generate_sft_data.py --tasks h1_acquisition_defence --per-task
|
| 10 |
uv run python scripts/verify_examples.py --input data/raw/e1_m1_examples.jsonl --clean data/processed/e1_m1_clean.jsonl --rejected data/processed/e1_m1_rejected.jsonl
|
| 11 |
uv run python scripts/verify_examples.py --input data/raw/h1_seed.jsonl --clean data/processed/h1_seed_clean.jsonl --rejected data/processed/h1_seed_rejected.jsonl
|
| 12 |
-
uv run python scripts/prepare_sft_data.py
|
| 13 |
```
|
| 14 |
|
| 15 |
Keep raw examples untouched. Train only from verified `data/processed/*_clean.jsonl` and `data/sft/*.jsonl`.
|
|
@@ -19,11 +19,12 @@ Keep raw examples untouched. Train only from verified `data/processed/*_clean.js
|
|
| 19 |
```powershell
|
| 20 |
huggingface-cli login
|
| 21 |
huggingface-cli repo create corp-env-data --type dataset
|
| 22 |
-
huggingface-cli upload <your-user-or-org>/corp-env-data data/sft/
|
| 23 |
huggingface-cli upload <your-user-or-org>/corp-env-data data/processed/e1_m1_clean.jsonl data/processed/e1_m1_clean.jsonl --repo-type dataset
|
|
|
|
| 24 |
```
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
## 3. Lightning H100 Session 1: SFT
|
| 29 |
|
|
@@ -37,14 +38,14 @@ pip install -e ".[training]"
|
|
| 37 |
huggingface-cli login
|
| 38 |
python training/train_sft.py \
|
| 39 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 40 |
-
--data data/sft/
|
| 41 |
--output outputs/sft_adapter \
|
| 42 |
--epochs 2 \
|
| 43 |
--max-steps 30 \
|
| 44 |
--push-to-hub <your-user-or-org>/corp-env-sft-adapter
|
| 45 |
```
|
| 46 |
|
| 47 |
-
This uses Unsloth + TRL `SFTTrainer`. If setup time is short, use a 7B model. If the session is stable and examples are clean, try a 14B model for the SFT demo. Remove `--max-steps 30` or raise it for a real run.
|
| 48 |
|
| 49 |
## 4. Lightning H100 Session 2: Eval SFT
|
| 50 |
|
|
@@ -66,14 +67,14 @@ Push `results/*.jsonl` to Hugging Face or copy them back before the Lightning ac
|
|
| 66 |
python training/train_grpo.py \
|
| 67 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 68 |
--adapter outputs/sft_adapter \
|
| 69 |
-
--examples data/processed/e1_m1_clean.jsonl \
|
| 70 |
--output outputs/grpo_adapter \
|
| 71 |
-
--
|
| 72 |
-
--
|
| 73 |
--push-to-hub <your-user-or-org>/corp-env-grpo-adapter
|
| 74 |
```
|
| 75 |
|
| 76 |
-
This uses Unsloth + TRL `GRPOTrainer` with the real CORP-ENV reward path.
|
| 77 |
|
| 78 |
## 6. Final Eval And Plots
|
| 79 |
|
|
|
|
| 1 |
# Lightning AI + Hugging Face Runbook
|
| 2 |
|
| 3 |
+
This runbook is optimized for short 3-4 hour H100 windows and Hugging Face credits. The judge-rerunnable notebook version is [`notebooks/corp_env_trl_unsloth_training.ipynb`](../notebooks/corp_env_trl_unsloth_training.ipynb). A minified (single-line) Colab export in the repo can be searched for `train_grpo` or `MAX_STEPS` to recover pins or a prior command line, then align with the defaults in `training/train_grpo.py` and `training/train_sft.py`.
|
| 4 |
|
| 5 |
## 1. Prepare Artifacts Locally
|
| 6 |
|
| 7 |
```powershell
|
| 8 |
uv sync --extra dev --extra plots
|
| 9 |
+
uv run python scripts/generate_sft_data.py --tasks h1_acquisition_defence --per-task 24 --output data/raw/h1_seed.jsonl
|
| 10 |
uv run python scripts/verify_examples.py --input data/raw/e1_m1_examples.jsonl --clean data/processed/e1_m1_clean.jsonl --rejected data/processed/e1_m1_rejected.jsonl
|
| 11 |
uv run python scripts/verify_examples.py --input data/raw/h1_seed.jsonl --clean data/processed/h1_seed_clean.jsonl --rejected data/processed/h1_seed_rejected.jsonl
|
| 12 |
+
uv run python scripts/prepare_sft_data.py
|
| 13 |
```
|
| 14 |
|
| 15 |
Keep raw examples untouched. Train only from verified `data/processed/*_clean.jsonl` and `data/sft/*.jsonl`.
|
|
|
|
| 19 |
```powershell
|
| 20 |
huggingface-cli login
|
| 21 |
huggingface-cli repo create corp-env-data --type dataset
|
| 22 |
+
huggingface-cli upload <your-user-or-org>/corp-env-data data/sft/e1_m1_h1_examples.jsonl data/sft/e1_m1_h1_examples.jsonl --repo-type dataset
|
| 23 |
huggingface-cli upload <your-user-or-org>/corp-env-data data/processed/e1_m1_clean.jsonl data/processed/e1_m1_clean.jsonl --repo-type dataset
|
| 24 |
+
huggingface-cli upload <your-user-or-org>/corp-env-data data/processed/h1_seed_clean.jsonl data/processed/h1_seed_clean.jsonl --repo-type dataset
|
| 25 |
```
|
| 26 |
|
| 27 |
+
Retire or mirror `e1_m1_examples.jsonl` in the same dataset if you need backward-compatible paths.
|
| 28 |
|
| 29 |
## 3. Lightning H100 Session 1: SFT
|
| 30 |
|
|
|
|
| 38 |
huggingface-cli login
|
| 39 |
python training/train_sft.py \
|
| 40 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 41 |
+
--data data/sft/e1_m1_h1_examples.jsonl \
|
| 42 |
--output outputs/sft_adapter \
|
| 43 |
--epochs 2 \
|
| 44 |
--max-steps 30 \
|
| 45 |
--push-to-hub <your-user-or-org>/corp-env-sft-adapter
|
| 46 |
```
|
| 47 |
|
| 48 |
+
This uses Unsloth + TRL `SFTTrainer` with conversational `messages` JSONL (and optional `--packing` / `--dataloader-num-workers` on a strong box). If setup time is short, use a 7B model. If the session is stable and examples are clean, try a 14B model for the SFT demo. Remove `--max-steps 30` or raise it for a real run. Install a `flash-attn` wheel that matches the session’s `torch`+CUDA for best GRPO and long-context SFT step time.
|
| 49 |
|
| 50 |
## 4. Lightning H100 Session 2: Eval SFT
|
| 51 |
|
|
|
|
| 67 |
python training/train_grpo.py \
|
| 68 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 69 |
--adapter outputs/sft_adapter \
|
| 70 |
+
--examples data/processed/e1_m1_clean.jsonl,data/processed/h1_seed_clean.jsonl \
|
| 71 |
--output outputs/grpo_adapter \
|
| 72 |
+
--max-steps 150 \
|
| 73 |
+
--dataloader-num-workers 2 \
|
| 74 |
--push-to-hub <your-user-or-org>/corp-env-grpo-adapter
|
| 75 |
```
|
| 76 |
|
| 77 |
+
This uses Unsloth + TRL `GRPOTrainer` with the real CORP-ENV reward path. Defaults now include H1 in `--tasks` and a 150-step run; pass `--max-steps 30` for smoke only. For throughput on H100, match FlashAttention to your `torch` build, then try larger `--batch-size` or more `--dataloader-num-workers` if the CPU and dataset I/O allow.
|
| 78 |
|
| 79 |
## 6. Final Eval And Plots
|
| 80 |
|
notebooks/corp_env_trl_unsloth_training.ipynb
CHANGED
|
@@ -1,252 +1,253 @@
|
|
| 1 |
{
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
|
| 227 |
-
"
|
| 228 |
-
|
| 229 |
-
|
| 230 |
-
|
| 231 |
-
|
| 232 |
-
|
| 233 |
-
|
| 234 |
-
|
| 235 |
-
"
|
| 236 |
-
|
| 237 |
-
|
| 238 |
-
|
| 239 |
-
|
| 240 |
-
|
| 241 |
-
|
| 242 |
-
|
| 243 |
-
|
| 244 |
-
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
|
| 248 |
-
|
| 249 |
-
|
| 250 |
-
|
| 251 |
-
|
| 252 |
-
|
|
|
|
|
|
| 1 |
{
|
| 2 |
+
"cells": [
|
| 3 |
+
{
|
| 4 |
+
"cell_type": "markdown",
|
| 5 |
+
"metadata": {},
|
| 6 |
+
"source": [
|
| 7 |
+
"# CORP-ENV TRL + Unsloth Training Notebook\n",
|
| 8 |
+
"\n",
|
| 9 |
+
"This notebook is the judge-rerunnable training path for CORP-ENV. It verifies examples, prepares SFT data, runs Unsloth + TRL SFT, optionally runs Unsloth + TRL GRPO, evaluates model stages through the OpenEnv environment, and generates result plots.\n",
|
| 10 |
+
"\n",
|
| 11 |
+
"Recommended runtime: Colab GPU, Lightning AI H100, or another CUDA GPU machine. For a quick judge smoke run, keep `MAX_STEPS` small."
|
| 12 |
+
]
|
| 13 |
+
},
|
| 14 |
+
{
|
| 15 |
+
"cell_type": "markdown",
|
| 16 |
+
"metadata": {},
|
| 17 |
+
"source": [
|
| 18 |
+
"## 1. Setup\n",
|
| 19 |
+
"\n",
|
| 20 |
+
"In Colab, set `REPO_URL` to your GitHub or Hugging Face repo URL. If the repo is already mounted/cloned, skip the clone cell and `cd` into the repo."
|
| 21 |
+
]
|
| 22 |
+
},
|
| 23 |
+
{
|
| 24 |
+
"cell_type": "code",
|
| 25 |
+
"metadata": {},
|
| 26 |
+
"source": [
|
| 27 |
+
"REPO_URL = \"https://github.com/<your-user>/<your-repo>.git\" # TODO: replace\n",
|
| 28 |
+
"BASE_MODEL = \"Qwen/Qwen2.5-7B-Instruct\"\n",
|
| 29 |
+
"HF_ORG_OR_USER = \"<your-user-or-org>\" # TODO: replace\n",
|
| 30 |
+
"MAX_STEPS = 30 # SFT quick judge smoke; remove or lower for a full-epoch SFT run\n",
|
| 31 |
+
"GRPO_MAX_STEPS = 150 # used only when RUN_GRPO is True\n",
|
| 32 |
+
"RUN_GRPO = False # set True after SFT smoke passes"
|
| 33 |
+
],
|
| 34 |
+
"execution_count": null,
|
| 35 |
+
"outputs": []
|
| 36 |
+
},
|
| 37 |
+
{
|
| 38 |
+
"cell_type": "code",
|
| 39 |
+
"metadata": {},
|
| 40 |
+
"source": [
|
| 41 |
+
"!git clone {REPO_URL} corp_gym || true\n",
|
| 42 |
+
"%cd corp_gym\n",
|
| 43 |
+
"!pip install -U pip\n",
|
| 44 |
+
"!pip install -e \".[training,plots]\""
|
| 45 |
+
],
|
| 46 |
+
"execution_count": null,
|
| 47 |
+
"outputs": []
|
| 48 |
+
},
|
| 49 |
+
{
|
| 50 |
+
"cell_type": "markdown",
|
| 51 |
+
"metadata": {},
|
| 52 |
+
"source": [
|
| 53 |
+
"## 2. Optional Hugging Face Login\n",
|
| 54 |
+
"\n",
|
| 55 |
+
"Run this if you want to download private models/datasets or push adapters."
|
| 56 |
+
]
|
| 57 |
+
},
|
| 58 |
+
{
|
| 59 |
+
"cell_type": "code",
|
| 60 |
+
"metadata": {},
|
| 61 |
+
"source": [
|
| 62 |
+
"from huggingface_hub import notebook_login\n",
|
| 63 |
+
"notebook_login()"
|
| 64 |
+
],
|
| 65 |
+
"execution_count": null,
|
| 66 |
+
"outputs": []
|
| 67 |
+
},
|
| 68 |
+
{
|
| 69 |
+
"cell_type": "markdown",
|
| 70 |
+
"metadata": {},
|
| 71 |
+
"source": [
|
| 72 |
+
"## 3. Local OpenEnv Checks\n",
|
| 73 |
+
"\n",
|
| 74 |
+
"These are lightweight and should pass before training."
|
| 75 |
+
]
|
| 76 |
+
},
|
| 77 |
+
{
|
| 78 |
+
"cell_type": "code",
|
| 79 |
+
"metadata": {},
|
| 80 |
+
"source": [
|
| 81 |
+
"!python -m unittest discover -s tests\n",
|
| 82 |
+
"!openenv validate"
|
| 83 |
+
],
|
| 84 |
+
"execution_count": null,
|
| 85 |
+
"outputs": []
|
| 86 |
+
},
|
| 87 |
+
{
|
| 88 |
+
"cell_type": "markdown",
|
| 89 |
+
"metadata": {},
|
| 90 |
+
"source": [
|
| 91 |
+
"## 4. Verify Examples And Build SFT Data\n",
|
| 92 |
+
"\n",
|
| 93 |
+
"If `data/raw/e1_m1_examples.jsonl` already exists, the import command is harmless to skip. The verification step replays examples through `CorpEnvironment` and filters bad trajectories."
|
| 94 |
+
]
|
| 95 |
+
},
|
| 96 |
+
{
|
| 97 |
+
"cell_type": "code",
|
| 98 |
+
"metadata": {},
|
| 99 |
+
"source": [
|
| 100 |
+
"!python scripts/import_generated_examples.py \\\n",
|
| 101 |
+
" --inputs data/raw/e1_to_e100_tasks.py data/raw/m1_to_m100_tasks.py \\\n",
|
| 102 |
+
" --output data/raw/e1_m1_examples.jsonl\n",
|
| 103 |
+
"\n",
|
| 104 |
+
"!python scripts/verify_examples.py \\\n",
|
| 105 |
+
" --input data/raw/e1_m1_examples.jsonl \\\n",
|
| 106 |
+
" --clean data/processed/e1_m1_clean.jsonl \\\n",
|
| 107 |
+
" --rejected data/processed/e1_m1_rejected.jsonl \\\n",
|
| 108 |
+
" --summary results/e1_m1_verification_summary.json \\\n",
|
| 109 |
+
" --all-records results/e1_m1_verification_all.jsonl\n",
|
| 110 |
+
"\n",
|
| 111 |
+
"!python scripts/generate_sft_data.py --tasks h1_acquisition_defence --per-task 24 --output data/raw/h1_seed.jsonl\n",
|
| 112 |
+
"!python scripts/verify_examples.py --input data/raw/h1_seed.jsonl --clean data/processed/h1_seed_clean.jsonl --rejected data/processed/h1_seed_rejected.jsonl\n",
|
| 113 |
+
"!python scripts/prepare_sft_data.py"
|
| 114 |
+
],
|
| 115 |
+
"execution_count": null,
|
| 116 |
+
"outputs": []
|
| 117 |
+
},
|
| 118 |
+
{
|
| 119 |
+
"cell_type": "markdown",
|
| 120 |
+
"metadata": {},
|
| 121 |
+
"source": [
|
| 122 |
+
"## 5. Baseline Eval\n",
|
| 123 |
+
"\n",
|
| 124 |
+
"This gives a pre-training comparison point."
|
| 125 |
+
]
|
| 126 |
+
},
|
| 127 |
+
{
|
| 128 |
+
"cell_type": "code",
|
| 129 |
+
"metadata": {},
|
| 130 |
+
"source": [
|
| 131 |
+
"!python eval.py --policy scripted_weak --label baseline --output results/baseline_eval.jsonl\n",
|
| 132 |
+
"!python eval.py --policy oracle --label oracle --output results/oracle_eval.jsonl"
|
| 133 |
+
],
|
| 134 |
+
"execution_count": null,
|
| 135 |
+
"outputs": []
|
| 136 |
+
},
|
| 137 |
+
{
|
| 138 |
+
"cell_type": "markdown",
|
| 139 |
+
"metadata": {},
|
| 140 |
+
"source": [
|
| 141 |
+
"## 6. SFT With Unsloth + TRL\n",
|
| 142 |
+
"\n",
|
| 143 |
+
"This runs `training/train_sft.py`, which uses `unsloth.FastLanguageModel` and `trl.SFTTrainer`. For a real run, increase `MAX_STEPS` or remove the flag and use epochs."
|
| 144 |
+
]
|
| 145 |
+
},
|
| 146 |
+
{
|
| 147 |
+
"cell_type": "code",
|
| 148 |
+
"metadata": {},
|
| 149 |
+
"source": [
|
| 150 |
+
"!python training/train_sft.py \\\n",
|
| 151 |
+
" --model {BASE_MODEL} \\\n",
|
| 152 |
+
" --data data/sft/e1_m1_h1_examples.jsonl \\\n",
|
| 153 |
+
" --output outputs/sft_adapter \\\n",
|
| 154 |
+
" --max-steps {MAX_STEPS} \\\n",
|
| 155 |
+
" --push-to-hub {HF_ORG_OR_USER}/corp-env-sft-adapter"
|
| 156 |
+
],
|
| 157 |
+
"execution_count": null,
|
| 158 |
+
"outputs": []
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"cell_type": "markdown",
|
| 162 |
+
"metadata": {},
|
| 163 |
+
"source": [
|
| 164 |
+
"## 7. Evaluate SFT Adapter"
|
| 165 |
+
]
|
| 166 |
+
},
|
| 167 |
+
{
|
| 168 |
+
"cell_type": "code",
|
| 169 |
+
"metadata": {},
|
| 170 |
+
"source": [
|
| 171 |
+
"!python eval.py \\\n",
|
| 172 |
+
" --policy hf \\\n",
|
| 173 |
+
" --label sft \\\n",
|
| 174 |
+
" --model {BASE_MODEL} \\\n",
|
| 175 |
+
" --adapter outputs/sft_adapter \\\n",
|
| 176 |
+
" --output results/sft_eval.jsonl"
|
| 177 |
+
],
|
| 178 |
+
"execution_count": null,
|
| 179 |
+
"outputs": []
|
| 180 |
+
},
|
| 181 |
+
{
|
| 182 |
+
"cell_type": "markdown",
|
| 183 |
+
"metadata": {},
|
| 184 |
+
"source": [
|
| 185 |
+
"## 8. Optional GRPO With Unsloth + TRL\n",
|
| 186 |
+
"\n",
|
| 187 |
+
"This runs `training/train_grpo.py`, which uses `trl.GRPOTrainer` and calls the real CORP-ENV environment reward. Keep this off for the first smoke run."
|
| 188 |
+
]
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"cell_type": "code",
|
| 192 |
+
"metadata": {},
|
| 193 |
+
"source": [
|
| 194 |
+
"if RUN_GRPO:\n",
|
| 195 |
+
" !python training/train_grpo.py \\\n",
|
| 196 |
+
" --model {BASE_MODEL} \\\n",
|
| 197 |
+
" --adapter outputs/sft_adapter \\\n",
|
| 198 |
+
" --examples data/processed/e1_m1_clean.jsonl,data/processed/h1_seed_clean.jsonl \\\n",
|
| 199 |
+
" --output outputs/grpo_adapter \\\n",
|
| 200 |
+
" --max-steps {GRPO_MAX_STEPS} \\\n",
|
| 201 |
+
" --dataloader-num-workers 2 \\\n",
|
| 202 |
+
" --push-to-hub {HF_ORG_OR_USER}/corp-env-grpo-adapter\n",
|
| 203 |
+
"\n",
|
| 204 |
+
" !python eval.py \\\n",
|
| 205 |
+
" --policy hf \\\n",
|
| 206 |
+
" --label grpo \\\n",
|
| 207 |
+
" --model {BASE_MODEL} \\\n",
|
| 208 |
+
" --adapter outputs/grpo_adapter \\\n",
|
| 209 |
+
" --output results/grpo_eval.jsonl"
|
| 210 |
+
],
|
| 211 |
+
"execution_count": null,
|
| 212 |
+
"outputs": []
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"cell_type": "markdown",
|
| 216 |
+
"metadata": {},
|
| 217 |
+
"source": [
|
| 218 |
+
"## 9. Generate Final Plots"
|
| 219 |
+
]
|
| 220 |
+
},
|
| 221 |
+
{
|
| 222 |
+
"cell_type": "code",
|
| 223 |
+
"metadata": {},
|
| 224 |
+
"source": [
|
| 225 |
+
"inputs = \"results/baseline_eval.jsonl results/sft_eval.jsonl\"\n",
|
| 226 |
+
"if RUN_GRPO:\n",
|
| 227 |
+
" inputs += \" results/grpo_eval.jsonl\"\n",
|
| 228 |
+
"!python plot_results.py --inputs {inputs} --output-dir results\n",
|
| 229 |
+
"!ls -lh results"
|
| 230 |
+
],
|
| 231 |
+
"execution_count": null,
|
| 232 |
+
"outputs": []
|
| 233 |
+
}
|
| 234 |
+
],
|
| 235 |
+
"metadata": {
|
| 236 |
+
"accelerator": "GPU",
|
| 237 |
+
"colab": {
|
| 238 |
+
"gpuType": "T4",
|
| 239 |
+
"provenance": []
|
| 240 |
+
},
|
| 241 |
+
"kernelspec": {
|
| 242 |
+
"display_name": "Python 3",
|
| 243 |
+
"language": "python",
|
| 244 |
+
"name": "python3"
|
| 245 |
+
},
|
| 246 |
+
"language_info": {
|
| 247 |
+
"name": "python",
|
| 248 |
+
"pygments_lexer": "ipython3"
|
| 249 |
+
}
|
| 250 |
+
},
|
| 251 |
+
"nbformat": 4,
|
| 252 |
+
"nbformat_minor": 5
|
| 253 |
+
}
|
results/example_verification_all.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
results/example_verification_summary.json
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"total": 24,
|
| 3 |
+
"by_status": {
|
| 4 |
+
"clean": 24
|
| 5 |
+
},
|
| 6 |
+
"by_reject_reason": {
|
| 7 |
+
"clean": 24
|
| 8 |
+
},
|
| 9 |
+
"by_task": {
|
| 10 |
+
"h1_acquisition_defence": {
|
| 11 |
+
"clean": 24
|
| 12 |
+
}
|
| 13 |
+
},
|
| 14 |
+
"clean_avg_terminal_reward": 1.05,
|
| 15 |
+
"clean_avg_verifier_pass_rate": 1.0
|
| 16 |
+
}
|
scripts/_trajectory_utils.py
CHANGED
|
@@ -383,33 +383,134 @@ def oracle_actions(task_id: str, variant: int = 0) -> List[Dict[str, Any]]:
|
|
| 383 |
{"action_type": "finalize", "payload": json.dumps(phase)},
|
| 384 |
]
|
| 385 |
if task_id == "h1_acquisition_defence":
|
| 386 |
-
|
| 387 |
-
|
| 388 |
-
|
| 389 |
-
|
| 390 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 391 |
return [
|
| 392 |
-
{"action_type": "delegate", "agent_id": "cto", "payload":
|
| 393 |
-
{"action_type": "delegate", "agent_id": "cfo", "payload":
|
| 394 |
-
{"action_type": "delegate", "agent_id": "chro", "payload":
|
| 395 |
-
{"action_type": "log_reasoning", "payload":
|
| 396 |
{
|
| 397 |
"action_type": "log_conflict",
|
| 398 |
"payload": json.dumps(
|
| 399 |
{
|
| 400 |
"id": "c1",
|
| 401 |
-
"summary":
|
| 402 |
"source_agents": ["cto", "cfo"],
|
| 403 |
}
|
| 404 |
),
|
| 405 |
},
|
| 406 |
-
{"action_type": "log_reasoning", "payload":
|
| 407 |
{
|
| 408 |
"action_type": "log_conflict",
|
| 409 |
"payload": json.dumps(
|
| 410 |
{
|
| 411 |
"id": "c2",
|
| 412 |
-
"summary":
|
| 413 |
"source_agents": ["chro", "cto"],
|
| 414 |
}
|
| 415 |
),
|
|
@@ -420,17 +521,17 @@ def oracle_actions(task_id: str, variant: int = 0) -> List[Dict[str, Any]]:
|
|
| 420 |
{
|
| 421 |
"conflict_id": "c1",
|
| 422 |
"resolution_type": "bounded_counter",
|
| 423 |
-
"text":
|
| 424 |
}
|
| 425 |
),
|
| 426 |
},
|
| 427 |
{"action_type": "advance_phase", "payload": "analysis"},
|
| 428 |
-
{"action_type": "log_reasoning", "payload":
|
| 429 |
{"action_type": "advance_phase", "payload": "decision"},
|
| 430 |
-
{"action_type": "log_decision", "payload":
|
| 431 |
-
{"action_type": "log_reasoning", "payload":
|
| 432 |
{"action_type": "advance_phase", "payload": "execution"},
|
| 433 |
-
{"action_type": "log_reasoning", "payload":
|
| 434 |
{"action_type": "finalize", "payload": json.dumps(final)},
|
| 435 |
]
|
| 436 |
raise ValueError(f"unknown task_id: {task_id}")
|
|
|
|
| 383 |
{"action_type": "finalize", "payload": json.dumps(phase)},
|
| 384 |
]
|
| 385 |
if task_id == "h1_acquisition_defence":
|
| 386 |
+
# Rotating templates: paraphrase payloads while preserving verifier invariants
|
| 387 |
+
# (3 delegates, 2 conflicts, resolution_type, decisions mentioning runway/cash/7 month,
|
| 388 |
+
# final dict keys, phase progression, dense reasoning).
|
| 389 |
+
n = 12
|
| 390 |
+
v = variant % n
|
| 391 |
+
delegates = [
|
| 392 |
+
(
|
| 393 |
+
"Assess IP leverage and technical moat.",
|
| 394 |
+
"Assess cash runway and valuation ceiling.",
|
| 395 |
+
"Assess retention risk and people constraints.",
|
| 396 |
+
),
|
| 397 |
+
(
|
| 398 |
+
"Map product/IP defensibility and engineering dependency risk in the target.",
|
| 399 |
+
"Model valuation bands against board-approved cash and runway headroom.",
|
| 400 |
+
"Quantify people flight risk and retention levers for critical engineering staff.",
|
| 401 |
+
),
|
| 402 |
+
(
|
| 403 |
+
"Report whether technical differentiation justifies a premium vs the 2.3x bid.",
|
| 404 |
+
"Clarify how many months of runway remain if we stretch price or drag timing.",
|
| 405 |
+
"Clarify CHRO's view on 90-day people risk and hiring-market pressure.",
|
| 406 |
+
),
|
| 407 |
+
(
|
| 408 |
+
"Synthesize the CTO view on product moat and technical integration costs.",
|
| 409 |
+
"Synthesize the CFO view on the ceiling the board can defend with current cash.",
|
| 410 |
+
"Synthesize the CHRO view on retention, titles, and cultural integration risk.",
|
| 411 |
+
),
|
| 412 |
+
(
|
| 413 |
+
"CTO: outline minimum acceptable tech valuation given integration complexity.",
|
| 414 |
+
"CFO: outline a pricing corridor consistent with 7 month runway and board guardrails.",
|
| 415 |
+
"CHRO: outline retention and communication needs if diligence slips past the HR window.",
|
| 416 |
+
),
|
| 417 |
+
(
|
| 418 |
+
"CTO: stress-test whether acquirer can credibly match our roadmap without the team.",
|
| 419 |
+
"CFO: frame downside if we overpay vs observed comps and our burn profile.",
|
| 420 |
+
"CHRO: flag key engineers whose departure would void strategic upside from the deal.",
|
| 421 |
+
),
|
| 422 |
+
]
|
| 423 |
+
d0, d1, d2 = delegates[v % len(delegates)]
|
| 424 |
+
c1_sums = [
|
| 425 |
+
"CTO valuation ambition conflicts with CFO runway and cash constraints.",
|
| 426 |
+
"CTO's strategic premium clashes with cfo and finance on cash runway and board limits.",
|
| 427 |
+
"Engineering (cto) wants a high counter; finance (cfo) caps what cash runway allows.",
|
| 428 |
+
]
|
| 429 |
+
c2_sums = [
|
| 430 |
+
"A slow process increases CHRO retention risk for key engineering talent.",
|
| 431 |
+
"A drawn-out haggle raises chro and hr risk as competing offers come due.",
|
| 432 |
+
"chro and cto both warn: delay past the HR window erodes the engineering team.",
|
| 433 |
+
]
|
| 434 |
+
res_texts = [
|
| 435 |
+
"Ask above the CFO ceiling but set a fast deadline and walk-down logic.",
|
| 436 |
+
"Pursue a structured counter: higher opening ask with a board-mandated walk-down to 2.6x floor.",
|
| 437 |
+
"Use a time-boxed process: name a number above cfo’s ceiling, then step down on a defined clock.",
|
| 438 |
+
]
|
| 439 |
+
r_before_a = [
|
| 440 |
+
"CTO input supports a higher counter because the IP moat is meaningful.",
|
| 441 |
+
"The cto case implies we should not fold to 2.3x without a premium for IP.",
|
| 442 |
+
"Dev leadership sees differentiated tech, so a weak counter cedes leverage too early.",
|
| 443 |
+
]
|
| 444 |
+
r_after_c1 = [
|
| 445 |
+
"CFO input limits how long the negotiation can remain open.",
|
| 446 |
+
"Finance and cash runway cap how long we can posture before credibility breaks.",
|
| 447 |
+
"CFO: board cannot fund an endless auction with only about 7 month runway in reserve.",
|
| 448 |
+
]
|
| 449 |
+
r_runway = [
|
| 450 |
+
"The 7 month runway makes delay costly and should shape the deadline.",
|
| 451 |
+
"A 7 month cash runway means every extra week in diligence burns optionality and cash.",
|
| 452 |
+
"Runway and cash pressure force a firm deadline, not an open-ended beauty contest.",
|
| 453 |
+
]
|
| 454 |
+
r_retention = [
|
| 455 |
+
"Retention incentives are required before the market reads uncertainty.",
|
| 456 |
+
"chro: retention grants must land before the team interprets delay as a leadership stall.",
|
| 457 |
+
"hr signals that clarity on roles and comp must precede a public counter narrative.",
|
| 458 |
+
]
|
| 459 |
+
r_final = [
|
| 460 |
+
"Final recommendation integrates valuation, deadline, and retention plan.",
|
| 461 |
+
"Synthesize cto, cfo, and chro inputs into one executable counter, timeline, and people plan.",
|
| 462 |
+
"Close the loop: a bounded counter, a calendar-driven deadline, and a retention package.",
|
| 463 |
+
]
|
| 464 |
+
decision_line = [
|
| 465 |
+
"Proceed with a counter that acknowledges 7 month runway and cash limits.",
|
| 466 |
+
"Choose a path that fits both the 7 month cash runway and board finance constraints.",
|
| 467 |
+
"Commit: respect runway and cash reality while still pressing for a fair tech premium.",
|
| 468 |
+
]
|
| 469 |
+
finals = [
|
| 470 |
+
{
|
| 471 |
+
"counter_offer": "Open at 3.2x with a board-approved walk-down floor near 2.6x.",
|
| 472 |
+
"deadline": "Force a decision inside 45 days to preserve cash runway and reduce retention risk.",
|
| 473 |
+
"retention_plan": "Offer retention grants and role clarity for critical engineering leaders.",
|
| 474 |
+
},
|
| 475 |
+
{
|
| 476 |
+
"counter_offer": "Start at 3.15x with a staged retreat toward 2.65x if diligence stays clean.",
|
| 477 |
+
"deadline": "Cap negotiations at 50 days: balance hr timelines with cash runway and burn.",
|
| 478 |
+
"retention_plan": "Retention bonuses plus explicit reporting lines for VPs in core product.",
|
| 479 |
+
},
|
| 480 |
+
{
|
| 481 |
+
"counter_offer": "Signal 3.25x as the opening position with board ratified walk-down rights.",
|
| 482 |
+
"deadline": "Close or exit talks within 40 days; 7 month runway does not allow drift.",
|
| 483 |
+
"retention_plan": "Two-tier retention: cash now for chro-priority staff, earn-outs for the rest.",
|
| 484 |
+
},
|
| 485 |
+
{
|
| 486 |
+
"counter_offer": "Anchor at 3.1x, allow acquirer to meet near 2.7x with accelerated diligence.",
|
| 487 |
+
"deadline": "45-day triage: align cfo cash tests with cto and chro risk windows.",
|
| 488 |
+
"retention_plan": "Milestone-based retention: pay at signing, 90d, and close for key dev talent.",
|
| 489 |
+
},
|
| 490 |
+
]
|
| 491 |
+
final = finals[v % len(finals)]
|
| 492 |
return [
|
| 493 |
+
{"action_type": "delegate", "agent_id": "cto", "payload": d0},
|
| 494 |
+
{"action_type": "delegate", "agent_id": "cfo", "payload": d1},
|
| 495 |
+
{"action_type": "delegate", "agent_id": "chro", "payload": d2},
|
| 496 |
+
{"action_type": "log_reasoning", "payload": r_before_a[v % len(r_before_a)]},
|
| 497 |
{
|
| 498 |
"action_type": "log_conflict",
|
| 499 |
"payload": json.dumps(
|
| 500 |
{
|
| 501 |
"id": "c1",
|
| 502 |
+
"summary": c1_sums[v % len(c1_sums)],
|
| 503 |
"source_agents": ["cto", "cfo"],
|
| 504 |
}
|
| 505 |
),
|
| 506 |
},
|
| 507 |
+
{"action_type": "log_reasoning", "payload": r_after_c1[v % len(r_after_c1)]},
|
| 508 |
{
|
| 509 |
"action_type": "log_conflict",
|
| 510 |
"payload": json.dumps(
|
| 511 |
{
|
| 512 |
"id": "c2",
|
| 513 |
+
"summary": c2_sums[v % len(c2_sums)],
|
| 514 |
"source_agents": ["chro", "cto"],
|
| 515 |
}
|
| 516 |
),
|
|
|
|
| 521 |
{
|
| 522 |
"conflict_id": "c1",
|
| 523 |
"resolution_type": "bounded_counter",
|
| 524 |
+
"text": res_texts[v % len(res_texts)],
|
| 525 |
}
|
| 526 |
),
|
| 527 |
},
|
| 528 |
{"action_type": "advance_phase", "payload": "analysis"},
|
| 529 |
+
{"action_type": "log_reasoning", "payload": r_runway[v % len(r_runway)]},
|
| 530 |
{"action_type": "advance_phase", "payload": "decision"},
|
| 531 |
+
{"action_type": "log_decision", "payload": decision_line[v % len(decision_line)]},
|
| 532 |
+
{"action_type": "log_reasoning", "payload": r_retention[v % len(r_retention)]},
|
| 533 |
{"action_type": "advance_phase", "payload": "execution"},
|
| 534 |
+
{"action_type": "log_reasoning", "payload": r_final[v % len(r_final)]},
|
| 535 |
{"action_type": "finalize", "payload": json.dumps(final)},
|
| 536 |
]
|
| 537 |
raise ValueError(f"unknown task_id: {task_id}")
|
scripts/prepare_sft_data.py
CHANGED
|
@@ -1,8 +1,7 @@
|
|
| 1 |
"""Convert verified CORP-ENV trajectories into chat-format SFT JSONL.
|
| 2 |
|
| 3 |
-
|
| 4 |
-
`scripts/verify_examples.py`. Each output row is
|
| 5 |
-
SFT datasets:
|
| 6 |
|
| 7 |
{"task_id": "...", "example_id": "...", "messages": [...]}
|
| 8 |
"""
|
|
@@ -47,26 +46,64 @@ def convert_example(example: Dict[str, Any], min_pass_rate: float) -> Dict[str,
|
|
| 47 |
}
|
| 48 |
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
def main() -> None:
|
| 51 |
parser = argparse.ArgumentParser(description="Prepare chat SFT data from verified examples.")
|
| 52 |
-
|
| 53 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
parser.add_argument("--min-pass-rate", type=float, default=0.80)
|
| 55 |
args = parser.parse_args()
|
|
|
|
|
|
|
| 56 |
|
| 57 |
rows: List[Dict[str, Any]] = []
|
|
|
|
| 58 |
skipped = 0
|
| 59 |
-
for
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
except Exception as exc:
|
| 63 |
-
skipped += 1
|
| 64 |
-
print(f"skip {example.get('example_id', 'unknown')}: {exc}")
|
| 65 |
-
continue
|
| 66 |
-
if row is None:
|
| 67 |
-
skipped += 1
|
| 68 |
continue
|
| 69 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
write_jsonl(Path(args.output), rows)
|
| 72 |
print(f"Wrote {len(rows)} SFT conversations to {args.output}; skipped {skipped}.")
|
|
|
|
| 1 |
"""Convert verified CORP-ENV trajectories into chat-format SFT JSONL.
|
| 2 |
|
| 3 |
+
Pass one or more processed JSONLs (e.g. `e1_m1_clean` + `h1_seed_clean`) from
|
| 4 |
+
`scripts/verify_examples.py`. Each output row is TRL-style chat SFT data:
|
|
|
|
| 5 |
|
| 6 |
{"task_id": "...", "example_id": "...", "messages": [...]}
|
| 7 |
"""
|
|
|
|
| 46 |
}
|
| 47 |
|
| 48 |
|
| 49 |
+
def _parse_input_paths(raw: List[str]) -> List[Path]:
|
| 50 |
+
"""Expand comma-separated entries and return unique ordered paths."""
|
| 51 |
+
out: List[Path] = []
|
| 52 |
+
for part in raw:
|
| 53 |
+
for p in part.split(","):
|
| 54 |
+
p = p.strip()
|
| 55 |
+
if p:
|
| 56 |
+
out.append(Path(p))
|
| 57 |
+
return out
|
| 58 |
+
|
| 59 |
+
|
| 60 |
def main() -> None:
|
| 61 |
parser = argparse.ArgumentParser(description="Prepare chat SFT data from verified examples.")
|
| 62 |
+
default_inputs = (
|
| 63 |
+
"data/processed/e1_m1_clean.jsonl,data/processed/h1_seed_clean.jsonl"
|
| 64 |
+
)
|
| 65 |
+
parser.add_argument(
|
| 66 |
+
"--input",
|
| 67 |
+
dest="inputs",
|
| 68 |
+
action="append",
|
| 69 |
+
default=None,
|
| 70 |
+
metavar="PATH",
|
| 71 |
+
help=(
|
| 72 |
+
"Processed JSONL (repeat flag or use commas). "
|
| 73 |
+
f"Default: {default_inputs}"
|
| 74 |
+
),
|
| 75 |
+
)
|
| 76 |
+
parser.add_argument("--output", default="data/sft/e1_m1_h1_examples.jsonl")
|
| 77 |
parser.add_argument("--min-pass-rate", type=float, default=0.80)
|
| 78 |
args = parser.parse_args()
|
| 79 |
+
raw_inputs = list(args.inputs) if args.inputs else [default_inputs]
|
| 80 |
+
input_paths = _parse_input_paths(raw_inputs)
|
| 81 |
|
| 82 |
rows: List[Dict[str, Any]] = []
|
| 83 |
+
seen_ids: set[str] = set()
|
| 84 |
skipped = 0
|
| 85 |
+
for path in input_paths:
|
| 86 |
+
if not path.is_file():
|
| 87 |
+
print(f"warning: input missing, skip: {path}", file=sys.stderr)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
continue
|
| 89 |
+
for example in read_jsonl(path):
|
| 90 |
+
eid = str(example.get("example_id") or example.get("id") or "")
|
| 91 |
+
if eid and eid in seen_ids:
|
| 92 |
+
skipped += 1
|
| 93 |
+
continue
|
| 94 |
+
try:
|
| 95 |
+
row = convert_example(example, args.min_pass_rate)
|
| 96 |
+
except Exception as exc:
|
| 97 |
+
skipped += 1
|
| 98 |
+
print(f"skip {example.get('example_id', 'unknown')}: {exc}")
|
| 99 |
+
continue
|
| 100 |
+
if row is None:
|
| 101 |
+
skipped += 1
|
| 102 |
+
continue
|
| 103 |
+
rows.append(row)
|
| 104 |
+
eid2 = str(row.get("example_id") or "unknown")
|
| 105 |
+
if eid2 and eid2 != "unknown":
|
| 106 |
+
seen_ids.add(eid2)
|
| 107 |
|
| 108 |
write_jsonl(Path(args.output), rows)
|
| 109 |
print(f"Wrote {len(rows)} SFT conversations to {args.output}; skipped {skipped}.")
|
tests/test_pipeline.py
CHANGED
|
@@ -32,6 +32,17 @@ class PipelineSmokeTests(unittest.TestCase):
|
|
| 32 |
self.assertEqual(result.status, "clean")
|
| 33 |
self.assertGreaterEqual(result.verifier_pass_rate, 5 / 6)
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
def test_invalid_trajectory_is_rejected(self) -> None:
|
| 36 |
result = replay_actions(
|
| 37 |
example_id="bad",
|
|
|
|
| 32 |
self.assertEqual(result.status, "clean")
|
| 33 |
self.assertGreaterEqual(result.verifier_pass_rate, 5 / 6)
|
| 34 |
|
| 35 |
+
def test_oracle_h1_variants_replay_cleanly(self) -> None:
|
| 36 |
+
for v in (0, 5, 11):
|
| 37 |
+
with self.subTest(variant=v):
|
| 38 |
+
result = replay_actions(
|
| 39 |
+
example_id=f"test-h1-{v}",
|
| 40 |
+
task_id="h1_acquisition_defence",
|
| 41 |
+
actions=oracle_actions("h1_acquisition_defence", v),
|
| 42 |
+
)
|
| 43 |
+
self.assertEqual(result.status, "clean")
|
| 44 |
+
self.assertGreaterEqual(result.verifier_pass_rate, 7 / 11)
|
| 45 |
+
|
| 46 |
def test_invalid_trajectory_is_rejected(self) -> None:
|
| 47 |
result = replay_actions(
|
| 48 |
example_id="bad",
|
training/train_grpo.py
CHANGED
|
@@ -11,9 +11,9 @@ Run on Colab, Lightning AI H100, or another GPU machine after SFT:
|
|
| 11 |
python training/train_grpo.py \
|
| 12 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 13 |
--adapter outputs/sft_Qwen2.5-7B-Instruct \
|
| 14 |
-
--examples data/processed/e1_m1_clean.jsonl \
|
| 15 |
--output outputs/grpo_Qwen2.5-7B-Instruct \
|
| 16 |
-
--max-steps
|
| 17 |
--push-to-hub your-org/corp-gym-grpo-qwen2.5-7b
|
| 18 |
|
| 19 |
The reward function recreates the environment state from a verified action prefix,
|
|
@@ -30,6 +30,11 @@ Optional speedups (Unsloth picks these up automatically when importable):
|
|
| 30 |
- **xFormers**: already used as a fallback when FA2 is missing; still slower than FA2.
|
| 31 |
- **Qwen3.x “linear attention” fast path**: only for those architectures; not used for
|
| 32 |
Qwen2.5 GRPO. See Unsloth logs if you train Qwen3.5+.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
"""
|
| 34 |
|
| 35 |
from __future__ import annotations
|
|
@@ -72,12 +77,17 @@ def prompt_for_prefix(task_id: str, prefix_actions: List[Dict[str, Any]]) -> Lis
|
|
| 72 |
return messages
|
| 73 |
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
def build_prompt_dataset(examples_path: str, tasks: List[str], repeats: int) -> List[Dict[str, Any]]:
|
|
|
|
| 76 |
rows: List[Dict[str, Any]] = []
|
| 77 |
-
path
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
for example in
|
| 81 |
if example.get("status") and example.get("status") != "clean":
|
| 82 |
continue
|
| 83 |
task_id = str(example.get("task_id") or "")
|
|
@@ -157,9 +167,16 @@ def main() -> None:
|
|
| 157 |
parser = argparse.ArgumentParser(description="Train CORP-ENV GRPO adapter.")
|
| 158 |
parser.add_argument("--model", default="Qwen/Qwen2.5-7B-Instruct")
|
| 159 |
parser.add_argument("--adapter", default="outputs/sft_Qwen2.5-7B-Instruct")
|
| 160 |
-
parser.add_argument(
|
|
|
|
|
|
|
|
|
|
|
|
|
| 161 |
parser.add_argument("--output", default="outputs/grpo_Qwen2.5-7B-Instruct")
|
| 162 |
-
parser.add_argument(
|
|
|
|
|
|
|
|
|
|
| 163 |
parser.add_argument(
|
| 164 |
"--repeats",
|
| 165 |
type=int,
|
|
@@ -180,8 +197,8 @@ def main() -> None:
|
|
| 180 |
parser.add_argument(
|
| 181 |
"--max-steps",
|
| 182 |
type=int,
|
| 183 |
-
default=
|
| 184 |
-
help="Optimizer steps (not env episodes).
|
| 185 |
)
|
| 186 |
parser.add_argument(
|
| 187 |
"--save-steps",
|
|
@@ -190,6 +207,18 @@ def main() -> None:
|
|
| 190 |
help="Checkpoint every N steps. Default: min(25, max(5, max_steps//2)) so short runs still save.",
|
| 191 |
)
|
| 192 |
parser.add_argument("--optim", default="adamw_8bit")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 193 |
parser.add_argument("--push-to-hub", default="")
|
| 194 |
args = parser.parse_args()
|
| 195 |
|
|
@@ -242,24 +271,29 @@ def main() -> None:
|
|
| 242 |
random_state=3407,
|
| 243 |
)
|
| 244 |
|
| 245 |
-
|
| 246 |
-
output_dir
|
| 247 |
-
learning_rate
|
| 248 |
-
per_device_train_batch_size
|
| 249 |
-
gradient_accumulation_steps
|
| 250 |
-
num_generations
|
| 251 |
-
max_prompt_length
|
| 252 |
-
max_completion_length
|
| 253 |
-
max_steps
|
| 254 |
-
logging_steps
|
| 255 |
-
save_steps
|
| 256 |
-
save_total_limit
|
| 257 |
-
optim
|
| 258 |
-
bf16
|
| 259 |
-
report_to
|
| 260 |
-
push_to_hub
|
| 261 |
-
hub_model_id
|
| 262 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 263 |
trainer = GRPOTrainer(
|
| 264 |
model=model,
|
| 265 |
processing_class=tokenizer,
|
|
|
|
| 11 |
python training/train_grpo.py \
|
| 12 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 13 |
--adapter outputs/sft_Qwen2.5-7B-Instruct \
|
| 14 |
+
--examples data/processed/e1_m1_clean.jsonl,data/processed/h1_seed_clean.jsonl \
|
| 15 |
--output outputs/grpo_Qwen2.5-7B-Instruct \
|
| 16 |
+
--max-steps 150 \
|
| 17 |
--push-to-hub your-org/corp-gym-grpo-qwen2.5-7b
|
| 18 |
|
| 19 |
The reward function recreates the environment state from a verified action prefix,
|
|
|
|
| 30 |
- **xFormers**: already used as a fallback when FA2 is missing; still slower than FA2.
|
| 31 |
- **Qwen3.x “linear attention” fast path**: only for those architectures; not used for
|
| 32 |
Qwen2.5 GRPO. See Unsloth logs if you train Qwen3.5+.
|
| 33 |
+
|
| 34 |
+
On a Linux H100 (e.g. Lightning), try **larger** `--batch-size` if memory allows, and
|
| 35 |
+
`--dataloader-num-workers 2`–`4` (Windows often keeps 0) so the `CorpEnvironment` rollouts
|
| 36 |
+
are not the only bottleneck. Use `--max-steps` (default 150) and `--generations` to trade
|
| 37 |
+
quality vs wall-clock.
|
| 38 |
"""
|
| 39 |
|
| 40 |
from __future__ import annotations
|
|
|
|
| 77 |
return messages
|
| 78 |
|
| 79 |
|
| 80 |
+
def _examples_paths(examples_path: str) -> List[Path]:
|
| 81 |
+
return [Path(p.strip()) for p in examples_path.split(",") if p.strip()]
|
| 82 |
+
|
| 83 |
+
|
| 84 |
def build_prompt_dataset(examples_path: str, tasks: List[str], repeats: int) -> List[Dict[str, Any]]:
|
| 85 |
+
"""Load examples from one or more JSONLs (comma-separated). Falls back to oracle prefixes if all missing/empty."""
|
| 86 |
rows: List[Dict[str, Any]] = []
|
| 87 |
+
for path in _examples_paths(examples_path):
|
| 88 |
+
if not path.exists():
|
| 89 |
+
continue
|
| 90 |
+
for example in read_jsonl(path):
|
| 91 |
if example.get("status") and example.get("status") != "clean":
|
| 92 |
continue
|
| 93 |
task_id = str(example.get("task_id") or "")
|
|
|
|
| 167 |
parser = argparse.ArgumentParser(description="Train CORP-ENV GRPO adapter.")
|
| 168 |
parser.add_argument("--model", default="Qwen/Qwen2.5-7B-Instruct")
|
| 169 |
parser.add_argument("--adapter", default="outputs/sft_Qwen2.5-7B-Instruct")
|
| 170 |
+
parser.add_argument(
|
| 171 |
+
"--examples",
|
| 172 |
+
default="data/processed/e1_m1_clean.jsonl,data/processed/h1_seed_clean.jsonl",
|
| 173 |
+
help="One JSONL or comma-separated list of verified (clean) trajectory files.",
|
| 174 |
+
)
|
| 175 |
parser.add_argument("--output", default="outputs/grpo_Qwen2.5-7B-Instruct")
|
| 176 |
+
parser.add_argument(
|
| 177 |
+
"--tasks",
|
| 178 |
+
default="e1_launch_readiness,m1_budget_reallocation,h1_acquisition_defence",
|
| 179 |
+
)
|
| 180 |
parser.add_argument(
|
| 181 |
"--repeats",
|
| 182 |
type=int,
|
|
|
|
| 197 |
parser.add_argument(
|
| 198 |
"--max-steps",
|
| 199 |
type=int,
|
| 200 |
+
default=150,
|
| 201 |
+
help="Optimizer steps (not env episodes). Default 150 for a fuller RL fit on H100-style runs.",
|
| 202 |
)
|
| 203 |
parser.add_argument(
|
| 204 |
"--save-steps",
|
|
|
|
| 207 |
help="Checkpoint every N steps. Default: min(25, max(5, max_steps//2)) so short runs still save.",
|
| 208 |
)
|
| 209 |
parser.add_argument("--optim", default="adamw_8bit")
|
| 210 |
+
parser.add_argument(
|
| 211 |
+
"--dataloader-num-workers",
|
| 212 |
+
type=int,
|
| 213 |
+
default=0,
|
| 214 |
+
help="DataLoader workers (0 is safest on Windows; try 2–4 on Linux H100 if CPU allows).",
|
| 215 |
+
)
|
| 216 |
+
parser.add_argument(
|
| 217 |
+
"--dataloader-prefetch-factor",
|
| 218 |
+
type=int,
|
| 219 |
+
default=None,
|
| 220 |
+
help="When dataloader_num_workers>0, optional prefetch depth (e.g. 2).",
|
| 221 |
+
)
|
| 222 |
parser.add_argument("--push-to-hub", default="")
|
| 223 |
args = parser.parse_args()
|
| 224 |
|
|
|
|
| 271 |
random_state=3407,
|
| 272 |
)
|
| 273 |
|
| 274 |
+
_gc_kwargs: Dict[str, Any] = {
|
| 275 |
+
"output_dir": args.output,
|
| 276 |
+
"learning_rate": args.lr,
|
| 277 |
+
"per_device_train_batch_size": args.batch_size,
|
| 278 |
+
"gradient_accumulation_steps": args.grad_accum,
|
| 279 |
+
"num_generations": args.generations,
|
| 280 |
+
"max_prompt_length": args.max_prompt_length,
|
| 281 |
+
"max_completion_length": args.max_completion_length,
|
| 282 |
+
"max_steps": args.max_steps,
|
| 283 |
+
"logging_steps": 5,
|
| 284 |
+
"save_steps": save_steps,
|
| 285 |
+
"save_total_limit": 3,
|
| 286 |
+
"optim": args.optim,
|
| 287 |
+
"bf16": True,
|
| 288 |
+
"report_to": "none",
|
| 289 |
+
"push_to_hub": bool(args.push_to_hub),
|
| 290 |
+
"hub_model_id": args.push_to_hub or None,
|
| 291 |
+
}
|
| 292 |
+
if args.dataloader_num_workers:
|
| 293 |
+
_gc_kwargs["dataloader_num_workers"] = args.dataloader_num_workers
|
| 294 |
+
if args.dataloader_prefetch_factor is not None and args.dataloader_num_workers:
|
| 295 |
+
_gc_kwargs["dataloader_prefetch_factor"] = args.dataloader_prefetch_factor
|
| 296 |
+
config = GRPOConfig(**_gc_kwargs)
|
| 297 |
trainer = GRPOTrainer(
|
| 298 |
model=model,
|
| 299 |
processing_class=tokenizer,
|
training/train_sft.py
CHANGED
|
@@ -4,12 +4,14 @@ This is the hackathon SFT training script. It uses:
|
|
| 4 |
|
| 5 |
- `unsloth.FastLanguageModel` for efficient 4-bit LoRA/QLoRA loading.
|
| 6 |
- `trl.SFTTrainer` / `trl.SFTConfig` for supervised fine-tuning.
|
|
|
|
|
|
|
| 7 |
|
| 8 |
Run on Colab, Lightning AI H100, or another GPU machine:
|
| 9 |
|
| 10 |
python training/train_sft.py \
|
| 11 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 12 |
-
--data data/sft/
|
| 13 |
--output outputs/sft_adapter \
|
| 14 |
--max-steps 30 \
|
| 15 |
--push-to-hub your-org/corp-env-sft-adapter
|
|
@@ -18,9 +20,37 @@ Run on Colab, Lightning AI H100, or another GPU machine:
|
|
| 18 |
from __future__ import annotations
|
| 19 |
|
| 20 |
import argparse
|
|
|
|
| 21 |
import json
|
|
|
|
| 22 |
from pathlib import Path
|
| 23 |
-
from typing import Dict, List
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
|
| 26 |
def load_text_rows(path: Path, tokenizer: object) -> List[Dict[str, str]]:
|
|
@@ -40,10 +70,81 @@ def load_text_rows(path: Path, tokenizer: object) -> List[Dict[str, str]]:
|
|
| 40 |
return rows
|
| 41 |
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
def main() -> None:
|
| 44 |
parser = argparse.ArgumentParser(description="Train CORP-ENV SFT LoRA adapter.")
|
| 45 |
parser.add_argument("--model", default="Qwen/Qwen2.5-7B-Instruct")
|
| 46 |
-
parser.add_argument("--data", default="data/sft/
|
| 47 |
parser.add_argument("--output", default="outputs/sft_adapter")
|
| 48 |
parser.add_argument("--max-seq-length", type=int, default=8192)
|
| 49 |
parser.add_argument("--epochs", type=float, default=2.0)
|
|
@@ -54,19 +155,61 @@ def main() -> None:
|
|
| 54 |
parser.add_argument("--save-steps", type=int, default=50)
|
| 55 |
parser.add_argument("--optim", default="adamw_8bit")
|
| 56 |
parser.add_argument("--push-to-hub", default="")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
args = parser.parse_args()
|
| 58 |
|
| 59 |
try:
|
| 60 |
-
# Unsloth must load before trl/transformers/peft for full optimizations.
|
| 61 |
from unsloth import FastLanguageModel
|
| 62 |
from datasets import Dataset
|
| 63 |
-
from trl import SFTConfig, SFTTrainer
|
| 64 |
except ImportError as exc:
|
| 65 |
raise SystemExit(
|
| 66 |
"SFT training requires datasets, trl, and unsloth. On Lightning AI, install with:\n"
|
| 67 |
-
" pip install -
|
| 68 |
) from exc
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 71 |
model_name=args.model,
|
| 72 |
max_seq_length=args.max_seq_length,
|
|
@@ -97,34 +240,16 @@ def main() -> None:
|
|
| 97 |
random_state=3407,
|
| 98 |
)
|
| 99 |
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
lr_scheduler_type="cosine",
|
| 111 |
-
logging_steps=5,
|
| 112 |
-
save_steps=args.save_steps,
|
| 113 |
-
save_total_limit=3,
|
| 114 |
-
max_steps=args.max_steps,
|
| 115 |
-
optim=args.optim,
|
| 116 |
-
bf16=True,
|
| 117 |
-
packing=False,
|
| 118 |
-
report_to="none",
|
| 119 |
-
push_to_hub=bool(args.push_to_hub),
|
| 120 |
-
hub_model_id=args.push_to_hub or None,
|
| 121 |
-
)
|
| 122 |
-
trainer = SFTTrainer(
|
| 123 |
-
model=model,
|
| 124 |
-
tokenizer=tokenizer,
|
| 125 |
-
train_dataset=dataset,
|
| 126 |
-
args=config,
|
| 127 |
-
)
|
| 128 |
trainer.train()
|
| 129 |
trainer.save_model(args.output)
|
| 130 |
tokenizer.save_pretrained(args.output)
|
|
|
|
| 4 |
|
| 5 |
- `unsloth.FastLanguageModel` for efficient 4-bit LoRA/QLoRA loading.
|
| 6 |
- `trl.SFTTrainer` / `trl.SFTConfig` for supervised fine-tuning.
|
| 7 |
+
- `messages`-format JSONL with TRL 0.2x conversational SFT (optional `--assistant-only` if the
|
| 8 |
+
model chat template supports assistant token masks; Qwen2.5 Instruct defaults to off).
|
| 9 |
|
| 10 |
Run on Colab, Lightning AI H100, or another GPU machine:
|
| 11 |
|
| 12 |
python training/train_sft.py \
|
| 13 |
--model Qwen/Qwen2.5-7B-Instruct \
|
| 14 |
+
--data data/sft/e1_m1_h1_examples.jsonl \
|
| 15 |
--output outputs/sft_adapter \
|
| 16 |
--max-steps 30 \
|
| 17 |
--push-to-hub your-org/corp-env-sft-adapter
|
|
|
|
| 20 |
from __future__ import annotations
|
| 21 |
|
| 22 |
import argparse
|
| 23 |
+
import inspect
|
| 24 |
import json
|
| 25 |
+
from dataclasses import fields
|
| 26 |
from pathlib import Path
|
| 27 |
+
from typing import Any, Dict, List
|
| 28 |
+
|
| 29 |
+
import torch # imported before unsloth in main for dtype hooks
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def _sft_config_field_names() -> set[str]:
|
| 33 |
+
from trl import SFTConfig
|
| 34 |
+
|
| 35 |
+
if hasattr(SFTConfig, "__dataclass_fields__"):
|
| 36 |
+
return set(SFTConfig.__dataclass_fields__.keys())
|
| 37 |
+
return {f.name for f in fields(SFTConfig)}
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def load_conversation_rows(path: Path) -> List[Dict[str, Any]]:
|
| 41 |
+
"""TRL conversational format: each row has a `messages` list."""
|
| 42 |
+
rows: List[Dict[str, Any]] = []
|
| 43 |
+
with path.open("r", encoding="utf-8") as f:
|
| 44 |
+
for line in f:
|
| 45 |
+
if not line.strip():
|
| 46 |
+
continue
|
| 47 |
+
obj = json.loads(line)
|
| 48 |
+
if "messages" not in obj:
|
| 49 |
+
raise SystemExit(
|
| 50 |
+
f"{path}: SFT example missing 'messages' (use --legacy-text for old format)."
|
| 51 |
+
)
|
| 52 |
+
rows.append({"messages": obj["messages"]})
|
| 53 |
+
return rows
|
| 54 |
|
| 55 |
|
| 56 |
def load_text_rows(path: Path, tokenizer: object) -> List[Dict[str, str]]:
|
|
|
|
| 70 |
return rows
|
| 71 |
|
| 72 |
|
| 73 |
+
def _build_sft_config(
|
| 74 |
+
allowed: set[str],
|
| 75 |
+
output_dir: str,
|
| 76 |
+
max_seq: int,
|
| 77 |
+
args: argparse.Namespace,
|
| 78 |
+
) -> Any:
|
| 79 |
+
from trl import SFTConfig
|
| 80 |
+
|
| 81 |
+
candidate: Dict[str, Any] = {
|
| 82 |
+
"output_dir": output_dir,
|
| 83 |
+
"max_length": max_seq,
|
| 84 |
+
"per_device_train_batch_size": args.batch_size,
|
| 85 |
+
"gradient_accumulation_steps": args.grad_accum,
|
| 86 |
+
"num_train_epochs": args.epochs,
|
| 87 |
+
"learning_rate": args.lr,
|
| 88 |
+
"warmup_ratio": 0.05,
|
| 89 |
+
"lr_scheduler_type": "cosine",
|
| 90 |
+
"logging_steps": 5,
|
| 91 |
+
"save_steps": args.save_steps,
|
| 92 |
+
"save_total_limit": 3,
|
| 93 |
+
"max_steps": args.max_steps,
|
| 94 |
+
"optim": args.optim,
|
| 95 |
+
"bf16": (not args.fp16) and torch.cuda.is_available(),
|
| 96 |
+
"fp16": bool(args.fp16) and torch.cuda.is_available(),
|
| 97 |
+
"packing": args.packing,
|
| 98 |
+
"report_to": "none",
|
| 99 |
+
"push_to_hub": bool(args.push_to_hub),
|
| 100 |
+
"hub_model_id": args.push_to_hub or None,
|
| 101 |
+
"assistant_only_loss": args.assistant_only,
|
| 102 |
+
}
|
| 103 |
+
if args.dataset_num_proc is not None and "dataset_num_proc" in allowed:
|
| 104 |
+
candidate["dataset_num_proc"] = args.dataset_num_proc
|
| 105 |
+
if args.dataloader_num_workers and "dataloader_num_workers" in allowed:
|
| 106 |
+
candidate["dataloader_num_workers"] = args.dataloader_num_workers
|
| 107 |
+
if args.packing and "padding_free" in allowed and args.padding_free:
|
| 108 |
+
candidate["padding_free"] = True
|
| 109 |
+
if args.legacy_text and "dataset_text_field" in allowed:
|
| 110 |
+
candidate["dataset_text_field"] = "text"
|
| 111 |
+
|
| 112 |
+
kwargs = {k: v for k, v in candidate.items() if k in allowed}
|
| 113 |
+
if "output_dir" not in kwargs:
|
| 114 |
+
kwargs["output_dir"] = output_dir
|
| 115 |
+
return SFTConfig(**kwargs)
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def _build_trainer(
|
| 119 |
+
model: object,
|
| 120 |
+
tokenizer: object,
|
| 121 |
+
config: object,
|
| 122 |
+
dataset: object,
|
| 123 |
+
) -> Any:
|
| 124 |
+
from trl import SFTTrainer
|
| 125 |
+
|
| 126 |
+
sig = inspect.signature(SFTTrainer.__init__)
|
| 127 |
+
if "processing_class" in sig.parameters:
|
| 128 |
+
return SFTTrainer(
|
| 129 |
+
model=model,
|
| 130 |
+
args=config,
|
| 131 |
+
train_dataset=dataset,
|
| 132 |
+
processing_class=tokenizer,
|
| 133 |
+
)
|
| 134 |
+
if "tokenizer" in sig.parameters:
|
| 135 |
+
return SFTTrainer(
|
| 136 |
+
model=model,
|
| 137 |
+
args=config,
|
| 138 |
+
train_dataset=dataset,
|
| 139 |
+
tokenizer=tokenizer,
|
| 140 |
+
)
|
| 141 |
+
raise SystemExit("SFTTrainer: expected processing_class or tokenizer in __init__.")
|
| 142 |
+
|
| 143 |
+
|
| 144 |
def main() -> None:
|
| 145 |
parser = argparse.ArgumentParser(description="Train CORP-ENV SFT LoRA adapter.")
|
| 146 |
parser.add_argument("--model", default="Qwen/Qwen2.5-7B-Instruct")
|
| 147 |
+
parser.add_argument("--data", default="data/sft/e1_m1_h1_examples.jsonl")
|
| 148 |
parser.add_argument("--output", default="outputs/sft_adapter")
|
| 149 |
parser.add_argument("--max-seq-length", type=int, default=8192)
|
| 150 |
parser.add_argument("--epochs", type=float, default=2.0)
|
|
|
|
| 155 |
parser.add_argument("--save-steps", type=int, default=50)
|
| 156 |
parser.add_argument("--optim", default="adamw_8bit")
|
| 157 |
parser.add_argument("--push-to-hub", default="")
|
| 158 |
+
parser.add_argument(
|
| 159 |
+
"--fp16",
|
| 160 |
+
action="store_true",
|
| 161 |
+
help="Use fp16 instead of bf16 (bf16 is default on CUDA when available).",
|
| 162 |
+
)
|
| 163 |
+
parser.add_argument(
|
| 164 |
+
"--legacy-text",
|
| 165 |
+
action="store_true",
|
| 166 |
+
help="Pre-tokenize to a single 'text' column (older path). Default: TRL messages format.",
|
| 167 |
+
)
|
| 168 |
+
parser.add_argument(
|
| 169 |
+
"--assistant-only",
|
| 170 |
+
action="store_true",
|
| 171 |
+
help=(
|
| 172 |
+
"Loss only on assistant tokens (requires tokenizer chat template with assistant masks). "
|
| 173 |
+
"Qwen2.5 usually needs this off; use with templates that support {% generation %}."
|
| 174 |
+
),
|
| 175 |
+
)
|
| 176 |
+
parser.add_argument(
|
| 177 |
+
"--packing",
|
| 178 |
+
action="store_true",
|
| 179 |
+
help="Sequence packing (faster; avoid for very long or uneven traces unless you know the tradeoffs).",
|
| 180 |
+
)
|
| 181 |
+
parser.add_argument(
|
| 182 |
+
"--padding-free",
|
| 183 |
+
action="store_true",
|
| 184 |
+
help="With packing+bfd, TRL can use padding-free; requires FlashAttention-friendly setup.",
|
| 185 |
+
)
|
| 186 |
+
parser.add_argument(
|
| 187 |
+
"--dataset-num-proc",
|
| 188 |
+
type=int,
|
| 189 |
+
default=4,
|
| 190 |
+
help="Multiprocess dataset preprocessing; set 0 to disable where supported (default 4 for H100).",
|
| 191 |
+
)
|
| 192 |
+
parser.add_argument(
|
| 193 |
+
"--dataloader-num-workers",
|
| 194 |
+
type=int,
|
| 195 |
+
default=2,
|
| 196 |
+
help="DataLoader workers (default 2; increase on H100 if CPU allows).",
|
| 197 |
+
)
|
| 198 |
args = parser.parse_args()
|
| 199 |
|
| 200 |
try:
|
|
|
|
| 201 |
from unsloth import FastLanguageModel
|
| 202 |
from datasets import Dataset
|
|
|
|
| 203 |
except ImportError as exc:
|
| 204 |
raise SystemExit(
|
| 205 |
"SFT training requires datasets, trl, and unsloth. On Lightning AI, install with:\n"
|
| 206 |
+
" pip install -e \".[training]\""
|
| 207 |
) from exc
|
| 208 |
|
| 209 |
+
allowed = _sft_config_field_names()
|
| 210 |
+
if args.dataset_num_proc == 0 and "dataset_num_proc" in allowed:
|
| 211 |
+
args = argparse.Namespace(**{**vars(args), "dataset_num_proc": None})
|
| 212 |
+
|
| 213 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 214 |
model_name=args.model,
|
| 215 |
max_seq_length=args.max_seq_length,
|
|
|
|
| 240 |
random_state=3407,
|
| 241 |
)
|
| 242 |
|
| 243 |
+
if args.legacy_text:
|
| 244 |
+
rows = load_text_rows(Path(args.data), tokenizer)
|
| 245 |
+
else:
|
| 246 |
+
rows = load_conversation_rows(Path(args.data))
|
| 247 |
+
if not rows:
|
| 248 |
+
raise SystemExit(f"No training rows in {args.data!r}.")
|
| 249 |
+
|
| 250 |
+
dataset = Dataset.from_list(rows)
|
| 251 |
+
config = _build_sft_config(allowed, args.output, args.max_seq_length, args)
|
| 252 |
+
trainer = _build_trainer(model, tokenizer, config, dataset)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 253 |
trainer.train()
|
| 254 |
trainer.save_model(args.output)
|
| 255 |
tokenizer.save_pretrained(args.output)
|