Spaces:
Runtime error
Runtime error
File size: 4,208 Bytes
fb68239 f016eb7 fb68239 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | # Synthetic Data
OpenRange includes a snapshot-backed synthetic trajectory generator for SFT warm-start and offline data collection. The design is influenced by Open Trajectory Gym's split between world specification, executor, and teacher model, but it is implemented in the OpenRange training layer so it stays aligned with the existing `SnapshotSpec`, `RangeEnvironment`, and `TrajectoryLogger` types.
## Why It Lives In `training/`
Synthetic trace generation is a training concern, not a runtime concern:
- The live server still owns real `reset()` / `step()` episodes on Docker infrastructure.
- Synthetic generation reuses the same `SnapshotSpec` and reward/meta-command semantics, but swaps Docker execution for a fast simulator.
- Export still goes through `TrajectoryLogger`, so downstream SFT JSONL format does not fork.
This keeps OpenRange's real environment and synthetic data path close enough to share prompts, actions, and episode structure without turning the production server into a data-generation service.
## Components
- `SyntheticRangeEnvironment`: a fast `RangeEnvironment` subclass that simulates common Red and Blue commands from a loaded snapshot.
- `SyntheticTraceGenerator`: drives Red and Blue agents through synthetic episodes and records them with `TrajectoryLogger`.
- `build_teacher_agents()`: constructs LiteLLM-backed teacher agents for selected roles and scripted fallbacks for the rest.
- `randomize_snapshot_flags()`: clones a snapshot and rewrites flag values per episode so traces do not memorize static flag strings.
## LiteLLM Support
Install the optional dependency:
```bash
uv sync --extra synthetic
```
Any LiteLLM model string supported by `LLMRangeAgent` works. For Azure OpenAI, export the usual LiteLLM/Azure variables and pass the deployment name as the model:
```bash
export AZURE_API_KEY=...
export AZURE_API_BASE=...
export AZURE_API_VERSION=...
uv run openrange synthetic-data \
--manifest manifests/tier1_basic.yaml \
--output data/sft_red.jsonl \
--roles red \
--teacher-model azure/gpt-5.2-codex
```
Codex-style Azure deployments often reject `temperature`; `LLMRangeAgent` now omits it automatically for model names containing `codex`.
## CLI
Generate traces from an existing snapshot:
```bash
uv run openrange synthetic-data \
--snapshot snapshots/spec.json \
--output data/sft_red.jsonl \
--num-traces 25 \
--roles red
```
Merge previously collected bootstrap traces and append a reusable tool catalog to generated system prompts:
```bash
uv run openrange synthetic-data \
--manifest manifests/tier1_basic.yaml \
--output data/synthetic_sft_5.jsonl \
--num-traces 5 \
--roles red \
--teacher-model azure/gpt-5.2-codex \
--bootstrap-traces data/sft.jsonl \
--tool-info data/tool_info.md
```
Generate traces from a manifest using the deterministic builder:
```bash
uv run openrange synthetic-data \
--manifest manifests/tier1_basic.yaml \
--output data/sft_red_blue.jsonl \
--roles red,blue \
--num-traces 50
```
Generate traces from a manifest using both an LLM builder and LLM teachers:
```bash
uv run openrange synthetic-data \
--manifest manifests/tier1_basic.yaml \
--llm-builder \
--builder-model azure/gpt-5.2-codex \
--teacher-model azure/gpt-5.2-codex \
--roles red \
--output data/frontier_red.jsonl
```
## Python API
```python
from open_range.training import SyntheticTraceGenerator, build_teacher_agents
red, blue = build_teacher_agents(
teacher_model="azure/gpt-5.2-codex",
roles=("red",),
max_tokens=256,
)
generator = SyntheticTraceGenerator.from_manifest(
manifest=tier1_manifest,
red_agent=red,
blue_agent=blue,
template_only=True,
max_steps=8,
)
logger, lines = generator.export_jsonl(
"data/sft_red.jsonl",
num_traces=10,
roles=("red",),
)
```
## Testing
Unit coverage lives in `tests/test_synthetic.py`.
There is also a gated live-model smoke test that exercises the synthetic generator against a real LiteLLM model:
```bash
uv run --extra synthetic pytest tests/test_synthetic.py -m live_model -q
```
The live test is skipped automatically unless the required Azure environment variables are present.
|