---
status: accepted
date: 2026-05-29
deciders: [Codeseys, ARIA]
---

# ADR-010: Build a FeatureDeletionEnv synthetic-data subsystem over OSS SWE substrates

## Context and Problem Statement

Composer 2.5 trained on "25× more synthetic tasks than Composer 2," with
**Feature Deletion** as a named generator: take a repo with passing tests,
delete code/features, the agent must reimplement to make the tests pass —
tests are the verifiable reward. The Cursor blog also notes the data is a
**dynamic difficulty curriculum** ("we both select for and create harder
tasks dynamically throughout the run"); the Composer 2 tech report adds the
curriculum is keyed on rollout #turns + thinking-token count
(`research/10-composer2-techreport-mining.md`). The blog reports real
reward-hacking (decompiling Java bytecode, reverse-engineering Python
type-check caches to recover deleted signatures) mitigated by "agentic
monitoring tools."

The framework has **no synthetic-data-generation subsystem at all** — this is
genuinely greenfield. Codeseys' ask is to bring Composer's dataset-generation
approach into the framework as a real, reusable subsystem (useful beyond this
project). Research doc `research/06-feature-deletion-datagen.md` designs it
and identifies ready OSS substrates: SWE-bench-Lite, SWE-Gym (2.4k), R2E-Gym
(8.1k), SWE-rebench (21.3k), Nemotron/OpenHands (59k) — each ships
`(repo, gold_patch, FAIL_TO_PASS, PASS_TO_PASS, test_cmd)` tuples that invert
directly into Feature-Deletion tasks (revert the gold patch → broken repo;
`FAIL_TO_PASS` tests = the reward signal; `PASS_TO_PASS` = the don't-break-
existing-behavior guard).

### Relationship to prior ADRs — `preserves`

No prior ADR governs synthetic data generation. ADR-002 (trace source) chose
Claude Code JSONL for *ingestion of real traces* — orthogonal to *generating*
tasks. This ADR preserves everything; it adds a new `composer_replication.datagen`
package.

## Decision Drivers

- Reuse existing verified OSS substrates rather than scraping repos from scratch — they already guarantee test-exercises-the-code via `FAIL_TO_PASS`.
- The env must fit the framework's RL side (TRL GRPOTrainer + verifiers / OpenEnv-compatible) so generated tasks feed the same training loop.
- Reward-hacking is a *real* reported failure, not theoretical — safeguards must be in the design, not deferred.
- The online difficulty curriculum is part of the recipe, not optional.
- Subsystem should be reusable for the owner's other project (general "verifiable-reward task generation over a code repo").

## Considered Options

- **A. `FeatureDeletionEnv` that inverts OSS SWE substrates (revert gold patch) + online pass-rate difficulty gate + sandbox + signature/patch-provenance reward-hacking safeguards** (chosen)
- **B. Greenfield repo-scraping generator (clone arbitrary GitHub repos, delete AST nodes, hope tests cover them)**
- **C. Skip generation; reuse SWE-bench-lite tasks as-is without a deletion/inversion layer**

## Decision Outcome

Chosen: **Option A** — a `composer_replication.datagen` package with a
`FeatureDeletionTask` dataclass, a `FeatureDeletionEnv` (Gym/OpenEnv-style
`reset`/`step`/`reward` where reward = masked test-pass fraction), substrate
adapters that invert the 5 OSS datasets by reverting their gold patch, a
4-gate solvability validator (broken repo fails `FAIL_TO_PASS`, passes
`PASS_TO_PASS`, gold patch restores green, deletion is reachable from tests),
an online pass-rate difficulty gate, and reward-hacking safeguards
(pre-task scrub of `__pycache__`/`.mypy_cache`/`.class`/`.git`; allowlisted
sandbox without `find`/`strings`/`unzip`/decompilers; signature + patch-provenance
monitor that masks reward when deleted symbols reappear via non-implementation
paths — including string-concat-obfuscated cache reads).
A TRL `reward_fn(prompts, completions, **kwargs) -> list[float]` adapter wires
it to the RL loop.

### Consequences

- **Positive**: Verifiable reward for free (tests already exist + are known to exercise the code via `FAIL_TO_PASS`); no need to generate or trust new tests.
- **Positive**: Reusable general subsystem — "invert a solved-repo dataset into a reimplement-to-pass task" works for the owner's other project.
- **Positive**: Online difficulty gate matches the actual recipe.
- **Negative**: Bounded to what the OSS substrates cover (Python-dominant; SWE-bench is Python/JS-heavy). Other languages need new substrates. Documented as a known coverage limit.
- **Negative**: Running tests in a sandbox requires Docker images per substrate; CPU-pool generation has real wall-clock cost (~15 node-days to invert all 21k SWE-rebench tasks per research/06). Mitigated by reusing the substrates' published Docker images and generating lazily.
- **Negative**: Reward-hacking safeguards are a moving target; the signature + patch-provenance monitor is heuristic and will have false negatives. Mitigated by treating it as defense-in-depth (sandbox lockdown is the primary control) and logging suspected hacks for review.
- **Neutral**: Adds a `[datagen]` optional extra (datasets, docker SDK).

## Pros and Cons of the Options

### A. Invert OSS substrates
- Good: tests guaranteed to exercise the deleted code; verified datasets; reusable; matches recipe.
- Bad: language/domain coverage bounded by the substrates; sandbox/Docker cost.

### B. Greenfield repo scraping
- Good: unbounded repo universe.
- Bad: no guarantee remaining tests exercise the deleted code (the hard part SWE-bench already solved); huge validation burden; reinvents SWE-Gym. High effort, low marginal value.

### C. Reuse SWE-bench-lite as-is
- Good: zero build.
- Bad: not *Feature Deletion* — it's issue-fixing; no controlled difficulty knob; no deletion mechanic; doesn't bring Composer's data-gen method in, just consumes an existing benchmark. Fails the actual ask.

## Post-acceptance cross-family review (2026-05-29)

The 5-family review (see ADR-008's review section) was **3 REJECT / 1
ACCEPT-WITH-FIXES on ADR-010** — the harshest of the three. The reviewers'
core, correct objection: the `[x]` gates were satisfied **against `FakeSandbox`
materializers that directly assign pass/fail booleans**, so they prove the
control-flow plumbing, NOT that feature-deletion-and-reimplement works on a real
repo. ADR-010's central claim ("invert OSS SWE substrates") is exactly the part
that is `[~]` (Docker-deferred) — the accepted gates are the easy half. This is
fair; the gate language below is re-scoped accordingly. Findings verified and
remediated where possible without Docker:

- **[FIXED — P0, 4/4 reviewers] Reward-hack sandbox lockdown was the ADR's
  claimed PRIMARY control but was UNIMPLEMENTED.** `LocalSubprocessSandbox.boot`
  only recorded the image string; no cache/.git scrub existed, so the
  (bypassable) command denylist was the *only* defense. Worse, the denylist
  checks only the first whitespace token, so `python -c "import marshal,dis;
  …read __pycache__/*.pyc"`, `/usr/bin/strings`, `sh -c "strings x"` all bypass
  it, and the `HackMonitor` regex is defeated by string-concat
  (`"__py"+"cache__"`). **Fix:** `boot()` now physically scrubs `__pycache__`,
  `.mypy_cache`, `.pytest_cache`, `.git`, `.hg`, and `*.pyc/*.pyo/*.class` from
  the working tree before the episode — the actual primary control. The denylist
  is now documented as cheap defense-in-depth, not the wall. Tested:
  `test_local_sandbox_scrubs_caches_and_git_on_boot`.
- **[FIXED — P0, 3/4 reviewers] Curriculum recorded partial multi-feature
  reward as a full pass.** `reward_fn` updated the curriculum with
  `int(res.reward > 0)`, so a 0.5 (1-of-2 features) reward logged as a 100% pass;
  `p_hat` crossed `tau_easy` and the task was **retired before the policy learned
  the second feature**. **Fix:** the curriculum now takes FRACTIONAL credit
  (`n_pass: float`), and `reward_fn` feeds the pass-fraction; a guard-broken or
  hack-flagged rollout contributes 0 credit but still counts as an exposure (so
  hack-only tasks trend to quarantine, not phantom passes). Tested:
  `test_partial_multifeature_reward_is_fractional_credit_not_a_full_pass`,
  `test_guard_broken_trajectory_does_not_pollute_curriculum_with_a_pass`.
- **[FIXED] `reward_fn` could silently return the wrong reward count.** The
  `zip(completions, task_id)` truncated on length mismatch, returning fewer
  rewards than TRL expects and corrupting the advantage computation. **Fix:**
  explicit length-guard raises. Tested: `test_reward_fn_rejects_length_mismatch`.
- **[FIXED] Shell injection / parameterized-test breakage in `run_tests`**
  (DeepSeek + Gemini). `f"{test_command} {node_ids}"` with `shell=True` breaks on
  SWE-bench parametrized node ids (`test_x.py::test_y[a b]`) and is injectable.
  **Fix:** `shlex.quote` each node id.
- **[FIXED] Duplicate `"unzip"` in `SANDBOX_DENYLIST`** (multiple reviewers).
- **[OPEN — honest, the central re-scope] Gate 2 ("deletion breaks the feature")
  and the "deletion reachable from tests" claim do NOT verify reachability**
  (GPT-5.5 P0, Grok P0). Gate 2 only checks that `FAIL_TO_PASS` tests fail in the
  broken state — it does not prove the failure was *caused by the reverted
  feature* rather than an unrelated breakage. A real reachability check
  (coverage of the changed region by the failing tests, or revert-provenance)
  needs the live Docker materializers. **This is the same `[~]` gate as the
  substrate-inversion e2e — see below.**
- **[RESOLVED — ADR-012] `HackMonitor` was a substring matcher, not the
  AST-provenance monitor the ADR advertised** (DeepSeek P0). It flagged
  cache/decompiler signatures in the trajectory but did no symbol-reappearance
  analysis, and was bypassable by string-concat. With the scrub now in place as
  the primary control, the monitor is correctly-scoped defense-in-depth. ADR-012
  re-scoped the language to "signature + patch-provenance monitor" (not "AST")
  and added a patch-provenance layer: a deleted symbol reappearing verbatim in
  the agent's patch alongside a cache/bytecode read — normalized to defeat
  string-concat obfuscation (`"__py"+"cache__"`) — is now flagged.
- **[OPEN — recipe fidelity] Curriculum ignores rollout-turns and
  thinking-token count** (DeepSeek, GPT-5.5). The Composer 2 tech report keys the
  curriculum on these; the implementation tracks only pass-rate. Follow-up:
  extend `DifficultyCurriculum.update` to accept and weight on turn/think-token
  signals.

**The three OPEN items all require the Docker substrate e2e to close** (real
reachability, real provenance on a materialized repo). That gate remains the
honest `[~]` — see the unblocked-by note. The fixable correctness defects
(scrub, fractional curriculum, length-guard, shlex) are fixed and tested.

## Acceptance gate (must be green before status flips to accepted)

Core gates green as of 2026-05-29 (19 tests in
`composer_replication/datagen/tests/test_feature_deletion.py`, all CPU via
`FakeSandbox`). The single Docker-dependent gate (real substrate inversion) is
implemented but its live run is the documented unblocked-by step — see note.

- [x] `FeatureDeletionTask` dataclass + `FeatureDeletionEnv` (`reset`/`step`/`reward`) implemented; reward = masked test-pass fraction — `test_reward_is_pass_fraction_when_guard_ok`, `test_reward_graded_for_multi_feature` (0.5 for 1-of-2), `test_reward_zeroed_when_functional_guard_broken`. `golden_diff` held out of `repr` (`test_golden_diff_not_in_repr`).
- [~] SWE-bench-Lite substrate adapter: **schema inversion implemented + tested** (`SweBenchAdapter.to_task` — `test_swebench_adapter_inverts_instance`, JSON-or-list FAIL_TO_PASS handling, copyleft filter). The **live revert-gold-patch → broken-repo → test-run** path requires a substrate Docker image; `LocalSubprocessSandbox` + `validate_task` are wired for it, and the gate is exercised in unit form via `FakeSandbox` materializers (`test_validator_accepts_well_formed_task`). **UPDATE 2026-05-29 (B6):** the `skipif(docker)` end-to-end test now EXISTS — `composer_replication/datagen/tests/test_docker_substrate_e2e.py` runs the 4 gates against a REAL `python:3.11-slim` container with a real subprocess pytest runner on a minimal synthetic feature-deletion task (proving the inversion *mechanics* on real hardware), plus a cache-scrub-in-container test. It SKIPS cleanly in this no-Docker CPU env and ACTIVELY RUNS the moment a Docker host executes the suite. Still `[~]` because (a) no Docker in this env to demonstrate a green run, and (b) the *full SWE-bench-Lite image* variant (pull one published instance image, revert its real gold patch) is a follow-up reusing the same driver. The mechanics gate is now ready-to-close, not just wired.
- [x] 4-gate solvability validator implemented; `test_validator_rejects_unreachable_deletion` (deletion that doesn't break the target → gate 2 fails) and `test_validator_rejects_when_guard_breaks` (gate 3 fails).
- [x] Reward-hacking safeguard: `SANDBOX_DENYLIST` blocks `find`/`strings`/`unzip`/decompilers/`git` (`test_sandbox_denies_decompiler_and_cache_tools`); `HackMonitor` flags cache/bytecode-provenance hacks (`test_monitor_flags_cache_provenance_hack`) and passes clean reimplementation (`test_monitor_passes_clean_reimplementation`); reward is masked to 0 when a hack is detected even if tests "pass" (`test_reward_masked_when_hack_detected`).
- [x] Online difficulty gate: `DifficultyCurriculum` up-weights the frontier (~0.5 pass-rate) over aced tasks and retires aced ones (`test_curriculum_upweights_frontier_over_solved`); quarantines all-fail tasks after `min_exposures` (`test_curriculum_quarantines_impossible_task`). NOTE: quarantine uses the *raw* observed rate, not the Laplace-smoothed `p_hat` (smoothing is for weighting, not the have-we-ever-passed decision).
- [x] TRL `reward_fn(prompts, completions, *, task_id, **kwargs) -> list[float]` adapter returns one float in [0,1] per completion = masked pass-fraction (`test_reward_fn_returns_one_float_per_completion`); requires the `task_id` column (`test_reward_fn_requires_task_id`).
- [x] `[datagen]` optional extra added to `pyproject.toml` (`datasets` + `docker`); pure-Python core needs only `datasets`.

## More Information

- `research/06-feature-deletion-datagen.md` — full design: substrates w/ HF ids + licenses, deletion mechanics, safeguards, env + reward_fn sketches, cost verdict.
- Substrates: SWE-bench (arXiv:2310.06770), SWE-Gym (arXiv:2412.21139), R2E-Gym, SWE-rebench.
- Composer data-gen: `docs/COMPOSER_RECIPE_MAPPING.md` §2, `research/10-composer2-techreport-mining.md` §1.2 (curriculum).