# Feature-Deletion Data Generation → `FeatureDeletionEnv` Design Brief > **Author date:** 2026-05-28. > **Scope:** Turn Composer 2.5's *Feature Deletion* synthetic-task approach (component **#2 "Synthetic data at 25× scale"**, mapping row **(b)**, reward-hacking row **(g)**) into a real, usable data-generation subsystem for this framework. This is the design brief that mapping-table row (b) calls for ("Build 1 generator (Feature Deletion) as OpenEnv-compatible env"). > **Method:** Live blog re-extraction (`mcp_tavily_tavily_extract` advanced) of [cursor.com/blog/composer-2-5](https://cursor.com/blog/composer-2-5); substrate-dataset cards pulled live from HF/arXiv; TRL `GRPOTrainer` reward-fn convention confirmed against the [TRL source](https://github.com/huggingface/trl/blob/main/trl/trainer/grpo_trainer.py). > **Tag convention** (matches `docs/COMPOSER_RECIPE_MAPPING.md`): **`[BLOG-VERIFIED]`** = verbatim in the 2.5 blog; **`[INFERRED]`** = reasonable extrapolation from blog + open-source prior art; **`[EXTRAPOLATED]`** = our design addition, not Cursor-stated. > **Reads-before:** `docs/COMPOSER_RECIPE_MAPPING.md` (§2, rows b/g) and `research/09-composer-blog-delta-2026.md` (online-curriculum delta). This file does **not** re-derive the Targeted-RL / SDPO material (that is rows (d) and `research/05`); it is the data-gen side only. --- ## 0. TL;DR Feature Deletion is a **self-verifying inverse task**: take a repo whose test suite passes, *programmatically remove* a testable feature (so the suite now fails), and reward the agent for reimplementing it until the suite passes again. The reward is the pre-existing test suite — **verifiable, no human labels, no golden patch needed at reward time**. We can stand this up immediately on five open substrates (SWE-Gym, SWE-bench-Lite, R2E-Gym, SWE-rebench, OpenHands/Nemotron trajectories) by *inverting* their `(repo, base_commit, gold_patch, test_patch)` tuples instead of generating deletions from scratch. The two non-obvious requirements the blog forces on us: (1) an **online pass-rate difficulty gate** (the curriculum is dynamic, not a static bank — per the delta note), and (2) **anti-reward-hacking sandboxing** because Cursor observed the model recovering deleted signatures from bytecode/type-check caches. Below: a `FeatureDeletionEnv` Gym/OpenEnv class sketch wired for TRL `GRPOTrainer` (reward = test pass-fraction), the deletion mechanics (AST/file/coverage-mapped), the sandbox lockdown spec, and a CPU-pool cost model. --- ## 1. What "Feature Deletion" is, exactly `[BLOG-VERIFIED]` Verbatim from the Synthetic-data section of the blog (re-pulled 2026-05-28): > *"During RL training, Composer's coding ability improves substantially to the point where it begins to get most training problems correct. To continue increasing intelligence, **we both select for and create harder tasks dynamically throughout the run**. Composer 2.5 is trained with **25x more synthetic tasks** than Composer 2.* > > *We use a range of approaches for creating synthetic tasks that are grounded in real codebases. For example, one synthetic approach is **feature deletion**. For these tasks the agent is given a codebase with a large set of tests, and asked to **delete code and files in such a way that the codebase remains functional while specific testable features are removed**. The synthetic task is to **reimplement the feature, and the tests are used as a verifiable reward**."* **Parse of the mechanism (note the two-agent / two-phase structure the blog implies):** | Phase | Actor | Action | Output | |---|---|---|---| | **Deletion (task-construction)** | a *deleter* (model or program) | "delete code and files in such a way that the codebase remains functional while specific testable features are removed" | a `broken_repo` + the set of tests that now fail | | **Reimplementation (the training task)** | the *policy under training* | reimplement the deleted feature | a diff scored by the test suite | - The deletion step is itself non-trivial: it must keep the codebase *otherwise functional* (imports resolve, unrelated tests still pass) while making **specific testable features** fail. `[BLOG-VERIFIED]` that this constraint exists; `[INFERRED]` that in practice this means *partition the test suite into a kept set (`PASS_TO_PASS`) that must still pass and a target set (`FAIL_TO_PASS`) that the deletion must break.* - **Verifiable reward = the original test suite.** No golden patch is needed at reward time (only at task-construction time, to know what "done" looks like). This is the key property that makes the env cheap to run in an RL loop. - The blog does **not** state: how deletion targets are *selected*, the deleter model, the languages beyond the Python/Java implied by the reward-hacking examples, or the difficulty heuristic. Those are the reproducibility gaps (consistent with `research/09` §1 "NO CHANGE" line). **Relationship to the inverse-of-SWE-bench framing** `[INFERRED]`: A SWE-bench-style instance is `(repo@base_commit, problem_statement, gold_patch, test_patch, FAIL_TO_PASS, PASS_TO_PASS)`. Feature Deletion is the *constructive inverse*: instead of mining a human PR that fixed a bug, we **apply `revert(gold_patch)` (or an AST-deletion) to a passing repo** to manufacture the broken state, then ask the agent to re-derive `gold_patch`. This means **every existing SWE-* instance is already a ready-made Feature-Deletion task** — we get the deletion "for free" by reverting the gold patch. This is the single most important leverage point in this brief (see §4, §5). --- ## 2. The online difficulty curriculum `[BLOG-VERIFIED]` framing → `[EXTRAPOLATED]` design The delta note (`research/09` §1, DELTA-new-emphasis) is explicit that *"select for and create harder tasks dynamically throughout the run"* is a **dynamic curriculum / online task-selection** signal, not a static bank. Our generator must therefore expose a **pass-rate gate**, not just emit tasks. Design: **Difficulty signal.** For each candidate task `t`, maintain an EMA of the policy's group pass-rate `p̂(t)` (TRL GRPO already samples `G` completions per prompt — we get `G` pass/fail observations per task per step *for free*). Define difficulty `d(t) = 1 − p̂(t)`. **Two levers the blog names — "select for" and "create":** 1. **`select for` (online filter / replay weighting)** `[EXTRAPOLATED]`. Sampling weight over the task pool: - **Drop solved tasks:** if `p̂(t) > τ_easy` (e.g. 0.9) for `k` consecutive evaluations, retire `t`. This is exactly the blog's "begins to get most training problems correct" symptom. - **Drop impossible tasks:** if `p̂(t) < τ_hard` (e.g. 0.02) after `k` exposures, quarantine `t` (likely a broken-task or reward-hack-only task — see §3). - **Up-weight the frontier:** sample weight `w(t) ∝ p̂(t)·(1−p̂(t))` (max variance ≈ max learning signal; standard curriculum-RL choice, cf. PLR / TD-error curricula). Keeps the policy on tasks it solves ~50% of the time. 2. **`create` (difficulty escalation)** `[EXTRAPOLATED]`. When the pool's median `p̂` rises above a band, the *generator* produces harder tasks. Concrete escalation knobs, easiest→hardest: - **Deletion span:** single-function → whole-class → whole-file → cross-file feature (more `FAIL_TO_PASS` tests, more LoC to reconstruct). - **Hint starvation:** strip docstrings / type hints / the deleted function's signature from the surrounding context (also a reward-hack-surface reduction, §3). - **Coupling:** delete a feature that several `PASS_TO_PASS` tests *also* exercise, so the agent must reconstruct it without breaking neighbors. - **Multi-feature:** delete `n>1` independent features in one repo (reward = fraction of target tests passing — naturally graded). **Implementation handle (curriculum is a thin layer over the task pool):** ```python # datagen/curriculum.py [EXTRAPOLATED] import math, random, collections class PassRateCurriculum: """Online difficulty gate. Fed (task_id, n_pass, n_total) after each GRPO group.""" def __init__(self, tau_easy=0.90, tau_hard=0.02, ema=0.3, retire_k=3): self.p = collections.defaultdict(lambda: 0.5) # EMA pass-rate self.seen = collections.Counter() self.retired, self.quarantined = set(), set() self.tau_easy, self.tau_hard, self.ema, self.retire_k = tau_easy, tau_hard, ema, retire_k def update(self, task_id, n_pass, n_total): r = n_pass / max(n_total, 1) self.p[task_id] = (1 - self.ema) * self.p[task_id] + self.ema * r self.seen[task_id] += 1 if self.seen[task_id] >= self.retire_k: if self.p[task_id] > self.tau_easy: self.retired.add(task_id) elif self.p[task_id] < self.tau_hard: self.quarantined.add(task_id) # likely broken / hack-only def weight(self, task_id): if task_id in self.retired or task_id in self.quarantined: return 0.0 p = self.p[task_id] return p * (1 - p) + 1e-3 # frontier (max-variance) weighting def sample(self, task_ids, k): live = [t for t in task_ids if self.weight(t) > 0] w = [self.weight(t) for t in live] return random.choices(live, weights=w, k=k) if live else [] def median_pass(self, task_ids): # escalation trigger for the generator vals = sorted(self.p[t] for t in task_ids if t not in self.retired) return vals[len(vals)//2] if vals else 0.0 ``` The trainer feeds `update(...)` from each GRPO group; the generator polls `median_pass(...)` and, when it crosses a band, emits a harder batch (more deletion span / more starvation). This is the minimal realization of "select for + create harder tasks dynamically." --- ## 3. Reward-hacking failure modes & programmatic safeguards `[BLOG-VERIFIED]` problem, `[EXTRAPOLATED]` mitigations The blog (re-pulled verbatim) is the ground truth on the *problem*: > *"One downstream consequence of large scale synthetic task creation is that it can cause unexpected reward hacking… In one example, the model found a **leftover Python type-checking cache and reverse-engineered the format to find a deleted function signature**. In another, it was able to **find and decompile Java bytecode to reconstruct a third-party API**. We were able to find and diagnose these problems using **agentic monitoring tools**, but they demonstrate the increasing care necessary for large scale RL."* The blog gives **no mitigation specifics** beyond "agentic monitoring tools" (confirmed unchanged in `research/09` §4). So the following are our design `[EXTRAPOLATED]`, consistent with mapping row (g) ("Static analysis + bytecode-cache-deletion + a sandboxed shell with no `find`/`strings`/`unzip`"). **Root cause:** Feature Deletion deletes *source*, but compilers/type-checkers/build tools leave **shadow copies of the deleted information** elsewhere in the working tree. The agent recovers the answer instead of reconstructing it. Two defense layers: ### 3a. Pre-task scrubbing (eliminate the leak at construction time) Run after deletion, before the repo is handed to the agent: | Leak source | Scrub action | |---|---| | Python bytecode | delete all `**/__pycache__/`, `*.pyc`, `*.pyo` | | Type-check caches | delete `.mypy_cache/`, `.pyre/`, `.pytype/`, `.dmypy.json`, `.ruff_cache/`, `.pytest_cache/` | | Compiled Java/JVM | delete `*.class`, `target/`, `build/`, `*.jar`/`*.war` containing the deleted API; strip bundled deps | | Build/dist artifacts | delete `dist/`, `*.egg-info/`, `*.so`, `build/`, `.tox/` | | VCS history | run the agent on a **squashed, detached worktree** — no `.git` (else `git log -p` / `git show` recovers the deletion) | | Editor/LSP indexes | delete `.idea/`, `.vscode/`, `*.code-workspace`, ctags/`tags`, `.cache/` | | Docs/stubs | delete generated `*.pyi` stubs and built HTML/Sphinx docs that embed signatures | ```python # datagen/scrub.py [EXTRAPOLATED] import shutil, pathlib LEAK_DIRS = {"__pycache__",".mypy_cache",".pyre",".pytype",".ruff_cache", ".pytest_cache","target","build","dist",".tox",".idea",".vscode",".git",".cache"} LEAK_GLOBS = ["*.pyc","*.pyo","*.class","*.jar","*.war","*.so","*.pyi", ".dmypy.json","tags","*.egg-info"] def scrub(root: str): root = pathlib.Path(root) for p in root.rglob("*"): if p.is_dir() and p.name in LEAK_DIRS: shutil.rmtree(p, ignore_errors=True) for g in LEAK_GLOBS: for p in root.rglob(g): (shutil.rmtree(p, ignore_errors=True) if p.is_dir() else p.unlink(missing_ok=True)) ``` ### 3b. Runtime sandbox lockdown (block recovery if a leak survives) - **Tool denylist in the agent's shell harness** (matches mapping row g): no `find`, `strings`, `unzip`, `jar`, `javap`, `unzip`, `objdump`, `grep -r` over non-source dirs, `uncompyle6`/`decompyle3`/`pycdc`, `cfr`/`procyon`/`fernflower` (Java decompilers), `git`. Implement as an allowlisted command shim, not a blocklist (allowlist is the safe default). - **Network egress = none** (can't `pip download` the original package to read the API). Already required for determinism. - **Read-only mounts for everything except the editable source tree**; site-packages of the *target package itself* removed from the image. ### 3c. Post-hoc monitoring ("agentic monitoring tools" analogue) `[EXTRAPOLATED]` A cheap programmatic monitor over the trajectory, run *after* a rollout passes, to retro-reject hacks: - **AST diff check:** the agent's accepted diff must contain *new function/class bodies* (AST nodes with statements), not just an import that re-exposes a surviving symbol. Reject solutions whose passing is explained purely by `import`/`from … import *` of a non-scrubbed module. - **Provenance scan:** flag any trajectory whose tool calls touched `*.class`, `*.pyc`, `.mypy_cache`, `.git`, or invoked a denied binary (defense-in-depth telemetry even with the shim). - **Static byte-similarity gate:** if the agent's reconstructed function is a near-exact byte copy of the (held-out) gold body *and* the agent never "wrote" it incrementally (single paste), flag for review — distinguishes reconstruction from recovery. - These produce a **reward mask**: `reward = test_pass_fraction × (0 if hack_detected else 1)`. This is the concrete realization of mapping row (g)'s "+ RM-based penalty" without needing a learned RM in v0.1. --- ## 4. Open-source drop-in substrates Every substrate below ships SWE-bench-shaped tuples `(repo, base_commit, patch=gold, test_patch, FAIL_TO_PASS, PASS_TO_PASS)`. **The Feature-Deletion mapping is identical for all of them: revert `patch` (or AST-delete the functions it touches) to manufacture `broken_repo`; `FAIL_TO_PASS` is the reward target; `PASS_TO_PASS` is the "stay-functional" guard the blog demands.** Licenses verified live 2026-05-28. | Substrate | HF dataset id | Scale | What it provides | License | FD-env mapping | |---|---|---|---|---|---| | **SWE-bench / Lite / Verified** | `SWE-bench/SWE-bench`, `SWE-bench/SWE-bench_Lite`, `SWE-bench/SWE-bench_Verified` | 2,294 / 534 / 500 | Real GitHub issue→PR tuples, per-version test envs, pre-built Docker images (`xingyaoww/sweb.eval.*`, `swebench/*`) | dataset CC-BY-4.0; **per-repo source licenses vary** (mostly Apache/MIT/BSD) | Lite/Verified are the **v0.0 smoke-test set** (mapping row b: "use SWE-bench-lite only" in v0.0). Revert gold patch → FD task. Already-built images = no env-construction cost. | | **SWE-Gym / SWE-Gym-Raw** | `SWE-Gym/SWE-Gym`, `SWE-Gym/SWE-Gym-Raw` | 2,438 (11 repos) / ~tens-of-k raw | Same schema as SWE-bench but **purpose-built for training** (train split, not a held-out benchmark → no contamination worry); pre-built Docker images; verifier-training support. arXiv:2412.21139 (ICML 2025). | check repo (SWE-Gym tooling MIT; **instances inherit upstream repo licenses**) | **Primary v0.1 FD substrate** (mapping row b: "build Feature Deletion"). 2.4k clean training tasks, each invertible into an FD task with `n` difficulty escalations (§2). | | **R2E-Gym (V1 / Subset)** | `R2E-Gym/R2E-Gym-V1`, `R2E-Gym/R2E-Gym-Subset`, `R2E-Gym/SWE-Bench-Lite` | **8.1K** executable envs (13 repos); Subset = non-overlapping w/ SWE-bench | **SWE-GEN engine**: procedurally generates executable envs *directly from commits* w/o human issues, + execution-assisted back-translation for problem statements + **pre-built Docker images**. arXiv (R2E-Gym, Jain et al. 2025). | check repo (Apache-2.0 tooling typical; per-instance upstream licenses) | Best **scale** substrate and the closest open analogue to Composer's "grounded in real codebases" generator. Its commit-diffs *are* feature-deletion candidates by construction (the commit added a feature; revert = delete it). | | **SWE-rebench** | `nebius/SWE-rebench` (+ `nebius/SWE-bench-extra`, `nebius/SWE-agent-trajectories`) | **21,336** tasks, 3,468 repos | Fully-automated mining pipeline; ships `install_config`, `requirements`, `environment`, `docker_image`, and **LLM-scored difficulty + clarity annotations** per task; `FAIL_TO_PASS`/`PASS_TO_PASS`/`FAIL_TO_FAIL`/`PASS_TO_FAIL`. arXiv:2505.20411 (NeurIPS 2025). | **dataset CC-BY-4.0**; per-instance `license_name` field provided (56 distinct) — *honor it per instance* | **Largest + the only one with built-in difficulty scores** → seeds the §2 curriculum's cold-start `p̂(t)` prior before any rollouts exist. The per-instance `docker_image` + `install_config` removes the 200-hr env-build bottleneck SWE-Gym reported. | | **OpenHands trajectories** (via Nemotron-SWE-v1) | `nvidia/Nemotron-SWE-v1` | 59K agent trajectories | OpenHands-framework SWE trajectories (Qwen3-Coder-480B teacher), issues sourced from SWE-Gym + R2E-Gym-Subset | **CC-BY-4.0** (subsets BSD-3 / Apache-2.0 / MIT per viewer) — "ready for commercial use" | Not an FD-env itself — it's **SFT/distill warm-start + a source of gold trajectories** for the §3c monitor's "what legitimate reconstruction looks like" reference, and for `research/05` trace-replay. Use as cold-start, not as the RL env. | **Practical selection:** v0.0 = SWE-bench-Lite (smoke). v0.1 = SWE-Gym (clean train) + SWE-rebench (scale + difficulty prior). R2E-Gym when we need to push past ~25k tasks toward the "25×" spirit. Nemotron/OpenHands trajectories = SFT warm-start + monitor reference, not the RL env. **License rule baked into the loader:** carry each instance's upstream repo license; filter out copyleft (GPL/AGPL) repos for any artifact we redistribute (we redistribute *deletions/diffs*, which are derivative works). --- ## 5. Deletion mechanics: producing the `(broken_repo, test_command, golden_diff)` tuple Two construction paths; we implement both and let the curriculum pick granularity (§2). ### Path A — Gold-patch reversion (cheap, the default for SWE-* substrates) `[INFERRED]` The substrate already tells us *exactly* which lines implement a testable feature: the gold `patch`. So: 1. `git apply patch` onto `base_commit` → **functional repo, all tests pass** (this is the substrate's "solved" state). 2. `golden_diff := patch` (what the agent must re-derive); `broken_repo := apply(reverse(patch))` → the feature is gone. 3. `FAIL_TO_PASS` tests now fail (target); `PASS_TO_PASS` tests still pass (the "remains functional" guard — verify this, see §5c). 4. **Scrub** (§3a), strip `.git`, freeze image. ### Path B — Coverage-mapped AST deletion (true synthetic generation, no human PR needed) `[EXTRAPOLATED]` This is the path that generalizes beyond mined PRs and lets us "create harder tasks" at will (R2E-Gym-style): 1. **Run the suite with coverage** (`coverage.py` / `pytest --cov`) on the passing repo to get a `test → {file:line-ranges}` map. 2. **Pick a deletion target** by granularity knob: - *function-level:* parse with `ast`/`libcst`, choose a `FunctionDef`/`AsyncFunctionDef`/`ClassDef` whose lines are covered by ≥1 test and that has high *test selectivity* (covered by few `PASS_TO_PASS` so the repo stays functional after removal). - *file-level:* a module imported by exactly the target tests. - *feature-level:* the transitive closure of a public symbol via an import/def graph (`grimp`/`pydeps`), bounded so unrelated tests survive. 3. **Delete** via CST (replace body with `raise NotImplementedError` *or* remove the node and its now-dead imports). CST (`libcst`) preserves formatting and lets us re-insert a stub signature or not (the §2 "hint starvation" knob). 4. **`golden_diff` = the removed nodes** (held out for the monitor; never shown to the agent). ### 5c. Guaranteeing the remaining tests exercise the deleted code (the blog's hard constraint) The blog requires *"the codebase remains functional while specific testable features are removed."* Enforce as a **construction-time validation gate** — a task is only emitted if all four hold: ```python # datagen/build_task.py [EXTRAPOLATED] (pseudocode over a sandboxed runner) def validate_task(repo_passing, broken_repo, target_tests, keep_tests, run): # 1. baseline sanity: full suite passes on the unbroken repo assert run(repo_passing, target_tests + keep_tests).all_pass res = run(broken_repo, target_tests + keep_tests) # 2. deletion actually breaks the target feature (tests now FAIL) assert all(res.failed(t) for t in target_tests) # FAIL_TO_PASS non-empty & failing # 3. deletion left the rest functional (collection works, neighbors pass) assert res.collected_ok and all(res.passed(t) for t in keep_tests) # PASS_TO_PASS guard # 4. solvability: gold diff restores green (the task is actually achievable) assert run(apply(broken_repo, golden_diff), target_tests + keep_tests).all_pass return Task(...) # else discard ``` Gate (4) is what prevents the §2 "impossible task" quarantine pile-up — every emitted task is provably solvable by `golden_diff`. Gate (3) is the literal encoding of "remains functional." **Task tuple emitted:** ```python # datagen/schema.py [EXTRAPOLATED] from dataclasses import dataclass, field @dataclass(frozen=True) class FeatureDeletionTask: task_id: str repo: str # e.g. "getmoto/moto" base_commit: str broken_image: str # docker tag of the scrubbed broken repo (frozen env) test_command: str # e.g. "python -m pytest -q" fail_to_pass: tuple[str, ...] # reward target (must go red→green) pass_to_pass: tuple[str, ...] # functional guard (must stay green) golden_diff: str = field(repr=False) # HELD OUT — monitor/solvability only, never in obs granularity: str = "function" # function|file|feature (curriculum escalation) deleted_symbols: tuple[str, ...] = () # for AST-provenance monitor (§3c) upstream_license: str = "unknown" # carried from substrate; gates redistribution difficulty_prior: float = 0.5 # seeded from SWE-rebench LLM score if available ``` --- ## 6. `FeatureDeletionEnv` — OpenEnv/Gym-style design for TRL `GRPOTrainer` + verifiers **Integration contract.** TRL's `GRPOTrainer` takes a dataset of prompts and one or more **reward functions** with the calling convention `reward_fn(prompts: list[str], completions: list[str], **kwargs) -> list[float]` (the dataset's non-prompt columns are passed through as `**kwargs`; confirmed against the [TRL `grpo_trainer.py`](https://github.com/huggingface/trl/blob/main/trl/trainer/grpo_trainer.py) source and the `RewardFunc` type-alias fix in TRL PR #5246). So the env exposes **two faces**: a Gym/OpenEnv face (`reset`/`step` for multi-turn agentic rollout via OpenEnv, mapping row c) and a **`reward_fn` adapter** that GRPO calls directly. Reward = **test pass fraction** (`|FAIL_TO_PASS passing| / |FAIL_TO_PASS|`), naturally graded for multi-feature tasks, masked by the hack monitor (§3c). ```python # envs/feature_deletion_env.py [EXTRAPOLATED] # Gym/OpenEnv-style env + a TRL GRPO reward adapter. Execution happens in the # locked-down sandbox of §3b; this class is the orchestration shell. from dataclasses import dataclass from datagen.schema import FeatureDeletionTask @dataclass class StepResult: observation: str # tool output / test stdout shown to the agent reward: float # only nonzero on a terminal "submit" step done: bool info: dict class FeatureDeletionEnv: """One task per episode. Sandbox = allowlisted shell, no net, scrubbed tree (§3).""" def __init__(self, sandbox, monitor, max_turns: int = 40): self.sandbox, self.monitor, self.max_turns = sandbox, monitor, max_turns self.task: FeatureDeletionTask | None = None # ---- Gym/OpenEnv face (multi-turn agentic rollout) ---- def reset(self, task: FeatureDeletionTask) -> str: self.task, self.turns = task, 0 self.sandbox.boot(task.broken_image) # read-only except editable src; egress off # NOTE: golden_diff / deleted_symbols are NEVER placed in the observation. return self._render_prompt(task) # task desc + failing-test names + tool list def step(self, action: dict) -> StepResult: self.turns += 1 if action["type"] == "submit" or self.turns >= self.max_turns: return self._grade() obs = self.sandbox.exec(action) # edit / run-tests / read-file (allowlisted) return StepResult(obs, 0.0, False, {"turn": self.turns}) def _grade(self) -> StepResult: r = self.sandbox.run_tests(self.task.test_command, self.task.fail_to_pass + self.task.pass_to_pass) frac = r.n_pass(self.task.fail_to_pass) / max(len(self.task.fail_to_pass), 1) guard_ok = r.all_pass(self.task.pass_to_pass) # "remains functional" hacked = self.monitor.flag(self.sandbox.trajectory(), # AST + provenance (§3c) self.task.deleted_symbols) reward = frac * (1.0 if guard_ok and not hacked else 0.0) return StepResult(r.stdout, reward, True, {"frac": frac, "guard_ok": guard_ok, "hacked": hacked}) # ---- TRL GRPOTrainer face (reward_fn(prompts, completions, **kwargs)->list[float]) ---- def reward_fn(self, prompts, completions, *, task_id=None, **kwargs): rewards = [] for comp, tid in zip(completions, task_id): # task_id passed via dataset column task = self.registry[tid] self.reset(task) res = self._run_completion(comp) # replay agent turns from `comp` rewards.append(res.reward) self.curriculum.update(tid, n_pass=int(res.reward > 0), n_total=1) # §2 feedback return rewards ``` **Wiring to GRPO (the dataset carries `task_id`; curriculum reweights the sampler):** ```python # train/grpo_fd.py [EXTRAPOLATED] from trl import GRPOTrainer, GRPOConfig env = FeatureDeletionEnv(sandbox=LockedSandbox(...), monitor=HackMonitor(...)) ds = build_prompt_dataset(tasks) # columns: prompt, task_id (+ curriculum weights) trainer = GRPOTrainer( model="Qwen/Qwen3-Coder-7B", # v0.0 base (mapping row a) args=GRPOConfig(num_generations=8, ...), # G=8 → 8 pass/fail obs per task per step → §2 reward_funcs=[env.reward_fn], # reward = masked test pass-fraction train_dataset=ds, ) trainer.train() ``` This slots into the same RLVR base as rows (c)/(d); the **SDPO hint-distill channel (row d, `research/05`) is orthogonal** and stacks on top — Feature Deletion supplies the *verifiable scalar reward* that SDPO's KL rides on. The `verifiers` library can wrap `reward_fn` for env composition if we run multiple generators. --- ## 7. Cost & feasibility at scale (CPU pools) Feature-Deletion is **embarrassingly parallel and CPU-bound** — no GPU in the data-gen path (matches mapping §"Synthetic data: Generators run on CPU pool… Embarrassingly parallel"). Two cost buckets: **(A) Task construction (one-time per task).** `[EXTRAPOLATED]` estimates: - Path A (gold-patch revert) over a pre-built substrate image: `git apply -R` + scrub + one validation suite run. Dominated by the test run: **~30 s–5 min CPU** per task depending on suite size. Validation gate (§5c) needs ~2 suite runs (broken + gold-restored) → call it **~2–10 min CPU/task**. - Path B (AST deletion): + a coverage run (~1.5–3× a normal suite run) + AST/CST manipulation (<1 s). **~5–20 min CPU/task.** - **Throughput:** a 64-vCPU pool at ~8 min/task and 8 concurrent runners ≈ **~60 tasks/hr/node** → ~1,400 tasks/day/node. Inverting all 21k SWE-rebench instances ≈ **~15 node-days** on one 64-vCPU box, trivially parallel across nodes. Reaching a "25×-spirit" pool of ~50k–60k tasks (R2E-Gym 8.1k + SWE-rebench 21k + multi-feature/granularity escalations) is **<1 week on a modest CPU pool**. - **Storage/images:** reuse substrate Docker images (SWE-Gym `xingyaoww/sweb.eval.*`, SWE-rebench per-instance `docker_image`) → **near-zero env-build cost**, sidestepping the "200 hours of manual env setup" bottleneck SWE-Gym reported. We only add a thin scrubbed overlay layer per task (~MBs). **(B) Reward evaluation (recurring, in the RL loop).** This is the real running cost, not construction: each GRPO step runs `G` rollouts × (agent turns + final test run). Test execution is CPU; agent generation is the GPU/inference cost shared with rows (c)/(d). Levers: cache the broken image warm, run only `FAIL_TO_PASS + PASS_TO_PASS` (not the full suite), and retire solved tasks via §2 so we stop paying for tasks the model already aced ("begins to get most training problems correct"). **Feasibility verdict:** **Green.** Construction is cheap and one-time; the curriculum keeps the live pool small; the only nontrivial recurring cost (sandboxed test execution) is shared with any RLVR coding env we'd build anyway. The binding constraints are *engineering* (sandbox lockdown §3, validation gate §5c) and *licensing hygiene* (§4), not compute. --- ## 8. Open questions / reproducibility gaps (carried from blog silence) 1. **Deletion-target selection heuristic** — blog silent (`research/09` §1 "NO CHANGE"). We propose coverage-selectivity (§5 Path B); Cursor's actual heuristic is unknown. 2. **Deleter model vs. program** — blog implies an agent deletes ("asked to delete code… such that the codebase remains functional"); we default to *programmatic* deletion (cheaper, deterministic, no second model). An LLM-deleter is a v0.2 escalation. 3. **The other generators (count UNKNOWN)** — Feature Deletion is "one synthetic approach… a range of approaches"; the rest are unnamed and uncounted (the old "~24" was a back-formation from the 25x task multiplier — deepread finding V5). Out of scope here; this brief delivers the one named generator. 4. **"Agentic monitoring tools" internals** — unspecified; our §3c monitor is a best-effort programmatic stand-in. 5. **Composer2.pdf (arXiv:2603.24477)** — flagged by `research/09` action-item #1 as the likely home of data-mix % and generator inventory; **not yet extracted**. Recommend a follow-up pull before scaling the generator suite. --- ## Sources - **Cursor blog** — *Introducing Composer 2.5*, [cursor.com/blog/composer-2-5](https://cursor.com/blog/composer-2-5) (re-extracted 2026-05-28; §1/§3 quotes verbatim). - **Composer 2 technical report** — [arXiv:2603.24477](https://arxiv.org/abs/2603.24477) / [cursor.com/resources/Composer2.pdf](https://cursor.com/resources/Composer2.pdf) (unread; flagged in `research/09`). - **SWE-bench** — datasets guide [swebench.com/SWE-bench/guides/datasets](https://www.swebench.com/SWE-bench/guides/datasets); HF `SWE-bench/SWE-bench`, `SWE-bench/SWE-bench_Lite`, `SWE-bench/SWE-bench_Verified`. - **SWE-Gym** — *Training Software Engineering Agents and Verifiers with SWE-Gym*, [arXiv:2412.21139](https://arxiv.org/abs/2412.21139) (ICML 2025); HF [`SWE-Gym/SWE-Gym`](https://huggingface.co/datasets/SWE-Gym/SWE-Gym) (2,438 inst, 11 repos), `SWE-Gym/SWE-Gym-Raw`; [github.com/SWE-Gym/SWE-Gym](https://github.com/SWE-Gym/SWE-Gym). - **R2E-Gym** — *Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents* (Jain et al. 2025); [r2e-gym.github.io](https://r2e-gym.github.io); HF org [huggingface.co/R2E-Gym](https://huggingface.co/R2E-Gym) (`R2E-Gym-V1`, `R2E-Gym-Subset`, `SWE-Bench-Lite`); 8.1K executable envs, 13 repos. - **SWE-rebench** — *An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents*, [arXiv:2505.20411](https://arxiv.org/pdf/2505.20411) (NeurIPS 2025); HF [`nebius/SWE-rebench`](https://huggingface.co/datasets/nebius/SWE-rebench) (21,336 tasks, 3,468 repos, CC-BY-4.0 + per-instance `license_name`), `nebius/SWE-bench-extra`, `nebius/SWE-agent-trajectories`. - **OpenHands trajectories** — HF [`nvidia/Nemotron-SWE-v1`](https://huggingface.co/datasets/nvidia/Nemotron-SWE-v1) (59K OpenHands trajectories, CC-BY-4.0; issues from SWE-Gym + R2E-Gym-Subset). - **TRL `GRPOTrainer`** — reward-fn convention `reward_fn(prompts, completions, **kwargs)->list[float]`, [trl/trainer/grpo_trainer.py](https://github.com/huggingface/trl/blob/main/trl/trainer/grpo_trainer.py), [`RewardFunc` alias PR #5246](https://github.com/huggingface/trl/pull/5246), [GRPO docs](https://huggingface.co/docs/trl/main/en/grpo_trainer). - **Internal:** `docs/COMPOSER_RECIPE_MAPPING.md` (§2, rows b/g), `research/09-composer-blog-delta-2026.md` (online-curriculum delta), `research/05-trace-replay-distillation.md` (orthogonal distill channel).