Hugging Face's logo

Codeseys
/
composer-replication-framework

Reinforcement Learning
Transformers
English
post-training
distillation
agentic-coding
composer-2.5
cursor
kimi-k2
grpo
dapo
diloco
openenv
trl
verl
research
methodology
Model card Files
xet
 Community
composer-replication-framework / research /06-feature-deletion-datagen.md
Baladithya Balamurugan
Wave 21: Stage-0 dataset pipeline — swesmith engine, rollout harness, gates, contract
9a2ce20
preview code
|
Raw
History Blame Contribute Delete
33.8 kB

Feature-Deletion Data Generation → FeatureDeletionEnv Design Brief

Author date: 2026-05-28. Scope: Turn Composer 2.5's Feature Deletion synthetic-task approach (component #2 "Synthetic data at 25× scale", mapping row (b), reward-hacking row (g)) into a real, usable data-generation subsystem for this framework. This is the design brief that mapping-table row (b) calls for ("Build 1 generator (Feature Deletion) as OpenEnv-compatible env"). Method: Live blog re-extraction (mcp_tavily_tavily_extract advanced) of cursor.com/blog/composer-2-5; substrate-dataset cards pulled live from HF/arXiv; TRL GRPOTrainer reward-fn convention confirmed against the TRL source. Tag convention (matches docs/COMPOSER_RECIPE_MAPPING.md): [BLOG-VERIFIED] = verbatim in the 2.5 blog; [INFERRED] = reasonable extrapolation from blog + open-source prior art; [EXTRAPOLATED] = our design addition, not Cursor-stated. Reads-before: docs/COMPOSER_RECIPE_MAPPING.md (§2, rows b/g) and research/09-composer-blog-delta-2026.md (online-curriculum delta). This file does not re-derive the Targeted-RL / SDPO material (that is rows (d) and research/05); it is the data-gen side only.

0. TL;DR

Feature Deletion is a self-verifying inverse task: take a repo whose test suite passes, programmatically remove a testable feature (so the suite now fails), and reward the agent for reimplementing it until the suite passes again. The reward is the pre-existing test suite — verifiable, no human labels, no golden patch needed at reward time. We can stand this up immediately on five open substrates (SWE-Gym, SWE-bench-Lite, R2E-Gym, SWE-rebench, OpenHands/Nemotron trajectories) by inverting their (repo, base_commit, gold_patch, test_patch) tuples instead of generating deletions from scratch. The two non-obvious requirements the blog forces on us: (1) an online pass-rate difficulty gate (the curriculum is dynamic, not a static bank — per the delta note), and (2) anti-reward-hacking sandboxing because Cursor observed the model recovering deleted signatures from bytecode/type-check caches. Below: a FeatureDeletionEnv Gym/OpenEnv class sketch wired for TRL GRPOTrainer (reward = test pass-fraction), the deletion mechanics (AST/file/coverage-mapped), the sandbox lockdown spec, and a CPU-pool cost model.

1. What "Feature Deletion" is, exactly [BLOG-VERIFIED]

Verbatim from the Synthetic-data section of the blog (re-pulled 2026-05-28):

"During RL training, Composer's coding ability improves substantially to the point where it begins to get most training problems correct. To continue increasing intelligence, we both select for and create harder tasks dynamically throughout the run. Composer 2.5 is trained with 25x more synthetic tasks than Composer 2.

We use a range of approaches for creating synthetic tasks that are grounded in real codebases. For example, one synthetic approach is feature deletion. For these tasks the agent is given a codebase with a large set of tests, and asked to delete code and files in such a way that the codebase remains functional while specific testable features are removed. The synthetic task is to reimplement the feature, and the tests are used as a verifiable reward."

Parse of the mechanism (note the two-agent / two-phase structure the blog implies):

Phase Actor Action Output
Deletion (task-construction) a deleter (model or program) "delete code and files in such a way that the codebase remains functional while specific testable features are removed" a broken_repo + the set of tests that now fail
Reimplementation (the training task) the policy under training reimplement the deleted feature a diff scored by the test suite
  • The deletion step is itself non-trivial: it must keep the codebase otherwise functional (imports resolve, unrelated tests still pass) while making specific testable features fail. [BLOG-VERIFIED] that this constraint exists; [INFERRED] that in practice this means partition the test suite into a kept set (PASS_TO_PASS) that must still pass and a target set (FAIL_TO_PASS) that the deletion must break.
  • Verifiable reward = the original test suite. No golden patch is needed at reward time (only at task-construction time, to know what "done" looks like). This is the key property that makes the env cheap to run in an RL loop.
  • The blog does not state: how deletion targets are selected, the deleter model, the languages beyond the Python/Java implied by the reward-hacking examples, or the difficulty heuristic. Those are the reproducibility gaps (consistent with research/09 §1 "NO CHANGE" line).

Relationship to the inverse-of-SWE-bench framing [INFERRED]: A SWE-bench-style instance is (repo@base_commit, problem_statement, gold_patch, test_patch, FAIL_TO_PASS, PASS_TO_PASS). Feature Deletion is the constructive inverse: instead of mining a human PR that fixed a bug, we apply revert(gold_patch) (or an AST-deletion) to a passing repo to manufacture the broken state, then ask the agent to re-derive gold_patch. This means every existing SWE- instance is already a ready-made Feature-Deletion task* — we get the deletion "for free" by reverting the gold patch. This is the single most important leverage point in this brief (see §4, §5).

2. The online difficulty curriculum [BLOG-VERIFIED] framing → [EXTRAPOLATED] design

The delta note (research/09 §1, DELTA-new-emphasis) is explicit that "select for and create harder tasks dynamically throughout the run" is a dynamic curriculum / online task-selection signal, not a static bank. Our generator must therefore expose a pass-rate gate, not just emit tasks. Design:

Difficulty signal. For each candidate task t, maintain an EMA of the policy's group pass-rate p̂(t) (TRL GRPO already samples G completions per prompt — we get G pass/fail observations per task per step for free). Define difficulty d(t) = 1 − p̂(t).

Two levers the blog names — "select for" and "create":

  1. select for (online filter / replay weighting) [EXTRAPOLATED]. Sampling weight over the task pool:
    • Drop solved tasks: if p̂(t) > τ_easy (e.g. 0.9) for k consecutive evaluations, retire t. This is exactly the blog's "begins to get most training problems correct" symptom.
    • Drop impossible tasks: if p̂(t) < τ_hard (e.g. 0.02) after k exposures, quarantine t (likely a broken-task or reward-hack-only task — see §3).
    • Up-weight the frontier: sample weight w(t) ∝ p̂(t)·(1−p̂(t)) (max variance ≈ max learning signal; standard curriculum-RL choice, cf. PLR / TD-error curricula). Keeps the policy on tasks it solves ~50% of the time.
  2. create (difficulty escalation) [EXTRAPOLATED]. When the pool's median p̂ rises above a band, the generator produces harder tasks. Concrete escalation knobs, easiest→hardest:
    • Deletion span: single-function → whole-class → whole-file → cross-file feature (more FAIL_TO_PASS tests, more LoC to reconstruct).
    • Hint starvation: strip docstrings / type hints / the deleted function's signature from the surrounding context (also a reward-hack-surface reduction, §3).
    • Coupling: delete a feature that several PASS_TO_PASS tests also exercise, so the agent must reconstruct it without breaking neighbors.
    • Multi-feature: delete n>1 independent features in one repo (reward = fraction of target tests passing — naturally graded).

Implementation handle (curriculum is a thin layer over the task pool):

# datagen/curriculum.py  [EXTRAPOLATED]
import math, random, collections

class PassRateCurriculum:
    """Online difficulty gate. Fed (task_id, n_pass, n_total) after each GRPO group."""
    def __init__(self, tau_easy=0.90, tau_hard=0.02, ema=0.3, retire_k=3):
        self.p   = collections.defaultdict(lambda: 0.5)   # EMA pass-rate
        self.seen = collections.Counter()
        self.retired, self.quarantined = set(), set()
        self.tau_easy, self.tau_hard, self.ema, self.retire_k = tau_easy, tau_hard, ema, retire_k

    def update(self, task_id, n_pass, n_total):
        r = n_pass / max(n_total, 1)
        self.p[task_id] = (1 - self.ema) * self.p[task_id] + self.ema * r
        self.seen[task_id] += 1
        if self.seen[task_id] >= self.retire_k:
            if self.p[task_id] > self.tau_easy: self.retired.add(task_id)
            elif self.p[task_id] < self.tau_hard: self.quarantined.add(task_id)  # likely broken / hack-only

    def weight(self, task_id):
        if task_id in self.retired or task_id in self.quarantined: return 0.0
        p = self.p[task_id]
        return p * (1 - p) + 1e-3          # frontier (max-variance) weighting

    def sample(self, task_ids, k):
        live = [t for t in task_ids if self.weight(t) > 0]
        w = [self.weight(t) for t in live]
        return random.choices(live, weights=w, k=k) if live else []

    def median_pass(self, task_ids):       # escalation trigger for the generator
        vals = sorted(self.p[t] for t in task_ids if t not in self.retired)
        return vals[len(vals)//2] if vals else 0.0

The trainer feeds update(...) from each GRPO group; the generator polls median_pass(...) and, when it crosses a band, emits a harder batch (more deletion span / more starvation). This is the minimal realization of "select for + create harder tasks dynamically."

3. Reward-hacking failure modes & programmatic safeguards [BLOG-VERIFIED] problem, [EXTRAPOLATED] mitigations

The blog (re-pulled verbatim) is the ground truth on the problem:

"One downstream consequence of large scale synthetic task creation is that it can cause unexpected reward hacking… In one example, the model found a leftover Python type-checking cache and reverse-engineered the format to find a deleted function signature. In another, it was able to find and decompile Java bytecode to reconstruct a third-party API. We were able to find and diagnose these problems using agentic monitoring tools, but they demonstrate the increasing care necessary for large scale RL."

The blog gives no mitigation specifics beyond "agentic monitoring tools" (confirmed unchanged in research/09 §4). So the following are our design [EXTRAPOLATED], consistent with mapping row (g) ("Static analysis + bytecode-cache-deletion + a sandboxed shell with no find/strings/unzip").

Root cause: Feature Deletion deletes source, but compilers/type-checkers/build tools leave shadow copies of the deleted information elsewhere in the working tree. The agent recovers the answer instead of reconstructing it. Two defense layers:

3a. Pre-task scrubbing (eliminate the leak at construction time)

Run after deletion, before the repo is handed to the agent:

Leak source Scrub action
Python bytecode delete all **/__pycache__/, *.pyc, *.pyo
Type-check caches delete .mypy_cache/, .pyre/, .pytype/, .dmypy.json, .ruff_cache/, .pytest_cache/
Compiled Java/JVM delete *.class, target/, build/, *.jar/*.war containing the deleted API; strip bundled deps
Build/dist artifacts delete dist/, *.egg-info/, *.so, build/, .tox/
VCS history run the agent on a squashed, detached worktree — no .git (else git log -p / git show recovers the deletion)
Editor/LSP indexes delete .idea/, .vscode/, *.code-workspace, ctags/tags, .cache/
Docs/stubs delete generated *.pyi stubs and built HTML/Sphinx docs that embed signatures 
# datagen/scrub.py  [EXTRAPOLATED]
import shutil, pathlib
LEAK_DIRS  = {"__pycache__",".mypy_cache",".pyre",".pytype",".ruff_cache",
              ".pytest_cache","target","build","dist",".tox",".idea",".vscode",".git",".cache"}
LEAK_GLOBS = ["*.pyc","*.pyo","*.class","*.jar","*.war","*.so","*.pyi",
              ".dmypy.json","tags","*.egg-info"]
def scrub(root: str):
    root = pathlib.Path(root)
    for p in root.rglob("*"):
        if p.is_dir() and p.name in LEAK_DIRS:
            shutil.rmtree(p, ignore_errors=True)
    for g in LEAK_GLOBS:
        for p in root.rglob(g):
            (shutil.rmtree(p, ignore_errors=True) if p.is_dir() else p.unlink(missing_ok=True))

3b. Runtime sandbox lockdown (block recovery if a leak survives)

  • Tool denylist in the agent's shell harness (matches mapping row g): no find, strings, unzip, jar, javap, unzip, objdump, grep -r over non-source dirs, uncompyle6/decompyle3/pycdc, cfr/procyon/fernflower (Java decompilers), git. Implement as an allowlisted command shim, not a blocklist (allowlist is the safe default).
  • Network egress = none (can't pip download the original package to read the API). Already required for determinism.
  • Read-only mounts for everything except the editable source tree; site-packages of the target package itself removed from the image.

3c. Post-hoc monitoring ("agentic monitoring tools" analogue) [EXTRAPOLATED]

A cheap programmatic monitor over the trajectory, run after a rollout passes, to retro-reject hacks:

  • AST diff check: the agent's accepted diff must contain new function/class bodies (AST nodes with statements), not just an import that re-exposes a surviving symbol. Reject solutions whose passing is explained purely by import/from … import * of a non-scrubbed module.
  • Provenance scan: flag any trajectory whose tool calls touched *.class, *.pyc, .mypy_cache, .git, or invoked a denied binary (defense-in-depth telemetry even with the shim).
  • Static byte-similarity gate: if the agent's reconstructed function is a near-exact byte copy of the (held-out) gold body and the agent never "wrote" it incrementally (single paste), flag for review — distinguishes reconstruction from recovery.
  • These produce a reward mask: reward = test_pass_fraction × (0 if hack_detected else 1). This is the concrete realization of mapping row (g)'s "+ RM-based penalty" without needing a learned RM in v0.1.

4. Open-source drop-in substrates

Every substrate below ships SWE-bench-shaped tuples (repo, base_commit, patch=gold, test_patch, FAIL_TO_PASS, PASS_TO_PASS). The Feature-Deletion mapping is identical for all of them: revert patch (or AST-delete the functions it touches) to manufacture broken_repo; FAIL_TO_PASS is the reward target; PASS_TO_PASS is the "stay-functional" guard the blog demands. Licenses verified live 2026-05-28.

Substrate HF dataset id Scale What it provides License FD-env mapping
SWE-bench / Lite / Verified SWE-bench/SWE-bench, SWE-bench/SWE-bench_Lite, SWE-bench/SWE-bench_Verified 2,294 / 534 / 500 Real GitHub issue→PR tuples, per-version test envs, pre-built Docker images (xingyaoww/sweb.eval.*, swebench/*) dataset CC-BY-4.0; per-repo source licenses vary (mostly Apache/MIT/BSD) Lite/Verified are the v0.0 smoke-test set (mapping row b: "use SWE-bench-lite only" in v0.0). Revert gold patch → FD task. Already-built images = no env-construction cost.
SWE-Gym / SWE-Gym-Raw SWE-Gym/SWE-Gym, SWE-Gym/SWE-Gym-Raw 2,438 (11 repos) / ~tens-of-k raw Same schema as SWE-bench but purpose-built for training (train split, not a held-out benchmark → no contamination worry); pre-built Docker images; verifier-training support. arXiv:2412.21139 (ICML 2025). check repo (SWE-Gym tooling MIT; instances inherit upstream repo licenses) Primary v0.1 FD substrate (mapping row b: "build Feature Deletion"). 2.4k clean training tasks, each invertible into an FD task with n difficulty escalations (§2).
R2E-Gym (V1 / Subset) R2E-Gym/R2E-Gym-V1, R2E-Gym/R2E-Gym-Subset, R2E-Gym/SWE-Bench-Lite 8.1K executable envs (13 repos); Subset = non-overlapping w/ SWE-bench SWE-GEN engine: procedurally generates executable envs directly from commits w/o human issues, + execution-assisted back-translation for problem statements + pre-built Docker images. arXiv (R2E-Gym, Jain et al. 2025). check repo (Apache-2.0 tooling typical; per-instance upstream licenses) Best scale substrate and the closest open analogue to Composer's "grounded in real codebases" generator. Its commit-diffs are feature-deletion candidates by construction (the commit added a feature; revert = delete it).
SWE-rebench nebius/SWE-rebench (+ nebius/SWE-bench-extra, nebius/SWE-agent-trajectories) 21,336 tasks, 3,468 repos Fully-automated mining pipeline; ships install_config, requirements, environment, docker_image, and LLM-scored difficulty + clarity annotations per task; FAIL_TO_PASS/PASS_TO_PASS/FAIL_TO_FAIL/PASS_TO_FAIL. arXiv:2505.20411 (NeurIPS 2025). dataset CC-BY-4.0; per-instance license_name field provided (56 distinct) — honor it per instance Largest + the only one with built-in difficulty scores → seeds the §2 curriculum's cold-start p̂(t) prior before any rollouts exist. The per-instance docker_image + install_config removes the 200-hr env-build bottleneck SWE-Gym reported.
OpenHands trajectories (via Nemotron-SWE-v1) nvidia/Nemotron-SWE-v1 59K agent trajectories OpenHands-framework SWE trajectories (Qwen3-Coder-480B teacher), issues sourced from SWE-Gym + R2E-Gym-Subset CC-BY-4.0 (subsets BSD-3 / Apache-2.0 / MIT per viewer) — "ready for commercial use" Not an FD-env itself — it's SFT/distill warm-start + a source of gold trajectories for the §3c monitor's "what legitimate reconstruction looks like" reference, and for research/05 trace-replay. Use as cold-start, not as the RL env.

Practical selection: v0.0 = SWE-bench-Lite (smoke). v0.1 = SWE-Gym (clean train) + SWE-rebench (scale + difficulty prior). R2E-Gym when we need to push past ~25k tasks toward the "25×" spirit. Nemotron/OpenHands trajectories = SFT warm-start + monitor reference, not the RL env. License rule baked into the loader: carry each instance's upstream repo license; filter out copyleft (GPL/AGPL) repos for any artifact we redistribute (we redistribute deletions/diffs, which are derivative works).

5. Deletion mechanics: producing the (broken_repo, test_command, golden_diff) tuple

Two construction paths; we implement both and let the curriculum pick granularity (§2).

Path A — Gold-patch reversion (cheap, the default for SWE-* substrates) [INFERRED]

The substrate already tells us exactly which lines implement a testable feature: the gold patch. So:

  1. git apply patch onto base_commit → functional repo, all tests pass (this is the substrate's "solved" state).
  2. golden_diff := patch (what the agent must re-derive); broken_repo := apply(reverse(patch)) → the feature is gone.
  3. FAIL_TO_PASS tests now fail (target); PASS_TO_PASS tests still pass (the "remains functional" guard — verify this, see §5c).
  4. Scrub (§3a), strip .git, freeze image.

Path B — Coverage-mapped AST deletion (true synthetic generation, no human PR needed) [EXTRAPOLATED]

This is the path that generalizes beyond mined PRs and lets us "create harder tasks" at will (R2E-Gym-style):

  1. Run the suite with coverage (coverage.py / pytest --cov) on the passing repo to get a test → {file:line-ranges} map.
  2. Pick a deletion target by granularity knob:
    • function-level: parse with ast/libcst, choose a FunctionDef/AsyncFunctionDef/ClassDef whose lines are covered by ≥1 test and that has high test selectivity (covered by few PASS_TO_PASS so the repo stays functional after removal).
    • file-level: a module imported by exactly the target tests.
    • feature-level: the transitive closure of a public symbol via an import/def graph (grimp/pydeps), bounded so unrelated tests survive.
  3. Delete via CST (replace body with raise NotImplementedError or remove the node and its now-dead imports). CST (libcst) preserves formatting and lets us re-insert a stub signature or not (the §2 "hint starvation" knob).
  4. golden_diff = the removed nodes (held out for the monitor; never shown to the agent).

5c. Guaranteeing the remaining tests exercise the deleted code (the blog's hard constraint)

The blog requires "the codebase remains functional while specific testable features are removed." Enforce as a construction-time validation gate — a task is only emitted if all four hold:

# datagen/build_task.py  [EXTRAPOLATED]  (pseudocode over a sandboxed runner)
def validate_task(repo_passing, broken_repo, target_tests, keep_tests, run):
    # 1. baseline sanity: full suite passes on the unbroken repo
    assert run(repo_passing, target_tests + keep_tests).all_pass
    res = run(broken_repo, target_tests + keep_tests)
    # 2. deletion actually breaks the target feature (tests now FAIL)
    assert all(res.failed(t) for t in target_tests)          # FAIL_TO_PASS non-empty & failing
    # 3. deletion left the rest functional (collection works, neighbors pass)
    assert res.collected_ok and all(res.passed(t) for t in keep_tests)  # PASS_TO_PASS guard
    # 4. solvability: gold diff restores green (the task is actually achievable)
    assert run(apply(broken_repo, golden_diff), target_tests + keep_tests).all_pass
    return Task(...)                                          # else discard

Gate (4) is what prevents the §2 "impossible task" quarantine pile-up — every emitted task is provably solvable by golden_diff. Gate (3) is the literal encoding of "remains functional." Task tuple emitted:

# datagen/schema.py  [EXTRAPOLATED]
from dataclasses import dataclass, field
@dataclass(frozen=True)
class FeatureDeletionTask:
    task_id: str
    repo: str                       # e.g. "getmoto/moto"
    base_commit: str
    broken_image: str               # docker tag of the scrubbed broken repo (frozen env)
    test_command: str               # e.g. "python -m pytest -q"
    fail_to_pass: tuple[str, ...]   # reward target (must go red→green)
    pass_to_pass: tuple[str, ...]   # functional guard (must stay green)
    golden_diff: str = field(repr=False)   # HELD OUT — monitor/solvability only, never in obs
    granularity: str = "function"   # function|file|feature  (curriculum escalation)
    deleted_symbols: tuple[str, ...] = ()  # for AST-provenance monitor (§3c)
    upstream_license: str = "unknown"      # carried from substrate; gates redistribution
    difficulty_prior: float = 0.5          # seeded from SWE-rebench LLM score if available

6. FeatureDeletionEnv — OpenEnv/Gym-style design for TRL GRPOTrainer + verifiers

Integration contract. TRL's GRPOTrainer takes a dataset of prompts and one or more reward functions with the calling convention reward_fn(prompts: list[str], completions: list[str], **kwargs) -> list[float] (the dataset's non-prompt columns are passed through as **kwargs; confirmed against the TRL grpo_trainer.py source and the RewardFunc type-alias fix in TRL PR #5246). So the env exposes two faces: a Gym/OpenEnv face (reset/step for multi-turn agentic rollout via OpenEnv, mapping row c) and a reward_fn adapter that GRPO calls directly. Reward = test pass fraction (|FAIL_TO_PASS passing| / |FAIL_TO_PASS|), naturally graded for multi-feature tasks, masked by the hack monitor (§3c).

# envs/feature_deletion_env.py  [EXTRAPOLATED]
# Gym/OpenEnv-style env + a TRL GRPO reward adapter. Execution happens in the
# locked-down sandbox of §3b; this class is the orchestration shell.
from dataclasses import dataclass
from datagen.schema import FeatureDeletionTask

@dataclass
class StepResult:
    observation: str            # tool output / test stdout shown to the agent
    reward: float               # only nonzero on a terminal "submit" step
    done: bool
    info: dict

class FeatureDeletionEnv:
    """One task per episode. Sandbox = allowlisted shell, no net, scrubbed tree (§3)."""
    def __init__(self, sandbox, monitor, max_turns: int = 40):
        self.sandbox, self.monitor, self.max_turns = sandbox, monitor, max_turns
        self.task: FeatureDeletionTask | None = None

    # ---- Gym/OpenEnv face (multi-turn agentic rollout) ----
    def reset(self, task: FeatureDeletionTask) -> str:
        self.task, self.turns = task, 0
        self.sandbox.boot(task.broken_image)           # read-only except editable src; egress off
        # NOTE: golden_diff / deleted_symbols are NEVER placed in the observation.
        return self._render_prompt(task)               # task desc + failing-test names + tool list

    def step(self, action: dict) -> StepResult:
        self.turns += 1
        if action["type"] == "submit" or self.turns >= self.max_turns:
            return self._grade()
        obs = self.sandbox.exec(action)                # edit / run-tests / read-file (allowlisted)
        return StepResult(obs, 0.0, False, {"turn": self.turns})

    def _grade(self) -> StepResult:
        r = self.sandbox.run_tests(self.task.test_command,
                                   self.task.fail_to_pass + self.task.pass_to_pass)
        frac = r.n_pass(self.task.fail_to_pass) / max(len(self.task.fail_to_pass), 1)
        guard_ok = r.all_pass(self.task.pass_to_pass)          # "remains functional"
        hacked = self.monitor.flag(self.sandbox.trajectory(),  # AST + provenance (§3c)
                                   self.task.deleted_symbols)
        reward = frac * (1.0 if guard_ok and not hacked else 0.0)
        return StepResult(r.stdout, reward, True,
                          {"frac": frac, "guard_ok": guard_ok, "hacked": hacked})

    # ---- TRL GRPOTrainer face (reward_fn(prompts, completions, **kwargs)->list[float]) ----
    def reward_fn(self, prompts, completions, *, task_id=None, **kwargs):
        rewards = []
        for comp, tid in zip(completions, task_id):            # task_id passed via dataset column
            task = self.registry[tid]
            self.reset(task)
            res = self._run_completion(comp)                   # replay agent turns from `comp`
            rewards.append(res.reward)
            self.curriculum.update(tid, n_pass=int(res.reward > 0), n_total=1)  # §2 feedback
        return rewards

Wiring to GRPO (the dataset carries task_id; curriculum reweights the sampler):

# train/grpo_fd.py  [EXTRAPOLATED]
from trl import GRPOTrainer, GRPOConfig
env = FeatureDeletionEnv(sandbox=LockedSandbox(...), monitor=HackMonitor(...))
ds  = build_prompt_dataset(tasks)                  # columns: prompt, task_id  (+ curriculum weights)
trainer = GRPOTrainer(
    model="Qwen/Qwen3-Coder-7B",                   # v0.0 base (mapping row a)
    args=GRPOConfig(num_generations=8, ...),       # G=8 → 8 pass/fail obs per task per step → §2
    reward_funcs=[env.reward_fn],                  # reward = masked test pass-fraction
    train_dataset=ds,
)
trainer.train()

This slots into the same RLVR base as rows (c)/(d); the SDPO hint-distill channel (row d, research/05) is orthogonal and stacks on top — Feature Deletion supplies the verifiable scalar reward that SDPO's KL rides on. The verifiers library can wrap reward_fn for env composition if we run multiple generators.

7. Cost & feasibility at scale (CPU pools)

Feature-Deletion is embarrassingly parallel and CPU-bound — no GPU in the data-gen path (matches mapping §"Synthetic data: Generators run on CPU pool… Embarrassingly parallel"). Two cost buckets:

(A) Task construction (one-time per task). [EXTRAPOLATED] estimates:

  • Path A (gold-patch revert) over a pre-built substrate image: git apply -R + scrub + one validation suite run. Dominated by the test run: ~30 s–5 min CPU per task depending on suite size. Validation gate (§5c) needs 2 suite runs (broken + gold-restored) → call it **2–10 min CPU/task**.
  • Path B (AST deletion): + a coverage run (1.5–3× a normal suite run) + AST/CST manipulation (<1 s). **5–20 min CPU/task.**
  • Throughput: a 64-vCPU pool at 8 min/task and 8 concurrent runners ≈ **60 tasks/hr/node** → 1,400 tasks/day/node. Inverting all 21k SWE-rebench instances ≈ **15 node-days** on one 64-vCPU box, trivially parallel across nodes. Reaching a "25×-spirit" pool of ~50k–60k tasks (R2E-Gym 8.1k + SWE-rebench 21k + multi-feature/granularity escalations) is <1 week on a modest CPU pool.
  • Storage/images: reuse substrate Docker images (SWE-Gym xingyaoww/sweb.eval.*, SWE-rebench per-instance docker_image) → near-zero env-build cost, sidestepping the "200 hours of manual env setup" bottleneck SWE-Gym reported. We only add a thin scrubbed overlay layer per task (~MBs).

(B) Reward evaluation (recurring, in the RL loop). This is the real running cost, not construction: each GRPO step runs G rollouts × (agent turns + final test run). Test execution is CPU; agent generation is the GPU/inference cost shared with rows (c)/(d). Levers: cache the broken image warm, run only FAIL_TO_PASS + PASS_TO_PASS (not the full suite), and retire solved tasks via §2 so we stop paying for tasks the model already aced ("begins to get most training problems correct").

Feasibility verdict: Green. Construction is cheap and one-time; the curriculum keeps the live pool small; the only nontrivial recurring cost (sandboxed test execution) is shared with any RLVR coding env we'd build anyway. The binding constraints are engineering (sandbox lockdown §3, validation gate §5c) and licensing hygiene (§4), not compute.

8. Open questions / reproducibility gaps (carried from blog silence)

  1. Deletion-target selection heuristic — blog silent (research/09 §1 "NO CHANGE"). We propose coverage-selectivity (§5 Path B); Cursor's actual heuristic is unknown.
  2. Deleter model vs. program — blog implies an agent deletes ("asked to delete code… such that the codebase remains functional"); we default to programmatic deletion (cheaper, deterministic, no second model). An LLM-deleter is a v0.2 escalation.
  3. The other generators (count UNKNOWN) — Feature Deletion is "one synthetic approach… a range of approaches"; the rest are unnamed and uncounted (the old "~24" was a back-formation from the 25x task multiplier — deepread finding V5). Out of scope here; this brief delivers the one named generator.
  4. "Agentic monitoring tools" internals — unspecified; our §3c monitor is a best-effort programmatic stand-in.
  5. Composer2.pdf (arXiv:2603.24477) — flagged by research/09 action-item #1 as the likely home of data-mix % and generator inventory; not yet extracted. Recommend a follow-up pull before scaling the generator suite.

Sources