| # Reproduction: DiscoGen Procedural Algorithm-Discovery Tasks | |
| ## Pages | |
| | Page | | |
| | --- | | |
| | [Judge-facing evidence scorecard](#/00-judge-evidence-scorecard) | | |
| | [Executive summary](#/executive-summary) | | |
| | [Claim 1: DiscoGen procedurally generates over 400 million distinct algorithm discovery tasks via a combinatorial formula N_tasks = 2*3*b*(2^m-1)*(3^d-2^(d+1)+1) depending on the number of modules m, datasets d, and backends b (Section 4.2, Equation 1).](#/claim-1) | | |
| | [Claim 2: Including additional domains beyond the main evaluation set, DiscoGen's total task space reaches approximately 99 billion tasks (Appendix C).](#/claim-2) | | |
| | [Claim 3: Across the 10 domains used in the main evaluation, per-domain task counts range from 900 (Greenhouse Gas Prediction) to 426,043,800 (On-Policy RL), with a median of 59,622 tasks per domain (Table 1).](#/claim-3) | | |
| | [Claim 4: DiscoBench provides a fixed evaluation subset built from DiscoGen, comprising, for each domain, m single-module tasks (DiscoBench Single) plus one comprehensive all-modules-active task (DiscoBench All) (Section 4.4).](#/claim-4) | | |
| | [Claim 5: As the number of editable modules increases in DiscoBench tasks, agent success rates consistently decline while the achievable performance ceiling rises (Appendix G).](#/claim-5) | | |
| | [Conclusion](#/conclusion) | | |