Data World: Scaling Data Environments from Scratch
Abstract
A multi-agent engine synthesizes data environments and tasks to train models for analytical insight discovery, improving performance across benchmarks without trajectory distillation.
Data agents use code to process heterogeneous files and derive analytical insights. Scaling their training faces costly environment curation, privacy and intellectual-property restrictions, and difficulty verifying open-ended findings without known generative truth. Our core insight is to specify the world behind the files before generating the files themselves. We introduce Data World, a multi-agent engine that synthesizes environments, tasks, and rubrics from explicit mechanisms. Independent audits check target recoverability from observed files; rubrics support evaluation and RL rewards. Human preference-based curation for question realism selects 100 tasks across 50 held-out worlds and 16 domains for DataWorld-Bench. The engine also yields 2,135 training tasks from 1,037 disjoint worlds. Evaluation of ten frontier models shows that the benchmark remains challenging, particularly for insight discovery, with a consistent computation-insight gap. RL-zero training on the synthesized tasks produces DataWorld-35B-A3B from Qwen3.6-35B-A3B without trajectory distillation. Under matched evaluation, it improves over its base by 7.06 points on DataWorld-Bench, 14.28 on DAComp, 20.90 on DABstep-Hard, and 12.13 on JobBench, a broader occupational benchmark.
Community
Since early this year, we've been following the data agent space, where a common pain point keeps surfacing: agentic data-analysis tasks are notoriously hard to verify — take "why did food-delivery sales drop?"; even human experts can't give one definitive answer. So we flipped the question: what if we played God of the data world, fixing the ground truth (say, sales dropped because of the weather) before any data exists? From that generative mechanism we synthesize the environment itself — data, tasks, and scoring rubrics all at once — making the ground truth nearly impossible to dispute while giving us full control over task taxonomy and environment scaling. That is the core idea of Data World: don't distill stronger models, build better training environments. Across ten frontier models we observe a consistent compute–insight gap, and after RL-zero training on 2,135 synthetic tasks — without distilling any solver traces — our DataWorld-35B-A3B delivers major gains on JobBench and DAComp.
Get this paper in your agent:
hf papers read 2610.21656 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash