KoeYe commited on
Commit
9ff787d
·
verified ·
1 Parent(s): 47e9361

Link the paper (arXiv:2609.36777): buttons, abstract, BibTeX

Browse files
Files changed (3) hide show
  1. README.md +2 -0
  2. cases.html +2 -2
  3. index.html +26 -15
README.md CHANGED
@@ -14,4 +14,6 @@ short_description: Benchmarking coding agents that build and edit 3D scenes
14
  Project page for *Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes*: the leaderboard on the
15
  95-case public set, the paper's takeaways, and selected cases whose saved Unreal Engine scenes can be viewed in 3D.
16
 
 
 
17
  Part of the [SimWorld](https://simworld.org) project.
 
14
  Project page for *Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes*: the leaderboard on the
15
  95-case public set, the paper's takeaways, and selected cases whose saved Unreal Engine scenes can be viewed in 3D.
16
 
17
+ Paper: [arXiv:2609.36777](https://arxiv.org/abs/2609.36777) · Code: [SimWorld-AI/Code4Scene](https://github.com/SimWorld-AI/Code4Scene)
18
+
19
  Part of the [SimWorld](https://simworld.org) project.
cases.html CHANGED
@@ -82,7 +82,7 @@ table.cs td,table.cs th{white-space:nowrap}.panel code{font-size:.82em;backgroun
82
  .in-open:hover{background:var(--primary);color:#fff}
83
  @media(max-width:900px){.in-i2s{grid-template-columns:1fr}}
84
  </style><script type="importmap">{"imports":{"three":"https://cdn.jsdelivr.net/npm/three@0.160.0/build/three.module.js","three/addons/":"https://cdn.jsdelivr.net/npm/three@0.160.0/examples/jsm/"}}</script></head><body>
85
- <nav class="navbar"><a class="navbar-brand" href="index.html"><svg viewBox="0 0 32 32" aria-hidden="true"><defs><linearGradient id="gi" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#92400e"/><stop offset="1" stop-color="#d97706"/></linearGradient></defs><rect x="1" y="1" width="30" height="30" rx="8" fill="url(#gi)"/><path d="M16 6.5 25 11.5v9L16 25.5 7 20.5v-9z" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/><path d="M7 11.5 16 16.5 25 11.5M16 16.5v9" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/></svg>Code4Scene</a><ul class="navbar-links"><li class=""><a href="index.html#leaderboard">Leaderboard</a></li><li class="hide-sm"><a href="index.html#takeaways">Takeaways</a></li><li class=""><a href="cases.html" class=on>Cases in 3D</a></li><li class="hide-sm"><a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> Code</a></li><li><a class="btn-nav-primary" href="https://simworld.org/" target="_blank" rel="noopener"><img src="img/simworld-logo.png" alt="">SimWorld Home</a></li></ul></nav>
86
  <section class="alt" style="padding-top:calc(var(--nav-h) + 3rem);padding-bottom:3rem"><div class="container">
87
  <div class="section-header" style="margin-bottom:1.6rem"><span class="section-label">Cases</span><h2 class="section-title">Every scene in 3D, with its scores</h2>
88
  <p class="section-desc">Each agent's result is the engine-native scene it saved, exported from Unreal Engine so you can turn it in the browser. Beside it, how the
@@ -95,7 +95,7 @@ evaluator scored that scene. 6 cases, 41 scenes in 3D. Pick a case, then an agen
95
  <div class="card" style="margin-top:1.5rem;padding:1.4rem 1.5rem"><div class="sub-title">Every agent on this case</div>
96
  <div class="sub-desc">The evaluator's scores for all 14 agents on the selected case. Click a row to open that agent's scene.</div><div class="board-wrap" id="ctable"></div></div>
97
  </div></section>
98
- <footer><p>Code4Scene &nbsp;·&nbsp; UC San Diego &amp; UC Berkeley &nbsp;·&nbsp; <a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener">GitHub</a> &nbsp;·&nbsp;
99
  <a href="https://simworld.org/" target="_blank" rel="noopener">SimWorld Home</a></p>
100
  <p style="margin-top:.5rem;font-size:.75rem;color:rgba(255,255,255,.25)">Part of the SimWorld project.</p></footer>
101
  <script src="assets/paper.js?v=5427505a"></script><script src="data/site.js?v=0b09af46"></script><script type="module">import * as THREE from "three";
 
82
  .in-open:hover{background:var(--primary);color:#fff}
83
  @media(max-width:900px){.in-i2s{grid-template-columns:1fr}}
84
  </style><script type="importmap">{"imports":{"three":"https://cdn.jsdelivr.net/npm/three@0.160.0/build/three.module.js","three/addons/":"https://cdn.jsdelivr.net/npm/three@0.160.0/examples/jsm/"}}</script></head><body>
85
+ <nav class="navbar"><a class="navbar-brand" href="index.html"><svg viewBox="0 0 32 32" aria-hidden="true"><defs><linearGradient id="gi" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#92400e"/><stop offset="1" stop-color="#d97706"/></linearGradient></defs><rect x="1" y="1" width="30" height="30" rx="8" fill="url(#gi)"/><path d="M16 6.5 25 11.5v9L16 25.5 7 20.5v-9z" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/><path d="M7 11.5 16 16.5 25 11.5M16 16.5v9" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/></svg>Code4Scene</a><ul class="navbar-links"><li class=""><a href="index.html#leaderboard">Leaderboard</a></li><li class="hide-sm"><a href="index.html#takeaways">Takeaways</a></li><li class=""><a href="cases.html" class=on>Cases in 3D</a></li><li><a href="https://arxiv.org/abs/2609.36777" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M4 0h5.5L14 4.5V14a2 2 0 01-2 2H4a2 2 0 01-2-2V2a2 2 0 012-2zm5 1.5V5h3.5L9 1.5zM5 8h6v1H5zm0 2.5h6v1H5z"/></svg> Paper</a></li><li class="hide-sm"><a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> Code</a></li><li><a class="btn-nav-primary" href="https://simworld.org/" target="_blank" rel="noopener"><img src="img/simworld-logo.png" alt="">SimWorld Home</a></li></ul></nav>
86
  <section class="alt" style="padding-top:calc(var(--nav-h) + 3rem);padding-bottom:3rem"><div class="container">
87
  <div class="section-header" style="margin-bottom:1.6rem"><span class="section-label">Cases</span><h2 class="section-title">Every scene in 3D, with its scores</h2>
88
  <p class="section-desc">Each agent's result is the engine-native scene it saved, exported from Unreal Engine so you can turn it in the browser. Beside it, how the
 
95
  <div class="card" style="margin-top:1.5rem;padding:1.4rem 1.5rem"><div class="sub-title">Every agent on this case</div>
96
  <div class="sub-desc">The evaluator's scores for all 14 agents on the selected case. Click a row to open that agent's scene.</div><div class="board-wrap" id="ctable"></div></div>
97
  </div></section>
98
+ <footer><p>Code4Scene &nbsp;·&nbsp; UC San Diego &amp; UC Berkeley &nbsp;·&nbsp; <a href="https://arxiv.org/abs/2609.36777" target="_blank" rel="noopener">arXiv:2609.36777</a> &nbsp;·&nbsp; <a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener">GitHub</a> &nbsp;·&nbsp;
99
  <a href="https://simworld.org/" target="_blank" rel="noopener">SimWorld Home</a></p>
100
  <p style="margin-top:.5rem;font-size:.75rem;color:rgba(255,255,255,.25)">Part of the SimWorld project.</p></footer>
101
  <script src="assets/paper.js?v=5427505a"></script><script src="data/site.js?v=0b09af46"></script><script type="module">import * as THREE from "three";
index.html CHANGED
@@ -21,7 +21,7 @@ svg.ch .dim{opacity:.42}svg.ch g.row:hover{opacity:1}svg.ch .hot{fill:var(--c4s-
21
  .bigstats{display:flex;flex-wrap:wrap;gap:12px 34px;margin:0 0 12px}.bigstats b{display:block;font-size:30px;letter-spacing:-.02em;font-variant-numeric:tabular-nums}
22
  .bigstats span{font-size:13px;color:var(--text-secondary)}
23
  </style></head><body>
24
- <nav class="navbar"><a class="navbar-brand" href="index.html"><svg viewBox="0 0 32 32" aria-hidden="true"><defs><linearGradient id="gi" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#92400e"/><stop offset="1" stop-color="#d97706"/></linearGradient></defs><rect x="1" y="1" width="30" height="30" rx="8" fill="url(#gi)"/><path d="M16 6.5 25 11.5v9L16 25.5 7 20.5v-9z" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/><path d="M7 11.5 16 16.5 25 11.5M16 16.5v9" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/></svg>Code4Scene</a><ul class="navbar-links"><li class=""><a href="index.html#leaderboard" class=on>Leaderboard</a></li><li class="hide-sm"><a href="index.html#takeaways">Takeaways</a></li><li class=""><a href="cases.html">Cases in 3D</a></li><li class="hide-sm"><a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> Code</a></li><li><a class="btn-nav-primary" href="https://simworld.org/" target="_blank" rel="noopener"><img src="img/simworld-logo.png" alt="">SimWorld Home</a></li></ul></nav>
25
  <section class="hero" id="hero"><div class="hero-inner">
26
  <h1 class="hero-title"><svg viewBox="0 0 32 32" aria-hidden="true"><defs><linearGradient id="gi" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#92400e"/><stop offset="1" stop-color="#d97706"/></linearGradient></defs><rect x="1" y="1" width="30" height="30" rx="8" fill="url(#gi)"/><path d="M16 6.5 25 11.5v9L16 25.5 7 20.5v-9z" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/><path d="M7 11.5 16 16.5 25 11.5M16 16.5v9" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/></svg><span class="highlight">Code4Scene</span>:<br>Benchmarking Coding Agents for<br>Constructing and Editing 3D Scenes</h1>
27
  <p class="hero-subtitle">190 Unreal Engine cases built from human-assembled scenes. Coding agents write and run code that builds a scene from text, or
@@ -32,7 +32,7 @@ repairs one from reference images, and Code4Scene scores the engine-native scene
32
  <span class="author">Lianhui Qin</span><sup>1,†</sup></p>
33
  <p class="hero-affiliations"><sup>1</sup>University of California San Diego &nbsp;&nbsp; <sup>2</sup>University of California, Berkeley &nbsp;&nbsp;
34
  <sup>*</sup>Equal contribution &nbsp;&nbsp; <sup>†</sup>Corresponding author</p>
35
- <div class="hero-buttons"><span class="btn btn-primary off" title="Coming soon"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M4 0h5.5L14 4.5V14a2 2 0 01-2 2H4a2 2 0 01-2-2V2a2 2 0 012-2zm5 1.5V5h3.5L9 1.5zM5 8h6v1H5zm0 2.5h6v1H5z"/></svg> Paper (soon)</span>
36
  <a class="btn btn-outline" href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> Code &amp; Data</a>
37
  <a class="btn btn-dark" href="#teaser"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0a8 8 0 110 16A8 8 0 018 0zM6.5 4.8v6.4L11.3 8z"/></svg> Watch the teaser</a><a class="btn btn-dark" href="cases.html"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="M8 1.5l5.5 3v7L8 14.5l-5.5-3v-7z"/><path d="M2.5 4.5L8 7.5l5.5-3M8 7.5v7"/></svg> Cases in 3D</a></div>
38
  <div class="hero-stats">
@@ -45,14 +45,20 @@ repairs one from reference images, and Code4Scene scores the engine-native scene
45
  <p class="section-desc">The leaderboard, one scene through the evaluator, two scenes compared, and what the benchmark finds, in 90 seconds.</p></div>
46
  <div class="teaser-frame"><video controls playsinline preload="metadata" poster="media/teaser-poster.jpg"><source src="media/teaser.mp4" type="video/mp4"></video></div>
47
  </div></section><section class="alt" id="overview"><div class="container">
48
- <div class="section-header centered"><span class="section-label">Overview</span><h2 class="section-title">Score the scene, not the code or a render</h2></div>
49
- <div class="card abstract-card"><p>Coding agents can now operate 3D engines: they write and run code, inspect the result and revise the scene. Code4Scene is a
50
- benchmark of 190 Unreal Engine cases built from human-assembled scenes that evaluates them on two complementary settings under a shared execution
51
- interface. <b>Construction</b> starts from an empty level and an open-ended text specification. <b>Editing</b> gives the agent a corrupted copy of a
52
- scene and reference images of the original, and asks it to restore the intended state while leaving everything else untouched.</p>
53
- <p>Rather than scoring code or rendered views, Code4Scene evaluates the engine-native scene the agent saved, on task fulfilment, artifact integrity
54
- and static physical validity, and compares edits against withheld ground truth. Across 14 agent configurations, construction and editing rank the
55
- agents differently, spatial relations remain the weakest requirement family, and a third of complete repairs still change the surrounding scene.</p></div>
 
 
 
 
 
 
56
  <div class="flow"><div class="step"><span class="n">1 · Input</span><b>Prompt or reference</b><span>An open-ended scene description, or a corrupted scene with reference images of the original.</span></div><div class="step"><span class="n">2 · Agent</span><b>Coding agent</b><span>A frontier or open-weight model in its own CLI harness, with the Unreal Engine editor as a tool.</span></div><div class="step"><span class="n">3 · Code</span><b>Code in the engine</b><span>It writes and runs Python against the editor: spawns, moves and edits actors, inspects, revises.</span></div><div class="step"><span class="n">4 · Scene</span><b>Engine-native scene</b><span>The saved level (.umap): every actor, transform and material, as the engine holds it.</span></div><div class="step eval"><span class="n">5 · Evaluator</span><b>Scored in the engine</b><span>Requirements located and judged, overview views judged, physics measured, edits diffed against the withheld ground truth.</span></div></div>
57
  <div class="grid2" style="margin-top:2rem">
58
  <div class="card setting-card"><span class="kicker">Setting 1 · construction</span><h3>Text-to-Scene</h3>
@@ -86,16 +92,21 @@ evaluator judges requirements rather than matching a reference. 20 public cases.
86
  <div style="text-align:center;margin-top:2rem"><a class="btn btn-primary" href="cases.html"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="M8 1.5l5.5 3v7L8 14.5l-5.5-3v-7z"/><path d="M2.5 4.5L8 7.5l5.5-3M8 7.5v7"/></svg> Open the case explorer</a></div>
87
  </div></section><section class="dark" id="citation"><div class="container">
88
  <div class="section-header centered"><span class="section-label">Citation</span><h2 class="section-title">Cite This Work</h2></div>
89
- <div class="cite-box"><button class="cite-copy-btn" id="copy-btn">Copy</button><pre id="bibtex-text">@article{code4scene2026,
90
- title = {Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes},
91
- author = {Ye, Xiaokang and Mantri, Siddhant Hitesh and Chen, Zimeng and Zhang, Edward and Zheng, Zhaoxu and Li, Yuanheng and Chen, Yizhao and Huang, Tianyang and Qin, Lianhui},
92
- year = {2026}
 
 
 
 
93
  }</pre></div>
94
  <div style="display:flex;flex-wrap:wrap;gap:1rem;justify-content:center;margin-top:2rem">
 
95
  <a class="btn btn-outline" href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> GitHub Repository</a>
96
  <a class="btn btn-outline" href="https://simworld.org/" target="_blank" rel="noopener">SimWorld Main Site</a></div>
97
  </div></section>
98
- <footer><p>Code4Scene &nbsp;·&nbsp; UC San Diego &amp; UC Berkeley &nbsp;·&nbsp; <a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener">GitHub</a> &nbsp;·&nbsp;
99
  <a href="https://simworld.org/" target="_blank" rel="noopener">SimWorld Home</a></p>
100
  <p style="margin-top:.5rem;font-size:.75rem;color:rgba(255,255,255,.25)">Part of the SimWorld project.</p></footer>
101
  <script src="assets/paper.js?v=5427505a"></script>
 
21
  .bigstats{display:flex;flex-wrap:wrap;gap:12px 34px;margin:0 0 12px}.bigstats b{display:block;font-size:30px;letter-spacing:-.02em;font-variant-numeric:tabular-nums}
22
  .bigstats span{font-size:13px;color:var(--text-secondary)}
23
  </style></head><body>
24
+ <nav class="navbar"><a class="navbar-brand" href="index.html"><svg viewBox="0 0 32 32" aria-hidden="true"><defs><linearGradient id="gi" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#92400e"/><stop offset="1" stop-color="#d97706"/></linearGradient></defs><rect x="1" y="1" width="30" height="30" rx="8" fill="url(#gi)"/><path d="M16 6.5 25 11.5v9L16 25.5 7 20.5v-9z" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/><path d="M7 11.5 16 16.5 25 11.5M16 16.5v9" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/></svg>Code4Scene</a><ul class="navbar-links"><li class=""><a href="index.html#leaderboard" class=on>Leaderboard</a></li><li class="hide-sm"><a href="index.html#takeaways">Takeaways</a></li><li class=""><a href="cases.html">Cases in 3D</a></li><li><a href="https://arxiv.org/abs/2609.36777" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M4 0h5.5L14 4.5V14a2 2 0 01-2 2H4a2 2 0 01-2-2V2a2 2 0 012-2zm5 1.5V5h3.5L9 1.5zM5 8h6v1H5zm0 2.5h6v1H5z"/></svg> Paper</a></li><li class="hide-sm"><a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> Code</a></li><li><a class="btn-nav-primary" href="https://simworld.org/" target="_blank" rel="noopener"><img src="img/simworld-logo.png" alt="">SimWorld Home</a></li></ul></nav>
25
  <section class="hero" id="hero"><div class="hero-inner">
26
  <h1 class="hero-title"><svg viewBox="0 0 32 32" aria-hidden="true"><defs><linearGradient id="gi" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#92400e"/><stop offset="1" stop-color="#d97706"/></linearGradient></defs><rect x="1" y="1" width="30" height="30" rx="8" fill="url(#gi)"/><path d="M16 6.5 25 11.5v9L16 25.5 7 20.5v-9z" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/><path d="M7 11.5 16 16.5 25 11.5M16 16.5v9" fill="none" stroke="#fff" stroke-width="2" stroke-linejoin="round"/></svg><span class="highlight">Code4Scene</span>:<br>Benchmarking Coding Agents for<br>Constructing and Editing 3D Scenes</h1>
27
  <p class="hero-subtitle">190 Unreal Engine cases built from human-assembled scenes. Coding agents write and run code that builds a scene from text, or
 
32
  <span class="author">Lianhui Qin</span><sup>1,†</sup></p>
33
  <p class="hero-affiliations"><sup>1</sup>University of California San Diego &nbsp;&nbsp; <sup>2</sup>University of California, Berkeley &nbsp;&nbsp;
34
  <sup>*</sup>Equal contribution &nbsp;&nbsp; <sup>†</sup>Corresponding author</p>
35
+ <div class="hero-buttons"><a class="btn btn-primary" href="https://arxiv.org/abs/2609.36777" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M4 0h5.5L14 4.5V14a2 2 0 01-2 2H4a2 2 0 01-2-2V2a2 2 0 012-2zm5 1.5V5h3.5L9 1.5zM5 8h6v1H5zm0 2.5h6v1H5z"/></svg> Paper</a>
36
  <a class="btn btn-outline" href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> Code &amp; Data</a>
37
  <a class="btn btn-dark" href="#teaser"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0a8 8 0 110 16A8 8 0 018 0zM6.5 4.8v6.4L11.3 8z"/></svg> Watch the teaser</a><a class="btn btn-dark" href="cases.html"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="M8 1.5l5.5 3v7L8 14.5l-5.5-3v-7z"/><path d="M2.5 4.5L8 7.5l5.5-3M8 7.5v7"/></svg> Cases in 3D</a></div>
38
  <div class="hero-stats">
 
45
  <p class="section-desc">The leaderboard, one scene through the evaluator, two scenes compared, and what the benchmark finds, in 90 seconds.</p></div>
46
  <div class="teaser-frame"><video controls playsinline preload="metadata" poster="media/teaser-poster.jpg"><source src="media/teaser.mp4" type="video/mp4"></video></div>
47
  </div></section><section class="alt" id="overview"><div class="container">
48
+ <div class="section-header centered"><span class="section-label">Abstract</span><h2 class="section-title">Score the scene, not the code or a render</h2></div>
49
+ <div class="card abstract-card"><p>Frontier coding agents can now write and execute code that authors 3D environments, but whether they reliably
50
+ understand 3D structure and precisely control scene state remains unclear. The generated 3D scene is a persistent, executable artifact: a convincing
51
+ render can hide incorrect spatial relations, intersecting objects, or unintended modifications. We introduce Code4Scene, a benchmark of 190 Unreal
52
+ Engine cases built from human-assembled scenes that evaluates coding agents on two complementary settings under a shared execution interface.
53
+ Construction tests scene-level spatial reasoning from open-ended language specifications, where many realizations are valid; editing tests precise
54
+ control of scene state, where the agent must recover the target scene from reference images while preserving everything else.</p>
55
+ <p>Rather than scoring code or rendered views, Code4Scene evaluates the generated engine-native scene for task fulfillment, artifact integrity, and
56
+ static physical validity, with edits additionally compared against withheld ground truth. Across 14 coding-agent configurations on the 95-case public
57
+ set, construction and editing performance are strongly correlated but not interchangeable (Spearman ρ = 0.78): Claude Fable 5.1 leads construction,
58
+ Gemini 3.8 Flash leads editing, and GPT-6 Astra narrowly leads overall. Spatial Composition is the weakest construction category for every agent,
59
+ while editing remains imprecise: the best Repair F1 is only 0.527, and 35.8% of edits that fully recover the target still introduce unintended
60
+ changes elsewhere in the scene. These results expose a gap between plausible 3D generation and reliable spatial reasoning and state control.</p>
61
+ <p style="margin-top:1rem;font-size:.9rem"><a href="https://arxiv.org/abs/2609.36777" target="_blank" rel="noopener">Read the paper on arXiv →</a></p></div>
62
  <div class="flow"><div class="step"><span class="n">1 · Input</span><b>Prompt or reference</b><span>An open-ended scene description, or a corrupted scene with reference images of the original.</span></div><div class="step"><span class="n">2 · Agent</span><b>Coding agent</b><span>A frontier or open-weight model in its own CLI harness, with the Unreal Engine editor as a tool.</span></div><div class="step"><span class="n">3 · Code</span><b>Code in the engine</b><span>It writes and runs Python against the editor: spawns, moves and edits actors, inspects, revises.</span></div><div class="step"><span class="n">4 · Scene</span><b>Engine-native scene</b><span>The saved level (.umap): every actor, transform and material, as the engine holds it.</span></div><div class="step eval"><span class="n">5 · Evaluator</span><b>Scored in the engine</b><span>Requirements located and judged, overview views judged, physics measured, edits diffed against the withheld ground truth.</span></div></div>
63
  <div class="grid2" style="margin-top:2rem">
64
  <div class="card setting-card"><span class="kicker">Setting 1 · construction</span><h3>Text-to-Scene</h3>
 
92
  <div style="text-align:center;margin-top:2rem"><a class="btn btn-primary" href="cases.html"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="M8 1.5l5.5 3v7L8 14.5l-5.5-3v-7z"/><path d="M2.5 4.5L8 7.5l5.5-3M8 7.5v7"/></svg> Open the case explorer</a></div>
93
  </div></section><section class="dark" id="citation"><div class="container">
94
  <div class="section-header centered"><span class="section-label">Citation</span><h2 class="section-title">Cite This Work</h2></div>
95
+ <div class="cite-box"><button class="cite-copy-btn" id="copy-btn">Copy</button><pre id="bibtex-text">@misc{ye2026code4scene,
96
+ title = {Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes},
97
+ author = {Xiaokang Ye and Siddhant Hitesh Mantri and Zimeng Chen and Edward Zhang and Zhaoxu Zheng and Yuanheng Li and Yizhao Chen and Tianyang Huang and Lianhui Qin},
98
+ year = {2026},
99
+ eprint = {2609.36777},
100
+ archivePrefix = {arXiv},
101
+ primaryClass = {cs.AI},
102
+ url = {https://arxiv.org/abs/2609.36777},
103
  }</pre></div>
104
  <div style="display:flex;flex-wrap:wrap;gap:1rem;justify-content:center;margin-top:2rem">
105
+ <a class="btn btn-outline" href="https://arxiv.org/abs/2609.36777" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M4 0h5.5L14 4.5V14a2 2 0 01-2 2H4a2 2 0 01-2-2V2a2 2 0 012-2zm5 1.5V5h3.5L9 1.5zM5 8h6v1H5zm0 2.5h6v1H5z"/></svg> arXiv</a>
106
  <a class="btn btn-outline" href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> GitHub Repository</a>
107
  <a class="btn btn-outline" href="https://simworld.org/" target="_blank" rel="noopener">SimWorld Main Site</a></div>
108
  </div></section>
109
+ <footer><p>Code4Scene &nbsp;·&nbsp; UC San Diego &amp; UC Berkeley &nbsp;·&nbsp; <a href="https://arxiv.org/abs/2609.36777" target="_blank" rel="noopener">arXiv:2609.36777</a> &nbsp;·&nbsp; <a href="https://github.com/SimWorld-AI/Code4Scene" target="_blank" rel="noopener">GitHub</a> &nbsp;·&nbsp;
110
  <a href="https://simworld.org/" target="_blank" rel="noopener">SimWorld Home</a></p>
111
  <p style="margin-top:.5rem;font-size:.75rem;color:rgba(255,255,255,.25)">Part of the SimWorld project.</p></footer>
112
  <script src="assets/paper.js?v=5427505a"></script>