mobileworldmodel / index.html
xwk123's picture
Add MobileWorldModel static project page
5b67f9c verified
Raw
History Blame Contribute Delete
24 kB
<!DOCTYPE html>
<html lang="zh-CN">
<!-- Assets use Hugging Face resolve/main URLs. Do not put HF tokens in static files. -->
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>How Mobile World Model Guides GUI Agents?</title>
<meta name="description" content="Project page for Mobile-GUI-Worldmodel-SFT and MobileWorldModel checkpoints." />
<style>
:root {
--ink: #18212f;
--muted: #607083;
--paper: #ffffff;
--wash: #f5f7fb;
--line: #dfe6ef;
--blue: #4c78a8;
--teal: #2f7f7f;
--gold: #e69f00;
--purple: #7a68a6;
--green: #2e7d32;
--shadow: 0 18px 55px rgba(20, 30, 50, .10);
--soft-shadow: 0 10px 30px rgba(20, 30, 50, .08);
--radius: 22px;
}
* { box-sizing: border-box; }
html { scroll-behavior: smooth; }
body {
margin: 0;
color: var(--ink);
background:
radial-gradient(circle at top left, rgba(76, 120, 168, .16), transparent 32rem),
radial-gradient(circle at top right, rgba(47, 127, 127, .15), transparent 34rem),
linear-gradient(180deg, #fbfcff 0%, #f4f7fb 46%, #eef3f8 100%);
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
line-height: 1.65;
}
a { color: inherit; }
.container { width: min(1180px, calc(100% - 40px)); margin: 0 auto; }
.nav {
position: sticky;
top: 0;
z-index: 20;
backdrop-filter: blur(16px);
background: rgba(250, 252, 255, .78);
border-bottom: 1px solid rgba(223, 230, 239, .85);
}
.nav-inner {
height: 62px;
display: flex;
align-items: center;
justify-content: space-between;
gap: 18px;
font-size: 14px;
font-weight: 650;
}
.brand { display: flex; align-items: center; gap: 10px; letter-spacing: -.01em; }
.brand-mark {
width: 30px; height: 30px;
border-radius: 10px;
background: linear-gradient(135deg, var(--blue), var(--teal));
box-shadow: 0 8px 18px rgba(76, 120, 168, .25);
}
.nav-links { display: flex; gap: 18px; color: #4a5a6d; }
.nav-links a { text-decoration: none; }
.nav-links a:hover { color: var(--blue); }
.hero { padding: 70px 0 38px; }
.hero-grid {
display: grid;
grid-template-columns: minmax(0, 1fr);
gap: 32px;
align-items: center;
max-width: 980px;
margin: 0 auto;
text-align: center;
}
.eyebrow {
display: inline-flex;
align-items: center;
gap: 8px;
padding: 7px 12px;
border: 1px solid rgba(76, 120, 168, .25);
border-radius: 999px;
background: rgba(255, 255, 255, .74);
color: var(--blue);
font-size: 13px;
font-weight: 750;
letter-spacing: .02em;
}
h1 {
margin: 18px 0 14px;
font-family: Georgia, "Times New Roman", serif;
font-size: clamp(42px, 6vw, 72px);
line-height: .98;
letter-spacing: -.055em;
}
.subtitle {
max-width: 760px;
color: #405064;
font-size: clamp(17px, 2vw, 20px);
margin: 0 auto 22px;
}
.actions { display: flex; flex-wrap: wrap; justify-content: center; gap: 12px; margin: 24px 0 0; }
.btn {
display: inline-flex;
align-items: center;
justify-content: center;
gap: 8px;
padding: 12px 17px;
border-radius: 999px;
border: 1px solid var(--line);
background: rgba(255, 255, 255, .72);
text-decoration: none;
font-weight: 750;
box-shadow: 0 8px 20px rgba(20, 30, 50, .06);
}
.btn.primary {
color: #fff;
border-color: transparent;
background: linear-gradient(135deg, var(--blue), var(--teal));
}
.resource-grid {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 12px;
margin-top: 26px;
}
.resource-card {
padding: 15px;
border: 1px solid rgba(223, 230, 239, .95);
border-radius: 18px;
background: rgba(255, 255, 255, .82);
}
.resource-card .label { color: var(--muted); font-size: 12px; font-weight: 800; text-transform: uppercase; letter-spacing: .06em; }
.resource-card .value { margin-top: 6px; font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace; font-size: 13px; word-break: break-all; }
.authors-card {
margin: 18px auto 0;
padding: 22px 24px;
border: 1px solid rgba(223, 230, 239, .95);
border-radius: 22px;
background: rgba(255, 255, 255, .82);
color: #35455a;
font-size: 18px;
box-shadow: 0 8px 20px rgba(20, 30, 50, .05);
text-align: center;
}
.authors-card strong {
color: var(--ink);
font-weight: 800;
}
.authors-list {
margin: 8px auto 14px;
line-height: 1.6;
max-width: 880px;
font-size: clamp(18px, 2.1vw, 22px);
color: #243145;
}
.affiliations {
margin: 12px auto 0;
display: grid;
gap: 6px;
color: var(--muted);
font-size: clamp(15px, 1.6vw, 18px);
max-width: 900px;
}
.hero-figure {
padding: 16px;
border-radius: 28px;
background: rgba(255, 255, 255, .86);
border: 1px solid rgba(223, 230, 239, .9);
box-shadow: var(--shadow);
}
.hero-figure img {
width: 100%;
display: block;
border-radius: 18px;
border: 1px solid var(--line);
background: #fff;
}
.caption { color: var(--muted); font-size: 13px; margin: 10px 4px 0; }
section { padding: 42px 0; }
.section-title {
display: flex;
align-items: end;
justify-content: space-between;
gap: 18px;
margin-bottom: 18px;
}
h2 {
margin: 0;
font-size: clamp(26px, 3vw, 36px);
letter-spacing: -.035em;
line-height: 1.08;
}
.section-kicker {
color: var(--muted);
font-size: 14px;
font-weight: 750;
letter-spacing: .06em;
text-transform: uppercase;
}
.panel {
border: 1px solid rgba(223, 230, 239, .95);
border-radius: var(--radius);
background: rgba(255, 255, 255, .86);
box-shadow: var(--soft-shadow);
overflow: hidden;
}
.panel.pad { padding: 24px; }
.abstract {
display: grid;
grid-template-columns: minmax(0, 1.2fr) minmax(280px, .8fr);
gap: 24px;
}
.abstract p { margin: 0 0 14px; color: #344255; }
.takeaways { display: grid; gap: 12px; }
.mini {
padding: 15px;
border-radius: 16px;
border: 1px solid var(--line);
background: linear-gradient(180deg, #fff, #f9fbff);
}
.mini strong { display: block; margin-bottom: 5px; }
.mini span { color: var(--muted); font-size: 14px; }
.fig-grid {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 18px;
}
.fig-card {
border: 1px solid var(--line);
border-radius: 20px;
background: #fff;
overflow: hidden;
box-shadow: 0 10px 24px rgba(20, 30, 50, .06);
}
.fig-card.wide { grid-column: 1 / -1; }
.fig-body {
min-height: 300px;
background: #f8fafc;
display: grid;
place-items: center;
border-bottom: 1px solid var(--line);
}
.fig-body.short { min-height: 250px; }
.fig-body.tall { min-height: 430px; }
.fig-body img { width: 100%; height: auto; display: block; background: #fff; }
.fig-meta { padding: 13px 15px 15px; }
.fig-meta h3 { margin: 0 0 4px; font-size: 16px; letter-spacing: -.01em; }
.fig-meta p { margin: 0; color: var(--muted); font-size: 13px; line-height: 1.5; }
.open-link {
display: inline-flex;
margin-top: 9px;
color: var(--blue);
font-size: 13px;
font-weight: 800;
text-decoration: none;
}
.findings { display: grid; gap: 22px; }
.finding {
display: grid;
grid-template-columns: 330px minmax(0, 1fr);
gap: 18px;
align-items: start;
}
.finding-copy {
padding: 22px;
border-radius: var(--radius);
background: #fff;
border: 1px solid var(--line);
box-shadow: var(--soft-shadow);
position: sticky;
top: 86px;
}
.badge {
display: inline-flex;
padding: 5px 9px;
border-radius: 999px;
color: #fff;
font-size: 12px;
font-weight: 900;
letter-spacing: .06em;
}
.badge.blue { background: var(--blue); }
.badge.gold { background: var(--gold); color: #2a1a00; }
.badge.purple { background: var(--purple); }
.finding h3 { margin: 13px 0 9px; font-size: 22px; line-height: 1.16; }
.finding p { margin: 0; color: #3d4c5e; font-size: 15px; }
.figure-stack { display: grid; gap: 14px; }
.small-grid {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 14px;
}
.case-grid {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 14px;
}
.case-card {
border: 1px solid var(--line);
border-radius: 18px;
background: #fff;
overflow: hidden;
box-shadow: 0 8px 18px rgba(20, 30, 50, .06);
}
.case-card img { width: 100%; display: block; }
.case-card p { margin: 0; padding: 10px 12px; color: var(--muted); font-size: 13px; }
footer {
padding: 38px 0 50px;
color: var(--muted);
font-size: 13px;
text-align: center;
}
@media (max-width: 980px) {
.hero-grid, .abstract, .finding { grid-template-columns: 1fr; }
.finding-copy { position: static; }
.fig-grid, .small-grid { grid-template-columns: 1fr; }
.case-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.nav-links { display: none; }
}
@media (max-width: 560px) {
.container { width: min(100% - 24px, 1180px); }
.resource-grid, .case-grid { grid-template-columns: 1fr; }
.hero { padding-top: 42px; }
}
</style>
</head>
<body>
<nav class="nav">
<div class="container nav-inner">
<div class="brand"><span class="brand-mark"></span><span>MobileWorldModel</span></div>
<div class="nav-links">
<a href="#abstract">Abstract</a>
<a href="#figures">Figures</a>
<a href="#findings">Findings</a>
<a href="#cases">Cases</a>
</div>
</div>
</nav>
<header class="hero">
<div class="container hero-grid">
<div>
<div class="eyebrow">Dataset + Models for Mobile GUI World Modeling</div>
<h1>How Mobile World Model Guides GUI Agents?</h1>
<p class="subtitle">
A unified study of text, image, and renderable-code world models for mobile GUI agents:
what they should predict, how they help at test time, and when imagined trajectories transfer to policy training.
</p>
<div class="actions">
<a class="btn primary" href="https://huggingface.co/datasets/xwk123/Mobile-GUI-Worldmodel-SFT">Open Dataset</a>
<a class="btn" href="https://hf.co/collections/xwk123/mobileworldmodel">Model Collection</a>
</div>
<div class="resource-grid">
<div class="resource-card">
<div class="label">Dataset</div>
<div class="value">xwk123/Mobile-GUI-Worldmodel-SFT</div>
</div>
<div class="resource-card">
<div class="label">Models</div>
<div class="value">hf.co/collections/xwk123/mobileworldmodel</div>
</div>
</div>
<div class="authors-card">
<strong>Authors</strong>
<div class="authors-list">
Weikai Xu<sup>1*</sup>, Kun Huang<sup>2*</sup>, Yunren Feng<sup>3*</sup>, Jiaxing Li<sup>1</sup>,
Yuhan Chen<sup>1</sup>, Yuxuan Liu<sup>4</sup>, Zhizheng Jiang<sup>3</sup>, Heng Qu<sup>5</sup>,
Pengzhi Gao<sup>2</sup>, Wei Liu<sup>2</sup>, Jian Luan<sup>2</sup>, Xiaolin Hu<sup>6</sup>,
Bo An<sup>1†</sup>
</div>
<div class="affiliations">
<span><strong>1</strong> Nanyang Technological University</span>
<span><strong>2</strong> MiLM Plus, Xiaomi Inc.</span>
<span><strong>3</strong> University of Electronic Science and Technology of China</span>
<span><strong>4</strong> Gaoling School of Artificial Intelligence, Renmin University of China</span>
<span><strong>5</strong> Wuhan University</span>
<span><strong>6</strong> Xiamen University</span>
</div>
</div>
</div>
</div>
</header>
<main>
<section id="abstract">
<div class="container">
<div class="section-title">
<div>
<div class="section-kicker">Paper Summary</div>
<h2>Abstract and contribution</h2>
</div>
</div>
<div class="panel pad abstract">
<div>
<p>
Recent advances in vision-language models have enabled mobile GUI agents to perceive visual interfaces and execute
user instructions, but reliable prediction of action consequences remains critical for long-horizon and high-risk interactions.
Existing mobile world models provide either text-based or image-based future states, yet it remains unclear which
representation is useful, whether generated rollouts can replace real environments, and how test-time guidance helps agents of different strengths.
</p>
<p>
We filter and annotate mobile world-model data, then train world models across four modalities: delta text, full text,
diffusion-based images, and renderable code. We evaluate their downstream utility on AITZ, AndroidControl, and AndroidWorld.
</p>
</div>
<div class="takeaways">
<div class="mini"><strong>Finding 1</strong><span>Renderable code is strong in-distribution; text feedback is more robust for online OOD execution.</span></div>
<div class="mini"><strong>Finding 2</strong><span>Posterior self-reflection is limited by overconfident, low-entropy action policies.</span></div>
<div class="mini"><strong>Finding 3</strong><span>World-model imagination can transfer interaction experience, but does not preserve the source distribution.</span></div>
</div>
</div>
</div>
</section>
<section id="figures">
<div class="container">
<div class="section-title">
<div>
<div class="section-kicker">Intro Figures</div>
<h2>World-model formats and headline results</h2>
</div>
</div>
<div class="fig-grid">
<article class="fig-card wide">
<div class="fig-body"><img src="assets/overview.png" alt="Overview" /></div>
<div class="fig-meta">
<h3>Figure 1 · Overview</h3>
<p>Empirical map from prediction formats to test-time guidance and imagination-based fine-tuning.</p>
</div>
</article>
<article class="fig-card">
<div class="fig-body tall"><img src="assets/wm_input_output.png" alt="Text vs image world model comparison" /></div>
<div class="fig-meta">
<h3>Figure 2 · Text vs image world models</h3>
<p>Input-output comparison for GUI state prediction.</p>
<a class="open-link" href="assets/WM_input_output.pdf">Open PDF</a>
</div>
</article>
<article class="fig-card">
<div class="fig-body tall"><img src="assets/wm_exp_bar.png" alt="World-modeling paradigm comparison" /></div>
<div class="fig-meta">
<h3>World-modeling paradigms</h3>
<p>Four prediction settings compared across judge models and MobileWorldModel.</p>
<a class="open-link" href="assets/wm_exp_bar.pdf">Open PDF</a>
</div>
</article>
</div>
</div>
</section>
<section id="findings">
<div class="container">
<div class="section-title">
<div>
<div class="section-kicker">Three Experimental Conclusions</div>
<h2>What the experiments show</h2>
</div>
</div>
<div class="findings">
<div class="finding">
<aside class="finding-copy">
<span class="badge blue">RQ1</span>
<h3>Choose the prediction format by deployment setting.</h3>
<p>Code2Image provides strong in-distribution reconstruction and visual guidance for capable agents, while text feedback is more reliable for online OOD execution. Diffusion images are visually rich but costly and brittle on text rendering.</p>
</aside>
<div class="figure-stack">
<div class="small-grid">
<article class="fig-card">
<div class="fig-body short"><img src="assets/androidworld_overall_sr.png" alt="AndroidWorld overall SR" /></div>
<div class="fig-meta"><h3>AndroidWorld SR</h3><p>Online M3A success rate under text/image feedback.</p></div>
</article>
<article class="fig-card">
<div class="fig-body short"><img src="assets/offline_task_radar_6d.png" alt="Offline radar" /></div>
<div class="fig-meta"><h3>Offline navigation radar</h3><p>Six-dimensional summary of action-selection performance.</p></div>
</article>
</div>
<article class="fig-card">
<div class="fig-body"><img src="assets/wm_exp_bar.png" alt="World-modeling paradigm comparison" /></div>
<div class="fig-meta"><h3>Prediction quality by format</h3><p>Reconstruction quality across Full Text, Delta Text, Diffusion Image, and Code2Image.</p></div>
</article>
</div>
</div>
<div class="finding">
<aside class="finding-copy">
<span class="badge gold">RQ2</span>
<h3>Test-time world-model guidance is bounded by action entropy.</h3>
<p>Posterior selection cannot help if a small or overconfident policy proposes low-diversity candidates. The world model works best when it exposes useful future-state signals before the agent commits, or when sampling budget increases diversity.</p>
</aside>
<div class="figure-stack">
<div class="small-grid">
<article class="fig-card">
<div class="fig-body short"><img src="assets/entropy_accuracy_main_panel.png" alt="Entropy accuracy" /></div>
<div class="fig-meta"><h3>Entropy vs accuracy</h3><p>Entropy-conditioned behavior across GUI and non-GUI settings.</p></div>
</article>
<article class="fig-card">
<div class="fig-body short"><img src="assets/self_reflection_change_main_panel.png" alt="Self reflection change" /></div>
<div class="fig-meta"><h3>Reflection change rate</h3><p>Higher entropy enables more meaningful action revision.</p></div>
</article>
</div>
<article class="fig-card">
<div class="fig-body"><img src="assets/test_time_scaling_gui.png" alt="Test-time scaling" /></div>
<div class="fig-meta"><h3>Test-time scaling</h3><p>Scaling trends on AITZ, AndroidControl, and GUI-Odyssey.</p></div>
</article>
</div>
</div>
<div class="finding">
<aside class="finding-copy">
<span class="badge purple">RQ3</span>
<h3>Imagined trajectories transfer, but not uniformly.</h3>
<p>Training on world-model imagination may hurt source-like AndroidControl metrics while improving AndroidWorld SR. The useful signal is transferable interaction experience rather than faithful source-distribution preservation.</p>
</aside>
<div class="figure-stack">
<div class="small-grid">
<article class="fig-card">
<div class="fig-body"><img src="assets/sft_offline.png" alt="Offline WM-SFT results" /></div>
<div class="fig-meta"><h3>Offline WM-SFT</h3><p>Training behavior under imagined trajectories.</p></div>
</article>
<article class="fig-card">
<div class="fig-body"><img src="assets/sft_online.png" alt="Online WM-SFT results" /></div>
<div class="fig-meta"><h3>Online evaluation</h3><p>AndroidWorld gains vs AndroidControl distribution shift.</p></div>
</article>
</div>
<article class="fig-card">
<div class="fig-body"><img src="assets/sft_click_position_dist.png" alt="Click position distribution statistics" /></div>
<div class="fig-meta"><h3>Click distribution</h3><p>Behavioral changes induced by WM-generated interaction traces.</p></div>
</article>
</div>
</div>
</div>
</div>
</section>
<section id="cases">
<div class="container">
<div class="section-title">
<div>
<div class="section-kicker">Qualitative Cases</div>
<h2>Where the world model helps and fails</h2>
</div>
</div>
<div class="fig-grid">
<article class="fig-card">
<div class="fig-body tall"><img src="assets/aitz_html_casestudy.png" alt="AITZ HTML world-model feedback case" /></div>
<div class="fig-meta"><h3>HTML world-model feedback</h3><p>AITZ downstream case study.</p><a class="open-link" href="assets/aitz_html_casestudy.pdf">Open PDF</a></div>
</article>
<article class="fig-card">
<div class="fig-body tall"><img src="assets/androidcontrol_delta_text_casestudy.png" alt="AndroidControl delta-text case study" /></div>
<div class="fig-meta"><h3>Delta-text feedback</h3><p>AndroidControl downstream example.</p><a class="open-link" href="assets/androidcontrol_delta_text_casestudy.pdf">Open PDF</a></div>
</article>
<article class="fig-card">
<div class="fig-body tall"><img src="assets/diffusionmodel_casestudy1.png" alt="Diffusion image case study" /></div>
<div class="fig-meta"><h3>Diffusion image case</h3><p>Visual prediction can be expressive but error-prone.</p><a class="open-link" href="assets/diffusionmodel_casestudy1.pdf">Open PDF</a></div>
</article>
<article class="fig-card">
<div class="fig-body tall"><img src="assets/code_case1.png" alt="Code2Image case study" /></div>
<div class="fig-meta"><h3>Code2Image case</h3><p>Renderable-code prediction and screenshot reconstruction.</p><a class="open-link" href="assets/code_case1.pdf">Open PDF</a></div>
</article>
</div>
<div style="height:22px"></div>
<div class="case-grid">
<div class="case-card"><img src="assets/real_stopwatch.png" alt="Real stopwatch" /><p>Stopwatch · real UI</p></div>
<div class="case-card"><img src="assets/render_stopwatch.png" alt="Rendered stopwatch" /><p>Stopwatch · imagined render</p></div>
<div class="case-card"><img src="assets/real_account.png" alt="Real account" /><p>Account · real UI</p></div>
<div class="case-card"><img src="assets/render_account.png" alt="Rendered account" /><p>Account · imagined render</p></div>
</div>
</div>
</section>
</main>
<footer>
<div class="container">
Built from <code>mobileworld.tex</code>. Assets are served from the Hugging Face dataset repository.
Do not place Hugging Face access tokens in static HTML.
</div>
</footer>
</body>
</html>