Papers
arxiv:2509.23690

HomeSafeBench: A Benchmark for Embodied Vision-Language Models in Free-Exploration Home Safety Inspection

Published on Aug 4
Authors:
,
,
,
,
,

Abstract

A benchmark for embodied home hazard inspection reveals large gaps in vision-language models and improves detection via an offline backtracking supervision method.

Safety hazards in the home are a leading cause of preventable domestic injuries, motivating an automated inspector that actively explores a home and reports hazards before they cause harm. We introduce HomeSafeBench, the first benchmark for free-exploration home safety inspection with egocentric visual feedback, in which an embodied agent navigates a fully interactive 3D home, adjusts its viewpoint, and reports hazards purely from rendered first-person views. Built on the VirtualHome simulator, it covers five categories of common household hazards and comprises 1,000 human-validated inspection tasks. Evaluating a broad range of state-of-the-art Vision-Language Models (VLMs) reveals a large gap, where the best model reaches only about 34.7% F1, far below the 98.0% of a human inspector. Moreover, precision far exceeds recall across models, revealing a systematic tendency to under-report hazards that reflects a shared deficiency in risk recognition. To close this gap at low cost, we propose CueBack, an offline data-construction method that exploits the clue-precedes-confirmation structure of inspection, backtracking a privileged trajectory to the earliest frame where a hazard cue becomes visible and rewriting it into executable supervision. Fine-tuning a 4B-size VLM on CueBack-constructed data raises the average F1 from 18.7% to 45.3% on an out-of-distribution test set, surpassing the strongest closed-source model performance 34.7%. The benchmark, training dataset, and code are available at https://github.com/BITHLP/HomeSafeBench.

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2509.23690
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2509.23690 in a model README.md to link it from this page.

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2509.23690 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.