SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis

Kathakoli Sengupta, Kai Ao, Paola Cascante-Bonilla

Keywords: Evaluation, Scalable 3D scenes, Ontology Dataset, Vision Language Models

TL;DR: A symbolic evaluator for floor-plan-level layouts grounded in SceneOnto dataset.

Abstract: Scaling spatial intelligence in embodied agents demands environments that capture rich compositional structure and precise spatial relationships and can support such diverse task requirements. Large Language Models and Vision-Language Models increasingly generate indoor scenes via intermediate structures like layouts and scene graphs, yet evaluation still relies on LLM or VLM judges whose scores are sensitive to viewpoint, prompt phrasing, and hallucination. This instability makes it hard to disentangle spatial plausibility from evaluation artifacts. We introduce SceneCritic, a symbolic evaluator for floor-plan-level layouts grounded in SceneOnto, a structured spatial ontology aggregated from 3D-FRONT, ScanNet, and Visual Genome. SceneCritic jointly verifies semantic, orientation, and geometric coherence across object relationships, providing object- and relationship-level assessments that pinpoint specific violations. We further propose an iterative refinement testbed that probes how models revise spatial structure under three critic modalities: a rule-based collision critic, an LLM critic operating on layout text, and a VLM critic operating on rendered observations. Extensive experiments show that (a) SceneCritic aligns substantially better with human judgments than VLM-based evaluators, (b) text-only LLMs can outperform VLMs on semantic layout quality, and (c) image-based VLM refinement is most effective for semantic and orientation correction.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 20