Geometry-Infused Latent Models for RGB-Only Visual Goal Planning

Xiaoxiang Li

Abstract: Latent forward models are increasingly used for offline visual planning, yet their training objectives often neglect geometric realism, which is critical for accurate goal-conditioned control in 3D environments. Conventional RGB-only approaches rely on latent prediction and terminal-distance minimization, leaving room for geometrically inconsistent imagined rollouts. We present Geo-FLM, a geometry-infused latent world modeling approach that enhances standard RGB-only frameworks with geometry-aware supervision and trajectory feasibility evaluation. Using pseudo-depth only during training, we align encoded observations and predicted latents with geometric priors. Moreover, we learn a latent transition feasibility score to discourage implausible imagined states during planning. The resulting planner enforces latent goal reaching, geometric endpoint alignment, and feasible trajectories, all under standard RGB-only evaluation. Initial results indicate that Geo-FLM improves the robustness and accuracy of latent-based visual planning, offering a flexible and backbone-agnostic path toward geometry-aware control.