ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding

ACL ARR 2026 January Submission5291 Authors

Keywords: LLM, MLLM

Abstract:

While Large Language Models (LLMs) excel at algorithmic code generation, they struggle with front-end development, where correctness is judged on rendered pixels and interaction. We present ReLook, an agentic, vision-grounded reinforcement learning framework that empowers an agent to close a robust generate--diagnose--refine loop by invoking a multimodal LLM (MLLM) as a tool.
During training, the agent employs an MLLM-in-the-loop to serve as a visual critic, evaluating code via screenshots and providing actionable feedback. Crucially, we enforce a strict zero-reward policy for invalid renders to guarantee renderability and mitigate reward hacking. To prevent behavioral collapse, we introduce Forced Optimization, a strict acceptance rule that admits only improving revisions, yielding monotonically better trajectories. At inference, we decouple the critic and run a lightweight, critic-free self-edit cycle, keeping latency comparable to base decoding while retaining most of the gains.
Across three widely used benchmarks, ReLook consistently outperforms strong baselines in vision-grounded front-end code generation, highlighting the benefits of agentic perception, visual rewards, and training–inference decoupling.

Paper Type: Long

Research Area: AI/LLM Agents

Research Area Keywords: NLP Applications

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Reproduction study

Languages Studied: English

Submission Number: 5291