Scaffolding Human Learning by Shaping Visual Environment

Yuji Zhang, Duo Zhou, Bo Chen, Adi Chalasani, Noah Schroeder, H Chad Lane, ChengXiang Zhai

Keywords: Visual environment shaping; Human learning support; AI4Education

TL;DR: We propose a new paradigm of proactively shaping environment to guide human learning.

Abstract: Learning is an interactive process in which timely guidance plays a crucial role in helping learners overcome difficulties. In now ubiquitous online learning environments, however, such guidance is often unavailable or requires learners to explicitly request help, interrupting their cognitive flow. In our work, we propose a novel paradigm, environment shaping, which proactively supports learners by adaptively modifying the learning environment based on learner behavior. We formulate this problem as a Markov decision process (MDP), where learner states are inferred from multimodal behavioral signals and actions correspond to environment-level interventions such as adaptive visual highlighting. We implement the MDP framework in a block-based coding environment (VEX VR) using vision-language models (VLM) to estimate learner engagement and guide learning. To enable controlled and reproducible evaluation, we further introduce a simulation framework with VLM-simulated agent learners. Preliminary experiments with simulated agents show that environment shaping significantly improves early-stage learning performance and guides learners toward more productive trajectories. These results demonstrate the great promise of proactive environment shaping as a scalable complement to traditional teacher assistance.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Track: Proceeding Track

Submission Number: 25