Augmenting Human Cognition with an AI-Mediated Intelligent Visual Feedback

Songlin Xu, Xinyu Zhang

Abstract: In this paper, we introduce an AI-mediated framework that can pro- vide intelligent feedback to augment human cognition. Specifically, we leverage deep reinforcement learning (DRL) to provide adaptive time pressure feedback to improve user performance in a math arithmetic task. Time pressure feedback could either improve or de- teriorate user performance by regulating user attention and anxiety. Adaptive time pressure feedback controlled by a DRL policy ac- cording to users’ real-time performance could potentially solve this trade-off problem. However, the DRL training and hyperparameter tuning may require large amounts of data and iterative user studies. Therefore, we propose a dual-DRL framework that trains a regula- tion DRL agent to regulate user performance by interacting with another simulation DRL agent that mimics user cognition behaviors from an existing dataset. Our user study demonstrates the feasibil- ity and effectiveness of the dual-DRL framework in augmenting user performance, in comparison to the baseline group.