Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Modulating Individual Human Percepts

Chen Wei, Chi Zhang, Jiachen Zou, Haotian Deng, Dietmar Heinke, Quanying Liu

Keywords: Perceptual variability, Diffusion model, Object Recognition, Behavior Modulation, Behavioral Alignment

TL;DR: By sampling along neural network perceptual boundaries, we generated natural images that induce high variability in human decision and can predict and modulate individual behavior on these samples.

Abstract: Human decision-making in cognitive tasks and daily life exhibits considerable variability, shaped by factors such as task difficulty, individual preferences, and personal experiences. Understanding this variability across individuals is essential for uncovering the perceptual and decision-making mechanisms that humans rely on when faced with uncertainty and ambiguity. In this study, we present a counterfactual-based approach to investigate the subject-level decision-making behaviors and reveal the underlying perceptual mechanisms by synthesizing visual stimuli. First, we developed an efficient generative model that samples along an artificial neural network (ANN)’s perceptual boundary, generating image samples designed to induce high variability in human perception. Using these generated samples, combined with behavioral data from 246 human participants across 116,715 trials, we constructed the varMNIST dataset. Then, we presented a subject-specific fine-tuning approach to align the perceptual variability of ANNs with that of humans. It allows us to successfully predict human decision-making behaviors on varMNIST. Finally, we verified the ability to selectively manipulate individual behaviors by generating tailored controversial stimuli, which highlighted significant inter-subject perceptual variability. Together, our work illuminated key distinctions between human and machine perceptual variability and established an effective strategy for manipulating individual decision-making behaviors. This study paves the way for artificial intelligence models with personalized perceptual capabilities.

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 12066