L(M)V-IQL: Multiple Intention Inverse Reinforcement Learning for Animal Behavior Characterization
Primary Area: applications to neuroscience & cognitive science
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Inverse Reinforcement Learning, Neuroscience, Decision-making
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: In the pursuit of comprehending decision-making, behavioral neuroscience has made significant progress, aided by mathematical models in recent years. Among various approaches, Inverse Reinforcement Learning (IRL) stands out as a promising technique, distinguishing itself from other paradigms through its ability to circumvent the necessity for a reward function in characterizing observed behavior. Nevertheless, the widespread adoption of IRL within the field of neuroscience remains limited. This constraint may be attributed, in part, to the prevailing assumption in many existing IRL frameworks that animals exhibit a singular intention throughout a given task, wherein their behavior is optimized based on a single static reward function. In an effort to overcome this limitation, we propose the class of Latent (Markov) Variable Inverse Q-learning (L(M)V-IQL) algorithms, a novel IRL framework designed to accommodate multiple discrete intrinsic rewards. We formulate an Expectation-Maximization approach to cluster observed trajectories into multiple intentions, and subsequently solve the IRL problem independently for each intention. We illustrate the application of L(M)V-IQL through simulated experiments, followed by its utilization on a dataset of mice engaged in a two-armed bandit task. Our methods exhibit exceptional proficiency in discerning animal intentions and yield interpretable reward functions corresponding to each identified intention. We anticipate that this progress will open up new possibilities in neuroscience and psychology, serving as an important advancement in elucidating the intricacies of animal decision-making and uncovering underlying brain mechanisms.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7249
Loading