Characterizing Learners’ Complex Attentional States During Online Multimedia Learning Using Eye-tracking, Egocentric Camera, Webcam, and Retrospective recalls

Prasanth Chandran, Yifeng Huang, Jeremy Munsell, Brian Howatt, Brayden Wallace, Lindsey Wilson, Sidney D'Mello, Minh Hoai, N. Sanjay Rebello, Lester Loschky

Abstract:

As online learning becomes increasingly ubiquitous, a key challenge
is maintaining learners’ sustained attention. Using eye-tracking, together
with observing and interviewing learners, we can characterize
both 1) whether they are looking at their learning materials, and
2) whether they are thinking about them. Critically, eye-tracking
only speaks to the first distinction, not the second. To overcome
this limitation, we supplemented eye-tracking with an egocentric
camera, a webcam, a retrospective recall, and mind-wandering
probes to capture a 2x2 matrix of attentional/cognitive states. We
then categorized N=101 learners’ attentional/cognitive states while
they completed a multimedia physics module. This meets two
goals: 1) allowing basic research to understand the relationship
between attentional/cognitive states and behavioral outcomes; and
2) facilitating applied research by generating rich ground truth for
future use in training machine learning to categorize this 2x2 set
of attentional states, for which eye-tracking is necessary, but not
sufficient.