Analyzing Brain Activity and User Experience Across Input Modalities Using Quantitative Ethnography

Shayla Sharmin, Behdokht Kiafar, Roghayeh Leila Barmaki

Abstract: This study investigates how brain activity and user experience differ when using a hand vs. a stylus for touch display interactions in a computer-based educational game. We used functional near-infrared spectroscopy to measure brain activity from the prefrontal cortex during the gameplay. We also collected self-reported user experience via questionnaires. Sixteen participants completed quiz tasks using both input methods. Epistemic network analysis showed differences in how brain activity and user experience patterns are linked together for each input method. Additionally, we conducted a delta-based correlation and network analysis, inspired by the principles of chronologically ordered representations of discourse and tool-related activity, to investigate how participant-level changes in brain and self-reported behavioral measures aligned across hand and stylus conditions. The result showed that changes in brain activity weakly align with subjective experience. Results also showed significantly higher brain activity with the stylus, which suggests increased cognitive demand, but users found the hand method easier to use, performed better, and felt less frustrated than with the stylus. These findings indicate that input modality has an impact on brain behavior and user experiences while using educational tools.

External IDs:doi:10.1007/978-3-032-12229-2_26