Towards Detecting Tonic Information Processing Activities with Physiological Data

Kaixin Ji, Damiano Spina, Danula Hettiachchi, Falk Scholer, Flora D. Salim

Abstract: Characterizing Information Processing Activities (IPAs) such as reading, listening, speaking, and writing, with physiological signals captured by wearable sensors can broaden the understanding of how people produce and consume information. However, sensors are highly sensitive to external conditions that are not trivial to control – not even in lab user studies. We conducted a pilot study (N = 7) to assess the robustness and sensitivity of physiological signals across four IPAs (READ, LISTEN, SPEAK, and WRITE) using multiple sensors. The collected signals include Electrodermal Activities, Blood Volume Pulse, gaze, and head motion. We observed consistent trends across participants, and ten features with statistically significant differences across the four IPAs. Our results provide preliminary quantitative evidence of differences in physiological responses when users encounter IPAs, revealing the necessity to inspect the signals separately according to the IPAs. The next step of this study moves into a specific context, information retrieval, and the IPAs are considered as the interaction modalities with the search system, for instance, submitting the search query by speaking or typing.