Improving searcher struggle detection via the reversal theory

Jiyun Luo, Yan Yang, Valerie Nayak, Grace Hui Yang

Abstract: Searcher struggle is important feedback to Web search engines. Existing Web search struggle detection methods rely on effort-based features to identify the struggling moments. Their underlying assumption is that the more effort a user spends, the more struggling the user may be. However, studies have shown that this simple association might be incorrect. This paper proposes a new feature modulation method for struggle detection and refers to the Reversal Theory in psychology. Reversal Theory points out that instead of having a static personality trait, people constantly switch between opposite psychological states, complicating the relationship between the efforts they spend and the level of frustration they feel. Supported by the theory, our method modulates the effort-based features based on Reversal Theory’s bi-modal arousal model. After modification, the users’ effort level is better aligned with their struggling experience. Evaluations on Pinterest search logs confirm that the proposed method can statistically significantly improve searcher struggle detection methods.