Exploring search task difficulty reasons in different task types and user knowledge groups

Jingjing Liu

Abstract:

Search task difficulty has been attracting much research attention in recent years, mostly
regarding its relationship with searchers’ behaviors and the prediction of task difficulty
from search behaviors. However, it remains unknown what makes searchers feel the
difficulty. A study consisting of 48 undergraduate students was conducted to explore this
question. Each participant was given 4 search tasks that were carefully designed following
a task classification scheme. Questionnaires were used to elicit participants’ ratings on task
difficulty and why they gave those ratings. Based on the collected difficulty reasons, a
coding scheme was developed, which covered various aspects of task, user, and user–task
interaction. Difficulty reasons were then categorized following this scheme. Results
showed that searchers reported some common reasons leading to task difficulty in different
tasks, but most of the difficulty reasons varied across tasks. In addition, task difficulty
had some common reasons between searchers with low and high levels of topic knowledge,
although there were also differences in top task difficulty reasons between high
and low knowledge users. These findings further our understanding of search task
difficulty, the relationship between task difficulty and task type, and that between task
difficulty and knowledge level. The findings can also be helpful with designing tasks for
information search experiments, and have implications on search system design both in
general and for personalization based on task type and searchers’ knowledge.