Modeling Optimal Switching Behavior

Mark D. Smucker, Charles L. A. Clarke

Abstract: Recently developed retrieval effectiveness measures have incorporated models of user behavior, but have limited themselves to predicting user performance over a single query and response. Accurate prediction of user performance with search systems must incorporate a means to model how users switch between different information sources. For example, a search session may consist of multiple queries with the user making decisions of when to switch from evaluating the current result list to a new result list produced by a query reformulation. Likewise, users may switch to a result list produced by a query suggestion or other interaction mechanism that produces a new search result list. In this paper, we simulate user behavior and investigate optimal switching behavior for a user who must decide when and if to issue their current query to another search engine. As a first step in understanding the problem space, we restrict our investigation and discussion to two top performing runs submitted to the TREC 2005 Robust track. We find four classes of switching behavior that a user would be faced with in making a decision about whether to switch from one result list to another.