User-SERP Interaction Prediction through Deep Multi-task Learning

Wei Jiang, Damien Jose, Gargi Ghosh

Abstract: User behavior signals such as clicks are strong indicators of a search engine’s performance. Many existing search algorithms focus on predicting user’s interactions, by optimizing a relevance cost function for the query and individual web documents. The result set (list) is then generated by ranking web documents with this score. However, the probability of user interaction with a web document on a Search Engine Result Page (SERP) depends not only on a web document in isolation, but also other documents/elements present on the SERP. Our approach better predicts user interactions on web documents by not only considering the relevance of individual documents for a query, but also their interdependencies by modeling the interactions of a User on a SERP with a Multi-task Bidirectional Recurrent Neural Network (RNN)

TL;DR: Using a novel Deep Multi-task Bi-Directional RNN model we can better predict User-SERP interactions and hence improve relevance of both individual and entire list of documents shown to Users

Keywords: Deep Multi-task Learning, Search Engine Relevance