2nd Workshop on Augmented Intelligence in Technology-Assisted Review Systems (ALTARS)

Giorgio Maria Di Nunzio, Evangelos Kanoulas, Prasenjit Majumder

Abstract: In this second edition of the workshop on Augmented Intelligence in Technology-Assisted Review Systems (ALTARS), we focus on the evaluation of High-recall Information Retrieval (IR) systems which tackle challenging tasks that require the finding of (nearly) all the relevant documents in a collection. In fact, despite the number of evaluation measures at our disposal to assess the effectiveness of a “traditional” retrieval approach, there are additional dimensions of evaluation for these systems. During the workshop, the organizers as well as the participants will discuss these issues and prepare a set of guidelines for the preparation of a correct evaluation of these kinds of systems.