Combining Counting Processes and Classification Improves a Stopping Rule for Technology Assisted Review

Reem Bin Hezam, Mark Stevenson

Submission Type: Regular Short Paper

Submission Track: Information Retrieval and Text Mining

Submission Track 2: NLP Applications

Keywords: Technology assisted review, TAR, total recall, stopping criteria, counting processes, classification

TL;DR: Combining Counting Processes and Classification Improves a Stopping Rule for Technology Assisted Review

Abstract: Technology Assisted Review (TAR) stopping rules aim to reduce the cost of manually assessing documents for relevance by minimising the number of documents that need to be examined to ensure a desired level of recall. This paper extends an effective stopping rule using information derived from a text classifier that can be trained without the need for any additional annotation. Experiments on multiple data sets (CLEF e-Health, TREC Total Recall, TREC Legal and RCV1) showed that the proposed approach consistently improves performance and outperforms several alternative methods.

Submission Number: 797