Semi-Supervised Contrastive Training for Similar Image Identification in a Large Collection of Historical Books

Ruilin Wang, Lidia Pivovarova, Yann Ciarán Ryan, Mikko Tolonen

Abstract: Historical books convey a lot of information with visual clues, such as illustrations and ornaments. While most applications of computational techniques to this material are concentrated on text recognition, we deal with a novel application, i.e. grouping book decorations according to their similarity. This reveals images that were potentially created using the same physical tools, which hints to connection between the book creators. This paper focuses on headpieces, i.e. ornaments used on the top of the page at the beginning of a chapter, extracted from ECCO, a large collection of 18th century books. We group images using a combination of supervised and self-supervised contrastive representation learning, thus revealing images that might be created from the same woodblock. We adjust the image augmentation pipeline and propose a novel semi-supervised fine-tuning procedure. The resulting semi-supervised model yields 94% precision. Ablation studies demonstrate that both contributions are crucial to obtain high-quality clustering results. Finally, the paper shows the practical usefulness of this approach for historical research: for some previously studied images we found by an order of magnitude more exemplars that were known before, thus challenging the previous practice to assume a hard one-to-one connection between an image and a publisher.