PaintTeR : Automatic Extraction of Text Spans for Generating Art-Centered Questions

Sujatha Das Gollapalli, See-Kiong Ng, Ying Kiat Tham, Shan Shan Chow, Jia Min Wong, Kevin Lim

Abstract:

We propose PaintTeR, our Paintings TextRank algorithm for extracting art-related text spans from passages on paintings.
PaintTeR combines a lexicon of painting words curated automatically through distant supervision with random walks
on a large-scale word co-occurrence graph for ranking passage spans for artistic characteristics. The spans extracted
with PaintTeR are used in state-of-the-art Question Generation and Reading Comprehension models for designing an
interactive aid that enables gallery and museum visitors focus on the artistic elements of paintings. We provide
experiments on two datasets of expert-written passages on paintings to showcase the effectiveness of PaintTeR. Evaluations by
both gallery experts as well as crowdworkers indicate that our proposed algorithm can be used to select relevant
and interesting art-centered questions. To the best of our knowledge, ours is the first work to effectively fine-tune question
generation models using minimal supervision for a low-resource, specialized context such as gallery visits.