MFSC: Matching by Few-Shot Classification

Roee Litman, Simon Korman, Daniel Shalam, Elie Abboud

Abstract:

The ability to accurately and efficiently match between sets of items has always been
fundamental in computer vision pipelines and applications with a wide variety of realiza-
tions that involve finding correspondences between sets of local features, small patches
or entire images collections. In recent years, the emergence of deep learning has fa-
cilitated significant improvements of matching based applications. This progress was
achieved through advancing improved data embedding and description, and less focus
was put on the matching process itself. Specifically, relying on simple pairwise or triplet
distance-based metric learning, ignoring the set-to-set nature of the problem.
We suggest a holistic approach to matching, by observing its natural connection to
few-shot classification (FSC), a largely growing research area that deals with learning
using limited amounts of data. We argue that certain popular FSC paradigms, such as
meta-learning and transductive learning, are particularly suitable for tackling the specific
challenges that arise in matching problems. Our approach, MFSC, builds upon state-
of-the-art features and FSC algorithms, significantly improving the quality of matching.
Moreover, we show how to construct a meta-learning scheme based on our approach,
which allows end-to-end training of the entire matching process. We validate our method
on the tasks of patch-correspondence, image-alignment and person re-identification