Improving Visual Matching

Michael S. Lew, Nicu Sebe, Thomas S. Huang

Abstract: Many visual matching algorithms can be described in terms of the features and the inter-feature distance or metric. The most commonly used metric is the sum of squared differences (SSD), which is valid from a maximum likelihood perspective when the real noise distribution is Gaussian. Based on real noise distributions measured from international test sets, we have found experimentally that the Gaussian noise distribution assumption is often invalid. This implies that other metrics, which have distributions closer to the real noise distribution, should be used. In this paper we considered two different visual matching applications: content-based retrieval in image databases and stereo matching. Towards broadening the results, we also implemented several sophisticated algorithms from the research literature. In each algorithm we compared the efficacy of the SSD metric, the SAD (sum of the absolute differences) metric, the Cauchy metric, and the Kullback relative information. Furthermore, in the case where sufficient training data is available, we discussed and experimentally tested a new metric based directly on the real noise distribution, which we denoted the maximum likelihood metric.