On accuracy, robustness, and security of bag-of-word search systems

Svyatoslav Voloshynovskiy, Maurits Diephuis, Dimche Kostadinov, Farzad Farhadzadeh, Taras Holotyak

Abstract: In this paper, we present a statistical framework for the analysis of the performance of Bag-of-Words (BOW) systems. The paper aims at establishing a better understanding of the impact of different elements of BOW systems such as the robustness of descriptors, accuracy of assignment, descriptor compression and pooling and finally decision making. We also study the impact of geometrical information on the BOW system performance and compare the results with different pooling strategies. The proposed framework can also be of interest for a security and privacy analysis of BOW systems. The experimental results on real images and descriptors confirm our theoretical findings. Notation: We use capital letters to denote scalar random variables X and X to denote vector random variables, corresponding small letters x and x to denote the realisations of scalar and vector random variables, respectively. We use X ~pX(x) or simply X ~p(x) to indicate that a random variable X is distributed according to pX(x). N(μ, σ² _X ) stands for the Gaussian distribution with mean μ and variance σ² _X . B(L, P_b) denotes the binomial distribution with sequence length L and probability of success P_b. ║.║denotes the Euclidean vector norm and Q(.) stands for the Q-function. D(.║.) denotes the divergence and E{.} denotes the expectation.