Mobile application for receipt fraud detection based on optical character recognition

Sorin Liviu Jurj, Allen-Jasmin Farcas, Flavius Opritoiu, Mircea Vladutiu

Abstract: This paper presents a method for detecting receipt fraud by implementing an Object Character Recognition (OCR) algorithm composed of Image Processing Techniques and Convolutional Neural Networks (CNNs). We implemented two CNN models into a smartphone application that gives customers the option to take pictures of products they intend to buy (also to crop their price tags) while present in a hypermarket/supermarket as well as of the paid receipt and succeeds to automatically identify and compare all prices (multiple digits including decimals) of the products seen at the shelf and all prices found in the paid receipt, received from the cashier. This application helps the customer detect a receipt fraud due to a computer or human error, in a cheap and convenient way. Experimental results show 99.96% overall test accuracy for the CNN responsible for identifying product prices and 99.35% overall test accuracy for the CNN responsible for identifying receipt prices.