MNSSD: A Real-time DNN based Companion Image Data Annotation using MobileNet and Single Shot Multibox Detector

Md Golam Morshed, Young-Koo Lee

Abstract: Images, text, and other forms of data collected through the device or sensors that we carry with us, such as a smart phone, camera, and so on, are referred to as companion data. There is considerable interest in automatic assignment of keywords to images in order to index, retrieve and analyze huge amounts of image data. In the past decade, many image annotation methods have been developed, many of which perform well on benchmark datasets. However, very little attention has been given to annotating the companion image data in real time using deep neural network. A new baseline technique for companion image data annotation using deep neural networks is introduced in this study that approaches annotation as a retrieval task. As an object detector, the Single Shot Multibox Detector is utilized, while the MobileNetV2 architecture serves as the classifier. This portable and lightweight architecture may also be utilized in integrated systems (such as smart phones and Raspberry Pis) to conduct real-time image annotation. We achieved 82.37% precision and 74.03% F1 score using the method described in this paper. For image annotation, our baseline model beats the existing best practices. We anticipate that adopting a baseline will provide a solid foundation for comparing and comprehending future annotation methods.