Image Based Food Energy Estimation With Depth Domain Adaptation

Gautham Vinod, Zeman Shao, Fengqing Zhu

Abstract: Assessment of dietary intake has primarily relied on self-report instruments, which are prone to measurement errors. Dietary assessment methods have increasingly incorporated tech-nological advances particularly mobile, image based approaches to address some of these limitations and further automation. Mobile, image-based methods can reduce user burden and bias by automatically estimating dietary intake from eating occasion images that are captured by mobile devices. In this paper, we propose an “Energy Density Map” which is a pixel-to-pixel mapping from the RGB image to the energy density of the food. We then incorporate the “Energy Density Map” with an associated depth map that is captured by a depth sensor to estimate the food energy. The proposed method is evaluated on the N utrition5k dataset. Experimental results show improved results compared to baseline methods with an average error of 13.29 kCal and average percentage error of 13.57 % between the ground-truth and the estimated energy of the food.