Automatic fish weight estimation and 3D surface reconstruction with a lightweight instance segmentation model

Guangxu Wang, Jiaxuan Yu, Sitao Liu, Wenkai Xu, Xin Li, Yinfeng Hao, Daoliang Li

Abstract: Automatic methods for fish weight estimation are crucial for biomass assessment, health monitoring, and precision feeding in aquaculture. However, variations in posture and rapid motion of fish present substantial challenges to the accurate estimation of the weight in free-swimming fish. This study presents an innovative surface area-based weight estimation system for free-swimming fish, using a binocular stereo vision system integrated with advanced instance segmentation techniques. Our proposed lightweight instance segmentation model employs an efficient vision transformer as the backbone network, integrated with a cross-scale feature fusion module to enhance feature perception and aggregation, and a segmentation head optimized through an attention mechanism. This approach effectively captures salient features of fish, enabling accurate segmentation and pose recognition. Furthermore, our methodology enables precise three-dimensional (3D) reconstruction of fish surfaces by leveraging stereo matching, depth estimation, 3D point cloud generation, and mesh reconstruction. The experimental results indicate that our model considerably surpasses current approaches, achieving a mean average precision (mAP50) of 94.0 % in segmentation accuracy and exhibiting robust performance in real-world scenarios. Addressing the challenges posed by posture variation and rapid motion in free-swimming fish, our study demonstrates a strong correlation (R = 0.967) between surface area and weight of fish, highlighting the potential of integrating stereo vision and instance segmentation for accurate biomass estimation in aquaculture.