Generating 3D fish motion skeleton via iterative optimization method and FishSkeletonNet

Min Shi, Guo-Liang Zhao, Shi-sheng Guo, Bi-lian Sun, Dengming Zhu, Xiu-juan Chai, Zhao-Xin Li, Xinru Zhuo

Abstract: The fish motion skeleton serves as the foundation for 3D fish motion modeling, enabling the manipulation of fish posture deformations and movements, while also providing a robust framework for analyzing fish behavior to assess their health status and overall performance. However, the joints within the fish motion skeleton, responsible for driving the fish’s movements, are not always stable, which undergo changes as the fish grows. The unstable topology of the skeleton poses a challenge when attempting to simulate a lifelike fish skeleton. In this paper, we present a novel method for generating a 3D fish skeleton based on fish posture data. Our approach establishes an initial motion skeleton including the spine and fins. We then determine its parameters, encompassing joint positions and the number of joints, through iterative optimization, employing collected data from fish with various shapes and five common postures as constraints. Furthermore, the skeletons generated through this optimization process are utilized as sample data for training the FishSkeletonNet network, a framework introduced in this paper for predicting fish motion skeletons of input 3D fish bodies. To validate the effectiveness of our approach, we introduce a new dataset of grass carp postures, on which we carry out experiments and conduct both quantitative and qualitative evaluations. The experiments illustrate that our method generates fish motion skeletons that closely emulate the actual motion skeleton structure of fish, demonstrating a higher level of biological plausibility compared to existing methods.