IMTFF-Networks: A deep learning model for cattle behavior classification integrating multimodal time-frequency features

Xinglin Ni, Hou zhenjie, En Lin, Xing Li, Jiuzhen Liang, Xinwen Zhou

Abstract: In modern livestock farming, understanding cattle behavior patterns is crucial for improving their health and production efficiency. For example, behaviors such as rumination (chewing food) and eating not only affect their health but also directly influence farm economic efficiency and food safety. However, these behaviors exhibit complex dynamic characteristics, showing significant nonlinearity and temporal dependence, which presents challenges in representing behavioral features for accurately capturing long-term dependencies and nonlinear relationships, thereby limiting classification accuracy. To address these challenges, we propose a deep learning model for cattle behavior classification Integrating Multimodal Time-Frequency Features (IMTFF-Networks). Compared to traditional deep learning models such as Long Short-Term Memory (LSTM) or Temporal Convolutional Networks (TCN), IMTFF-Networks can simultaneously capture both time-domain and frequency-domain features, overcoming the limitations of relying solely on time-domain or frequency-domain features when processing complex behavior patterns. IMTFF-Networks comprises modules for multimodal time-frequency fusion of sensor signals and multi-scale time-frequency feature analysis. The multimodal time-frequency fusion module integrates time-frequency features through Feature Mode Decomposition (FMD) and frequency domain processing, comprehensively capturing the dynamic characteristics of behavioral signals, such as frequency components and temporal patterns, thereby enhancing the accuracy and robustness of behavior recognition. The multi-scale time-frequency feature analysis module employs multi-scale temporal processing and attention mechanisms to capture long-term dependencies and complex nonlinear relationships among features across different time scales. The experiments were conducted using 3460 hours of behavioral data from 18 Limousin crossbred steer, encompassing rumination, eating, and other behavioral categories. The results demonstrate that IMTFF-Networks significantly outperforms methods relying solely on single time-domain or frequency-domain features in cattle behavior classification tasks. Specifically, IMTFF-Networks achieved an accuracy of 86.42 %, a balanced accuracy of 84.78 %, precision of 87.53 %, recall of 86.42 %, and F1 score of 86.64 % on the test set. This study not only validates the effectiveness of IMTFF-Networks in processing complex temporal data but also provides important references for future research in animal behavior monitoring.