Bird posture recognition based on target keypoints estimation in dual-task convolutional neural networks.

• The first study focuses on bird's posture recognition using bird's keypoints. • End-to-end dual-task network executes keypoints detection and posture recognition. • A component generation module utilizes the keypoints to generate the components. • Fusing global and local features improve the accuracy of bird posture recognition. • New bird posture dataset containing eight types of postures named IMLab-P8-2020. Bird behavior reflects its healthy and the habitat condition in which the bird's posture recognition is the basis of bird behavior research. This paper proposes an end-to-end bird posture recognition based on target keypoints estimation in the dual-task network. The network composes two sub-networks, keypoint feature extraction network and local feature extraction network. In the keypoint feature extraction network, we take the high-resolution network (HRNet) as the backbone to detect the bird's keypoints in the high-resolution branch and extract the global features from the low-resolution branch for posture recognition. We design the components generation module in the local feature extraction network, which takes detected keypoints into the clustering algorithm to generate the bird's components. We then utilize the components into convolutional neural networks (CNNs) to extract the local features for posture recognition. Finally, we fuse the global and local features to execute the bird's posture recognition. Our main contributions are fourfold: (1) We have been modified the first point of contribution. We take the bird's keypoints, which have never been used in the bird's posture recognition, to estimate the bird's posture and achieved good results. (2) We propose an end-to-end dual-task network for bird's keypoints detection and posture recognition. (3) The components generation module utilizes the bird's keypoints to crop the bird components and generates the local feature for bird posture recognition with a shallow network. (4) Creating a bird posture image dataset named IMLab-P8-2020 containing eight bird postures. We experimentally evaluated our proposed net on self-collected data set and verified that the proposed net yields better performance and greater accuracy relative to well-known counterparts concerning various metrics. [ABSTRACT FROM AUTHOR]