Search

The aim of this study is to develop a computer vision-based method to automatically detect aggressive behaviors among pigs. Ten repetitions of the same experiment were performed. In each of the experiment, 7 piglets were mixed from three litters in two pigpens and captured on video for a total of 6 h. From these videos, the first 3 h of video after mixing were recorded as a training set, and the 3 h of video after 24 h were recorded as a validation set. Connected area and adhesion index were used to locate aggressive pigs and to extract key frame sequences. The two pigs in aggression were regarded as a whole rectangle according to their characteristics of continuous and large-proportion adhesion. The acceleration feature was extracted by analyzing the displacement change of four sides of this rectangle between adjacent frames, and hierarchical clustering was used to calculate its threshold. Based on this feature, the recognition rules of medium and high aggression were designed. Testing 10 groups of pigs, the accuracy of recognizing medium aggression was 95.82% with a sensitivity of 90.57% and with a specificity of 96.95%, and the accuracy of recognizing high aggression was 97.04% with a sensitivity of 92.54% and with a specificity of 97.38%. These results indicate that the acceleration can be used to recognize pigs’ aggressive behaviors. [ABSTRACT FROM AUTHOR]

• Computer vision methods for livestock behaviour recognition were evaluated. • Journey from computer vision to deep learning was elaborated. • This paper involves with segmentation, identification and behaviour recognition. • Research trend of livestock behaviour recognition was illustrated from four aspects. • This review could provide researchers and producers with technical references. The increasing demand for sustainable livestock products also demands new considerations in animal breeding. Breeding programs are now seeking to integrate animal behavioural phenotypes, as these relate to the productivity, health and welfare of the animals and thereby can influence yield and economic benefits in the industry. Traditional manual observation of pig behaviour is time-consuming, laborious, subjective, and difficult to achieve in continuous and large-scale operations. It is not surprising that computer vision technology with the advantages of being objective, non-invasive and continuous has been widely researched for its use in the recognition of livestock behaviours over recent years. Nevertheless, in studies of livestock behaviour recognition, computer vision technology faces some challenges, e.g., complex scenes, variable illumination, occlusion, touching and overlapping between livestock, which has limited the fast translation of technology to industry. On the other hand, deep learning technology has proven to solve these difficulties to a certain extent and is being adopted to recognise livestock behaviours. This paper mainly evaluates the recent developments in computer vision methods for recognition of these behaviours in pigs and cattle. The focus on these species is made possible by the number of studies exist quantifying behaviours that are of importance for their health, welfare and productivity such as aggression, drinking, feeding, lameness, mounting, posture, tail-biting and nursing. This review paper especially analyses the development of image segmentation, identification and behaviour recognition using tradition computer vision and more recent deep learning methods, and evaluates the evolution of key research in the field. We elaborate the research trend of livestock behaviour recognition from four aspects, i.e., development of robust livestock identification algorithms, recognition of livestock behaviours for different growth stages, further quantification of the results of behaviour recognition, and building evaluation system of growth status, health and welfare. [ABSTRACT FROM AUTHOR]

• Feeding behaviour of multiple pigs at the nursery stage was recognised in this study. • Xception and LSTM were combined to extract discriminative spatial-temporal features. • These features can used to recognise feeding behaviour with an accuracy of 98.4%. • The identity and feeding time of each pig were determined in 4 feeding sub-regions. • The proposed algorithm could recognise 98.5% of feeding time of individual pigs. Monitoring the feeding behaviour of pigs and measuring their feeding time can help farmers to evaluate the pig health and welfare. The aim of this study was to develop a video-based deep learning algorithm to recognise feeding behaviour of nursery pigs and determine the feeding time of pigs on an individual level. In the experiment, two pens of pigs were video recorded for 3 days. In the video of feeding region of pen 1, 96,000 1 s feeding episodes and 96,000 1 s non-feeding episodes were generated. 70% of these data was randomly selected as training set and the remaining 30% as validation set. In the video of feeding region of pen 2, all the labelled 24,656 1 s feeding episodes and 61,744 1 s non-feeding episodes were used as test set. Firstly, the Convolutional Neural Network (CNN) architecture Xception was used to extract spatial features. These features were input into Long Short-term Memory (LSTM) framework to extract spatial-temporal features. Through the fully connected layer, the prediction function Softmax was finally used to classify these 1 s episodes as feeding or non-feeding. An image processing algorithm based on maximum entropy segmentation, HSV (Hue, Saturation and Value) colour space transformation and template matching was proposed to calculate the circularity of the head, the ratio of the head to the feeding sub-region, the accumulated pixels of the head motion, and the distance from the head to the number on pig back in order to determine the identity and feeding time of each pig. In the test set, the proposed algorithm could recognise feeding behaviour with an accuracy of 98.4%, a sensitivity of 98.8%, specificity of 98.3% and precision of 95.9%, and could correctly recognise 98.5% of feeding time of individual pigs from the total 45944 s feeding time of 8 pigs. The results indicate that the proposed method can be used to recognise feeding behavior of pigs and determine feeding time of each pig. [ABSTRACT FROM AUTHOR]

• It is the first time to use computer vision to recognise pig enrichment engagement. • A HSV colour space-based method was proposed to track the object region of interest. • InceptionV3 and LSTM were used to extract discriminative spatial-temporal features. • This algorithm has reference value for classifying similar behaviours of pigs. • Pigs preference to 3 enrichment objects was also determined in this study. As providing objects that pigs prefer can reduce the occurrence of tail-biting and aggression and consequently improve animal welfare, automatic recognition of pigs' engagement with different objects can have practical value. Therefore, aim of this study was to develop a computer vision based approach that utilised a recurrent neural network-based deep learning algorithm to recognise pig enrichment engagement (EE) behaviours and preliminarily determine the preference to objects. Two pig pens were studied. 1 day of video was recorded in pen 1, which generated 2400 1 s EE and 2400 1 s non-EE episodes. 80% of these data was randomly selected as training set and the remaining 20% as validation set. Moreover, 4 days of video were recorded and used as the test set in pen 2. Firstly, the HSV (Hue, Saturation, Value) colour space-based tracking algorithm was developed to locate object region of interest. Secondly, the convolutional neural network (CNN) architecture InceptionV3 was used to extract spatial features from each frame. These features were input into the long short-term memory (LSTM) framework to extract spatial-temporal features from each episode. Through the fully connected layer, the prediction function Softmax was finally used to classify these episodes as EE or non-EE behaviour. In the validation set, the proposed algorithm could recognise EE with blue ball, golden ball and wooden beam with an accuracy of 95.2%, 95.4% and 97.3%, respectively. By shortening the radius of the region of interest into a half of the average length of pig body, the corresponding accuracy could be improved into 96.9%, 97.1% and 97.9%, respectively. In the test set, the proposed algorithm could recognise EE with each of these 3 objects with an accuracy of 96.5%, 96.8% and 97.6%, respectively. The proportion of EE with each of these 3 objects was 75.8%, 6.0% and 18.2%, respectively. These results indicate that the proposed method can be used to recognise EE behaviours of pigs, and halving the radius of the region of interest can improve the recognition accuracy of EE behaviours. Moreover, the preference of pigs to objects based on EE duration were preliminarily determined as blue ball > wooden beam > golden ball. The obtained duration of EE behaviours can help farmers to evaluate the enrichment used and thereby to increase the health and welfare of the pigs in their care. Furthermore, the proposed algorithm has reference value for the classification of the behaviours with similar motion patterns. [ABSTRACT FROM AUTHOR]

• A deep learning method based on CNN and LSTM was used to recognise pig aggression. • This study directly process video episodes rather than individual frames. • CNN features were input into LSTM framework to extract spatial-temporal features. • The proposed method could recognise aggressive episodes with an accuracy of 97.2%. • A frame skipping approach can improve the accuracy to 98.4% and halve running time. Aggression is considered as a major animal welfare problem in commercial pig farming. The aim of this study is to develop a deep learning method based on convolutional neural network (CNN) and long short-term memory (LSTM) to recognise aggressive episodes of pigs. Compared to previous studies of pig behaviours based on deep learning, this study directly process video episodes rather than individual frames. In the experiment, nursery pigs (8/pen) were mixed for 3 days and then 8 h of video was recorded in each day. From these videos, 600 aggressive 2 s-episodes were manually selected and then augmented into 2400 episodes by using horizontal, vertical and diagonal mirroring. From the videos, 2400 non-aggressive 2 s-episodes were also manually selected. 80% of the data were randomly allocated as training set and the remaining 20% as validation set. Firstly, the CNN architecture VGG-16 was used to extract spatial features. These features were then input into LSTM framework to further extract temporal features. Through fully connected layer, the prediction function Softmax was finally used to determine if the current episode is aggression or non-aggression. Using the proposed method, aggressive episodes could be recognised with an accuracy of 97.2%. This result indicates that this method can be used to recognise aggressive episodes of pigs. Additionally, this paper further investigates the validity of this method under the conditions of skipping frames and reducing the episode length. The results show that a frame skipping approach whereby 30 fps is reduced into 15 fps within each 2 s-episode can improve the accuracy into 98.4% and halve the total running time. [ABSTRACT FROM AUTHOR]

• A new depth sensor of RealSense D435 was used for pig aggression detection. • Advantages of group analysis and 3D cameras were combined for the first time. • Connected area threshold was set to remove moving pixels without aggression. • SVM was used to classify the 4 features of motion shape index in each 3 s-unit. • The proposed algorithm could detect aggression with higher evaluation parameters. The aim of this study is to develop a depth image analysis method to automatically detect aggressive behaviours of group-housed pigs. In this work 2 experiments were performed. In each experiment, 8 pigs from 3 pens were mixed for 3 days and then 8 h of video was recorded in each day. In the 24 h data of the first experiment, all the 883 aggressive 3 s-units and manually selected 883 non-aggressive 3 s-units were used as training set. In the 24 h data of the second experiment, all the 856 aggressive 3 s-units and manually selected 856 non-aggressive 3 s-units were used as test set. Firstly, frame-to-frame distance was set and then frame difference method was used to obtain moving pixels. Secondly, moving pixels caused by non-aggressive behaviours were removed by setting threshold of connected area. Number of filtered moving pixels were summed and defined as motion shape index (MSI) in each frame. After dividing all videos into 3 s-units, the maximum, mean, variance and standard deviation of MSI in each unit were extracted as features. Finally, support vector machine (SVM) was used to classify these features in order to detect aggression. Using the proposed algorithm, aggressive behaviours could be detected with an accuracy of 97.5%, a sensitivity of 98.2%, specificity of 96.7% and precision of 96.8%. The results indicate that this algorithm can be used to detect aggressive behaviours of pigs. [ABSTRACT FROM AUTHOR]