Classification of drinking and drinker-playing in pigs by a video-based deep learning method.

Monitoring pig drinking has been a topic of interest to pig researchers and producers for many years. However, challenges still remain due to the fact that pigs like to play with drinkers in nursery environments and that the drinking pig is often touching others. These factors negatively influence the performance of camera-based pig drinking detection algorithms. The aim of this study is to investigate a deep learning method based on convolutional neural network (CNN) and long short-term memory (LSTM) to classify drinking and drinker-playing. In the experiment, two pens of pigs were video recorded for 3 days. In video from the first pen, 5400 2 s drinking episodes and 5400 2 s drinker-playing episodes were generated with 80% of these data being allocated as training set and the remaining 20% as validation set. In video from the second pen, 12,000 2 s drinking and drinker-playing episodes were generated as a test set. Firstly, the CNN architecture ResNet50 was used to extract spatial features. These features were input into LSTM framework to extract spatial–temporal features. Through the fully connected layer, the prediction function Softmax was finally used to classify these drinking and drinker-playing episodes. In the test set, the classification accuracy in the body and head regions of interest was 87.2% and 92.5%, respectively. The results indicate that the proposed method can be used to classify pigs' drinking and drinker-playing. These classification results have potential to improve the accuracy of pig drinking detection and help farmers to estimate pig welfare. • A novel video-based deep learning method was used to study pig drinking behaviour. • Long short-term memory was used to extract the spatial–temporal features. • Classification accuracy in the body and head regions of interest was 87.2% and 92.5%. • Drinking behaviour can be detected under conditions of pigs touching and overlapping. • Shortening the region of interest can be used to improve the classification accuracy. [ABSTRACT FROM AUTHOR]