Search

Abstract Video-based monitoring is essential nowadays in cattle farm management systems for automated evaluation of cow health, encompassing body condition scores, lameness detection, calving events, and other factors. In order to efficiently monitor the well-being of each individual animal, it is vital to automatically identify them in real time. Although there are various techniques available for cattle identification, a significant number of them depend on radio frequency or visible ear tags, which are prone to being lost or damaged. This can result in financial difficulties for farmers. Therefore, this paper presents a novel method for tracking and identifying the cattle with an RGB image-based camera. As a first step, to detect the cattle in the video, we employ the YOLOv8 (You Only Look Once) model. The sample data contains the raw video that was recorded with the cameras that were installed at above from the designated lane used by cattle after the milk production process and above from the rotating milking parlor. As a second step, the detected cattle are continuously tracked and assigned unique local IDs. The tracked images of each individual cattle are then stored in individual folders according to their respective IDs, facilitating the identification process. The images of each folder will be the features which are extracted using a feature extractor called VGG (Visual Geometry Group). After feature extraction task, as a final step, the SVM (Support Vector Machine) identifier for cattle identification will be used to get the identified ID of the cattle. The final ID of a cattle is determined based on the maximum identified output ID from the tracked images of that particular animal. The outcomes of this paper will act as proof of the concept for the use of combining VGG features with SVM is an effective and promising approach for an automatic cattle identification system

Abstract Recent advancements in machine learning and deep learning have revolutionized various computer vision applications, including object detection, tracking, and classification. This research investigates the application of deep learning for cattle lameness detection in dairy farming. Our study employs image processing techniques and deep learning methods for cattle detection, tracking, and lameness classification. We utilize two powerful object detection algorithms: Mask-RCNN from Detectron2 and the popular YOLOv8. Their performance is compared to identify the most effective approach for this application. Bounding boxes are drawn around detected cattle to assign unique local IDs, enabling individual tracking and isolation throughout the video sequence. Additionally, mask regions generated by the chosen detection algorithm provide valuable data for feature extraction, which is crucial for subsequent lameness classification. The extracted cattle mask region values serve as the basis for feature extraction, capturing relevant information indicative of lameness. These features, combined with the local IDs assigned during tracking, are used to compute a lameness score for each cattle. We explore the efficacy of various established machine learning algorithms, such as Support Vector Machines (SVM), AdaBoost and so on, in analyzing the extracted lameness features. Evaluation of the proposed system was conducted across three key domains: detection, tracking, and lameness classification. Notably, the detection module employing Detectron2 achieved an impressive accuracy of 98.98%. Similarly, the tracking module attained a high accuracy of 99.50%. In lameness classification, AdaBoost emerged as the most effective algorithm, yielding the highest overall average accuracy (77.9%). Other established machine learning algorithms, including Decision Trees (DT), Support Vector Machines (SVM), and Random Forests, also demonstrated promising performance (DT: 75.32%, SVM: 75.20%, Random Forest: 74.9%). The presented approach demonstrates the successful implementation for cattle lameness detection. The proposed system has the potential to revolutionize dairy farm management by enabling early lameness detection and facilitating effective monitoring of cattle health. Our findings contribute valuable insights into the application of advanced computer vision methods for livestock health management.

The global rise in the elderly population, which presents challenges to healthcare systems owing to labor shortages in caregiving facilities, necessitates innovative solutions for elderly care services. Smart aging technologies such as robotic companions and digital home gadgets, offer a solution to these challenges by improving the elderly’s quality of life and assisting caregivers. However, limitations in data privacy, real-time processing, and reliability often hinder the effectiveness of the existing technologies. Among these, privacy concerns are a major barrier to ensuring user trust and ethical implementation. Therefore, this study proposes a more effective approach for smart aging through elderly activity monitoring that prioritizes data privacy. The proposed system utilizes stereo depth cameras to monitor the activities of the elderly. Data were collected from real-world environments with the participation of six elderly individuals from a care center and hospital. This system focuses on recognizing common daily actions of the elderly including sitting, standing, lying down, and seated in a wheelchair. Additionally, it recognizes transition states (in-between actions such as changing from sitting to standing) that are crucial for assessing balance issues. By integrating motion information with a deep-learning architecture, the system achieved a high accuracy of 99.42% in recognizing daily actions in real-time. This high accuracy was maintained even with minimal data from new environments through transfer learning, and the adaptability of this model ensured its potential for real-world applications. For intuitive interaction between the caregivers and the system, a user-friendly graphical interface (GUI) was also designed in the proposed approach.

Abstract In modern cattle farm management systems, video-based monitoring has become important in analyzing the high-level behavior of cattle for monitoring their health and predicting calving for providing timely assistance. Conventionally, sensors have been used for detecting and tracking their activities. As the body-attached sensors cause stress, video cameras can be used as an alternative. However, identifying and tracking individual cattle can be difficult, especially for black and brown varieties that are so similar in appearance. Therefore, we propose a new method of using video cameras for recognizing cattle and tracking their whereabouts. In our approach, we applied a combination of deep learning and image processing techniques to build a robust system. The proposed system processes images in separate stages, namely data pre-processing, cow detection, and cow tracking. Cow detection is performed using a popular instance segmentation network. In the cow tracking stage, for successively associating each cow with the corresponding one in the next frame, we employed the following three features: cow location, appearance features, as well as recent features of the cow region. In doing so, we simply exploited the distance between two gravity center locations of the cow regions. As color and texture suitably define the appearance of an object, we analyze the most appropriate color space to extract color moment features and use a Co-occurrence Matrix (CM) for textural representation. Deep features are extracted from recent cow images using a Convolutional Neural Network (CNN features) and are also jointly applied in the tracking process to boost system performance. We also proposed a robust Multiple Object Tracking (MOT) algorithm for cow tracking by employing multiple features from the cow region. The experimental results proved that our proposed system could handle the problems of MOT and produce reliable performance.

The modeling of lactation curves is an essential aspect of formulating farm managerial practices in dairy cows. In this study, we propose and examine a Markov-Dependent stochastic approach to modeling lactation curves in dairy cows, with the aim of developing a model that accurately fits lactation curves for a maximum number of lactations. Specifically, we develop a special type of Gamma Type Markov Chain Model that considers the first-order linear regressive property, which makes the model more realistic and reliable. We compared the proposed model with three other models - quadratic model, mixed log function, and wood model - using various goodness of fit measures such as adjusted R2, root mean square error (RMSE), and Bayesian Information Criteria (BIC). Our results showed that lactation curve modeling using the proposed model could help set management strategies at the farm level. However, it is important to optimize the modeling process regularly before implementing these strategies to enhance productivity in dairy cows. Our study contributes to the existing literature by proposing a novel approach that accounts for Markov dependence and linear regression in modeling lactation curves, which can lead to more accurate and reliable predictions. This modeling approach has practical implications for dairy farmers who seek to maximize productivity and efficiency while minimizing costs.

This study innovates livestock health management, utilizing a top-view depth camera for accurate cow lameness detection, classification, and precise segmentation through integration with a 3D depth camera and deep learning, distinguishing it from 2D systems. It underscores the importance of early lameness detection in cattle and focuses on extracting depth data from the cow’s body, with a specific emphasis on the back region’s maximum value. Precise cow detection and tracking are achieved through the Detectron2 framework and Intersection Over Union (IOU) techniques. Across a three-day testing period, with observations conducted twice daily with varying cow populations (ranging from 56 to 64 cows per day), the study consistently achieves an impressive average detection accuracy of 99.94%. Tracking accuracy remains at 99.92% over the same observation period. Subsequently, the research extracts the cow’s depth region using binary mask images derived from detection results and original depth images. Feature extraction generates a feature vector based on maximum height measurements from the cow’s backbone area. This feature vector is utilized for classification, evaluating three classifiers: Random Forest (RF), K-Nearest Neighbor (KNN), and Decision Tree (DT). The study highlights the potential of top-view depth video cameras for accurate cow lameness detection and classification, with significant implications for livestock health management.

Ensuring precise calving time prediction necessitates the adoption of an automatic and precisely accurate cattle tracking system. Nowadays, cattle tracking can be challenging due to the complexity of their environment and the potential for missed or false detections. Most existing deep-learning tracking algorithms face challenges when dealing with track-ID switch cases caused by cattle occlusion. To address these concerns, the proposed research endeavors to create an automatic cattle detection and tracking system by leveraging the remarkable capabilities of Detectron2 while embedding tailored modifications to make it even more effective and efficient for a variety of applications. Additionally, the study conducts a comprehensive comparison of eight distinct deep-learning tracking algorithms, with the objective of identifying the most optimal algorithm for achieving precise and efficient individual cattle tracking. This research focuses on tackling occlusion conditions and track-ID increment cases for miss detection. Through a comparison of various tracking algorithms, we discovered that Detectron2, coupled with our customized tracking algorithm (CTA), achieves 99% in detecting and tracking individual cows for handling occlusion challenges. Our algorithm stands out by successfully overcoming the challenges of miss detection and occlusion problems, making it highly reliable even during extended periods in a crowded calving pen.

In recent years, countries around Japan and especially in Southeast Asia, white spot disease (WSD) is highly infectious and severely damages shrimp aquaculture. At the same time, the various diseases are occurring in shrimp farms. In the early stages of infection, shrimp shows three abnormal behaviors: (1) they appear in the shallow waters of the farm, (2) they do not move and do not eat even when feeding, and (3) they suddenly stop moving. Currently, infected shrimps are found by visual inspection, which places a burden on the farmers and delays the discovery. Therefore, in this paper, we proposed a system for detecting infected shrimp by using image processing technology in order to eliminate the delay of discovery and reduce the burden of farmers. According to our experimental results, the proposed system has 95% precision, 100% recall rate and an accuracy of 96.4% by using hold-out evaluation method.

The use of individual calving pens in modern farming is widely recognized as a good practice for promoting good animal welfare during parturition. However, determining the optimal time to move a pregnant cow to a calving pen can be a management challenge. Moving cows too early may result in prolonged occupancy of the pen, while moving them too late may increase the risk of calving complications and production-related diseases. In this paper, a simple random walk type Markov Chain Model to predict the optimal time for moving periparturient cows to individual calving pens was proposed. Behavior changes such as lying time, standing time, and rumination time were analyzed using a video monitoring system, and we formulated these changes as the states of a Markov Chain with an absorbing barrier. The model showed that the first time entering an absorbing state was the optimal time for a pregnant cow to be moved to a calving pen. The proposed method was validated through a series of experiments in a real-life dairy farm, showing promising results with high accuracy.

With the increasing number of cows per farmer in Japan, an automatic cow monitoring system is being introduced. One important aspect of such a system is the ability to identify individual cows and estimate their feeding time. In this study, we propose a method for achieving this goal through facial region analysis. We used a YOLO detector to extract the cow head region from video images captured during feeding with the head region cropped as a face region image. The face region image was used for cow identification and transfer learning was employed for identification. In the context of cow identification, transfer learning can be used to train a pre-existing deep neural network to recognize individual cows based on their unique physical characteristics, such as their head shape, markings, or ear tags. To estimate the time of feeding, we divided the feeding area into vertical strips for each cow and established a horizontal line just above the feeding materials to determine whether a cow was feeding or not by using Hough transform techniques. We tested our method using real-life data from a large farm, and the experimental results showed promise in achieving our objectives. This approach has the potential to diagnose diseases and movement disorders in cows and could provide valuable insights for farmers.

Effective livestock management is critical for cattle farms in today’s competitive era of smart modern farming. To ensure farm management solutions are efficient, affordable, and scalable, the manual identification and detection of cattle are not feasible in today’s farming systems. Fortunately, automatic tracking and identification systems have greatly improved in recent years. Moreover, correctly identifying individual cows is an integral part of predicting behavior during estrus. By doing so, we can monitor a cow’s behavior, and pinpoint the right time for artificial insemination. However, most previous techniques have relied on direct observation, increasing the human workload. To overcome this problem, this paper proposes the use of state-of-the-art deep learning-based Multi-Object Tracking (MOT) algorithms for a complete system that can automatically and continuously detect and track cattle using an RGB camera. This study compares state-of-the-art MOTs, such as Deep-SORT, Strong-SORT, and customized light-weight tracking algorithms. To improve the tracking accuracy of these deep learning methods, this paper presents an enhanced re-identification approach for a black cattle dataset in Strong-SORT. For evaluating MOT by detection, the system used the YOLO v5 and v7, as a comparison with the instance segmentation model Detectron-2, to detect and classify the cattle. The high cattle-tracking accuracy with a Multi-Object Tracking Accuracy (MOTA) was 96.88%. Using these methods, the findings demonstrate a highly accurate and robust cattle tracking system, which can be applied to innovative monitoring systems for agricultural applications. The effectiveness and efficiency of the proposed system were demonstrated by analyzing a sample of video footage. The proposed method was developed to balance the trade-off between costs and management, thereby improving the productivity and profitability of dairy farms; however, this method can be adapted to other domestic species.

Abnormal behavioral changes in the regular daily mobility routine of a pregnant dairy cow can be an indicator or early sign to recognize when a calving event is imminent. Image processing technology and statistical approaches can be effectively used to achieve a more accurate result in predicting the time of calving. We hypothesize that data collected using a 360-degree camera to monitor cows before and during calving can be used to establish the daily activities of individual pregnant cows and to detect changes in their routine. In this study, we develop an augmented Markov chain model to predict calving time and better understand associated behavior. The objective of this study is to determine the feasibility of this calving time prediction system by adapting a simple Markov model for use on a typical dairy cow dataset. This augmented absorbing Markov chain model is based on a behavior embedded transient Markov chain model for characterizing cow behavior patterns during the 48 h before calving and to predict the expected time of calving. In developing the model, we started with an embedded four-state Markov chain model, and then augmented that model by adding calving as both a transient state, and an absorbing state. Then, using this model, we derive (1) the probability of calving at 2 h intervals after a reference point, and (2) the expected time of calving, using their motions between the different transient states. Finally, we present some experimental results for the performance of this model on the dairy farm compared with other machine learning techniques, showing that the proposed method is promising.

Smart technologies are necessary for ambient assisted living (AAL) to help family members, caregivers, and health-care professionals in providing care for elderly people independently. Among these technologies, the current work is proposed as a computer vision-based solution that can monitor the elderly by recognizing actions using a stereo depth camera. In this work, we introduce a system that fuses together feature extraction methods from previous works in a novel combination of action recognition. Using depth frame sequences provided by the depth camera, the system localizes people by extracting different regions of interest (ROI) from UV-disparity maps. As for feature vectors, the spatial-temporal features of two action representation maps (depth motion appearance (DMA) and depth motion history (DMH) with a histogram of oriented gradients (HOG) descriptor) are used in combination with the distance-based features, and fused together with the automatic rounding method for action recognition of continuous long frame sequences. The experimental results are tested using random frame sequences from a dataset that was collected at an elder care center, demonstrating that the proposed system can detect various actions in real-time with reasonable recognition rates, regardless of the length of the image sequences.

Accurately predicting when calving will occur can provide great value in managing a dairy farm since it provides personnel with the ability to determine whether assistance is necessary. Not providing such assistance when necessary could prolong the calving process, negatively affecting the health of both mother cow and calf. Such prolongation could lead to multiple illnesses. Calving is one of the most critical situations for cows during the production cycle. A precise video-monitoring system for cows can provide early detection of difficulties or health problems, and facilitates timely and appropriate human intervention. In this paper, we propose an integrated approach for predicting when calving will occur by combining behavioral activities extracted from recorded video sequences with a Hidden Markov Model. Specifically, two sub-systems comprise our proposed system: (i) Behaviors extraction such as lying, standing, number of changing positions between lying down and standing up, and other significant activities, such as holding up the tail, and turning the head to the side; and, (ii) using an integrated Hidden Markov Model to predict when calving will occur. The experiments using our proposed system were conducted at a large dairy farm in Oita Prefecture in Japan. Experimental results show that the proposed method has promise in practical applications. In particular, we found that the high frequency of posture changes has played a central role in accurately predicting the time of calving.

In this paper, we introduce a simple method based on image analysis and deep learning that can be used in the objective assessment and measurement of tremors. A tremor is a neurological disorder that causes involuntary and rhythmic movements in a human body part or parts. There are many types of tremors, depending on their amplitude and frequency type. Appropriate treatment is only possible when there is an accurate diagnosis. Thus, a need exists for a technique to analyze tremors. In this paper, we propose a hybrid approach using imaging technology and machine learning techniques for quantification and extraction of the parameters associated with tremors. These extracted parameters are used to classify the tremor for subsequent identification of the disease. In particular, we focus on essential tremor and cerebellar disorders by monitoring the finger–nose–finger test. First of all, test results obtained from both patients and healthy individuals are analyzed using image processing techniques. Next, data were grouped in order to determine classes of typical responses. A machine learning method using a support vector machine is used to perform an unsupervised clustering. Experimental results showed the highest internal evaluation for distribution into three clusters, which could be used to differentiate the responses of healthy subjects, patients with essential tremor and patients with cerebellar disorders.

The Body Condition Score (BCS) for cows indicates their energy reserves, the scoring for which ranges from very thin to overweight. These measurements are especially useful during calving, as well as early lactation. Achieving a correct BCS helps avoid calving difficulties, losses and other health problems. Although BCS can be rated by experts, it is time-consuming and often inconsistent when performed by different experts. Therefore, the aim of our system is to develop a computerized system to reduce inconsistencies and to provide a time-saving solution. In our proposed system, the automatic body condition scoring system is introduced by using a 3D camera, image processing techniques and regression models. The experimental data were collected on a rotary parlor milking station on a large-scale dairy farm in Japan. The system includes an application platform for automatic image selection as a primary step, which was developed for smart monitoring of individual cows on large-scale farms. Moreover, two analytical models are proposed in two regions of interest (ROI) by extracting 3D surface roughness parameters. By applying the extracted parameters in mathematical equations, the BCS is automatically evaluated based on measurements of model accuracy, with one of the two models achieving a mean absolute percentage error (MAPE) of 3.9%, and a mean absolute error (MAE) of 0.13.

Advances in image processing technologies have provided more precise views in medical and health care management systems. Among many other topics, this paper focuses on several aspects of video-based monitoring systems for elderly people living independently. Major concerns are patients with chronic diseases and adults with a decline in physical fitness, as well as falling among elderly people, which is a source of life-threatening injuries and a leading cause of death. Therefore, in this paper, we propose a video-vision-based monitoring system using image processing technology and a Hidden Markov Model for differentiating falls from normal states for people. Specifically, the proposed system is composed of four modules: (1) object detection; (2) feature extraction; (3) analysis for differentiating normal states from falls; and (4) a decision-making process using a Hidden Markov Model for sequential states of abnormal and normal. In the object detection module, background and foreground segmentation is performed by applying the Mixture of Gaussians model, and graph cut is applied for foreground refinement. In the feature extraction module, the postures and positions of detected objects are estimated by applying the hybrid features of the virtual grounding point, inclusive of its related area and the aspect ratio of the object. In the analysis module, for differentiating normal, abnormal, or falling states, statistical computations called the moving average and modified difference are conducted, both of which are employed to estimate the points and periods of falls. Then, the local maximum or local minimum and the half width value are determined in the observed modified difference to more precisely estimate the period of a falling state. Finally, the decision-making process is conducted by developing a Hidden Markov Model. The experimental results used the Le2i fall detection dataset, and showed that our proposed system is robust and reliable and has a high detection rate.

Nowadays, for numerous reasons, smart farming systems focus on the use of image processing technologies and 5G communications. In this paper, we propose a tracking system for individual cows using an ear tag visual analysis. By using ear tags, the farmers can track specific data for individual cows such as body condition score, genetic abnormalities, etc. Specifically, a four-digit identification number is used, so that a farm can accommodate up to 9999 cows. In our proposed system, we develop an individual cow tracker to provide effective management with real-time upgrading enforcement. For this purpose, head detection is first carried out to determine the cow’s position in its related camera view. The head detection process incorporates an object detector called You Only Look Once (YOLO) and is then followed by ear tag detection. The steps involved in ear tag recognition are (1) finding the four-digit area, (2) digit segmentation using an image processing technique, and (3) ear tag recognition using a convolutional neural network (CNN) classifier. Finally, a location searching system for an individual cow is established by entering the ID numbers through the application’s user interface. The proposed searching system was confirmed by performing real-time experiments at a feeding station on a farm at Hokkaido prefecture, Japan. In combination with our decision-making process, the proposed system achieved an accuracy of 100% for head detection, and 92.5% for ear tag digit recognition. The results of using our system are very promising in terms of effectiveness.

In this paper, an innovative home care video monitoring system for detecting abnormal and normal events is proposed by introducing a virtual grounding point (VGP) concept. To be specific, the proposed system is composed of four main image processing components: (1) visual object detection, (2) feature extraction, (3) abnormal and normal event analysis, and (4) the decision-making process. In the object detection component, background subtraction is first achieved using a specific mixture of Gaussians (MoG) to model the foreground in the form of a low-rank matrix factorization. Then, a theory of graph cut is applied to refine the foreground. In the feature extraction component, the position and posture of the detected person is estimated by using a combination of the virtual grounding point, along with its related centroid, area, and aspect ratios. In analyzing the abnormal and normal events, the moving averages (MA) for the extracted features are calculated. After that, a new curve analysis is computed, specifically using the modified difference (MD). The local maximum (lmax), local minimum (lmin), and half width value (vhw) are determined on the observed curve of the modified difference. In the decision-making component, the support vector machine (SVM) method is applied to detect abnormal and normal events. In addition, a new concept called period detection (PD) is proposed to robustly detect the abnormal events. The experimental results were obtained using the Le2i fall detection dataset to confirm the reliability of the proposed method, and that it achieved a high detection rate.

Nowadays, with the emergence of sophisticated electronic devices, human daily activities are becoming more and more complex. On the other hand, research has begun on the use of reliable, cost-effective sensors, patient monitoring systems, and other systems that make daily life more comfortable for the elderly. Moreover, in the field of computer vision, human action recognition (HAR) has drawn much attention as a subject of research because of its potential for numerous cost-effective applications. Although much research has investigated the use of HAR, most has dealt with simple basic actions in a simplified environment; not much work has been done in more complex, real-world environments. Therefore, a need exists for a system that can recognize complex daily activities in a variety of realistic environments. In this paper, we propose a system for recognizing such activities, in which humans interact with various objects, taking into consideration object-oriented activity information, the use of deep convolutional neural networks, and a multi-class support vector machine (multi-class SVM). The experiments are performed on a publicly available cornell activity dataset: CAD-120 which is a dataset of human−object interactions featuring ten high-level daily activities. The outcome results show that the proposed system achieves an accuracy of 93.33%, which is higher than other state-of-the-art methods, and has great potential for applications recognizing complex daily activities.

This study proposes an advanced camera-based monitoring system for individual black cows in closed farms. By leveraging computer vision and deep learning, the system identifies five key cow actions: eating, drinking, sitting, standing, and walking. A multi-stage approach classifies actions first as static (eating, drinking, sitting, standing) or dynamic (walking) categories based on Kalman Filter velocity information. Further classification distinguishes among four static actions. A Convolutional Neural Network (CNN) refines especially for sitting and standing. On the other hand, cow head regions and specific zone locations help distinguish eating and drinking. The system achieves an overall accuracy of 80% in long data sequences, demonstrating its potential for precision livestock farming. [ABSTRACT FROM AUTHOR]

Utilizing computer vision for animal behavior monitoring provides significant benefits by minimizing direct handling and capturing diverse traits through a single sensor. However, accurately identifying animals remains a challenge. To address this, this study introduces an innovative approach to monitor black cattle in dynamic agricultural environments to ensure their health welfare. By integrating advanced techniques like DETIC for automated labeling and YOLOv8 for real-time detection, the research emphasizes improving accuracy and robustness in tracking black cattle tracking within complex open ranch environments. Moreover, the customized ByteTrack model tailored for ranch scenarios significantly enhances cattle tracking across intricate landscapes. Achieving a mean Average Precision (mAP) of 0.901 and a Multi-Object Tracking Accuracy (MOTA) of average accuracy 92.185% of four videos, this approach appears to offer a viable resolution for conducting individual cattle behavior analysis experiments through the application of computer vision. [ABSTRACT FROM AUTHOR]