Search

This study innovates livestock health management, utilizing a top-view depth camera for accurate cow lameness detection, classification, and precise segmentation through integration with a 3D depth camera and deep learning, distinguishing it from 2D systems. It underscores the importance of early lameness detection in cattle and focuses on extracting depth data from the cow’s body, with a specific emphasis on the back region’s maximum value. Precise cow detection and tracking are achieved through the Detectron2 framework and Intersection Over Union (IOU) techniques. Across a three-day testing period, with observations conducted twice daily with varying cow populations (ranging from 56 to 64 cows per day), the study consistently achieves an impressive average detection accuracy of 99.94%. Tracking accuracy remains at 99.92% over the same observation period. Subsequently, the research extracts the cow’s depth region using binary mask images derived from detection results and original depth images. Feature extraction generates a feature vector based on maximum height measurements from the cow’s backbone area. This feature vector is utilized for classification, evaluating three classifiers: Random Forest (RF), K-Nearest Neighbor (KNN), and Decision Tree (DT). The study highlights the potential of top-view depth video cameras for accurate cow lameness detection and classification, with significant implications for livestock health management.

Ensuring precise calving time prediction necessitates the adoption of an automatic and precisely accurate cattle tracking system. Nowadays, cattle tracking can be challenging due to the complexity of their environment and the potential for missed or false detections. Most existing deep-learning tracking algorithms face challenges when dealing with track-ID switch cases caused by cattle occlusion. To address these concerns, the proposed research endeavors to create an automatic cattle detection and tracking system by leveraging the remarkable capabilities of Detectron2 while embedding tailored modifications to make it even more effective and efficient for a variety of applications. Additionally, the study conducts a comprehensive comparison of eight distinct deep-learning tracking algorithms, with the objective of identifying the most optimal algorithm for achieving precise and efficient individual cattle tracking. This research focuses on tackling occlusion conditions and track-ID increment cases for miss detection. Through a comparison of various tracking algorithms, we discovered that Detectron2, coupled with our customized tracking algorithm (CTA), achieves 99% in detecting and tracking individual cows for handling occlusion challenges. Our algorithm stands out by successfully overcoming the challenges of miss detection and occlusion problems, making it highly reliable even during extended periods in a crowded calving pen.

The use of individual calving pens in modern farming is widely recognized as a good practice for promoting good animal welfare during parturition. However, determining the optimal time to move a pregnant cow to a calving pen can be a management challenge. Moving cows too early may result in prolonged occupancy of the pen, while moving them too late may increase the risk of calving complications and production-related diseases. In this paper, a simple random walk type Markov Chain Model to predict the optimal time for moving periparturient cows to individual calving pens was proposed. Behavior changes such as lying time, standing time, and rumination time were analyzed using a video monitoring system, and we formulated these changes as the states of a Markov Chain with an absorbing barrier. The model showed that the first time entering an absorbing state was the optimal time for a pregnant cow to be moved to a calving pen. The proposed method was validated through a series of experiments in a real-life dairy farm, showing promising results with high accuracy.

With the increasing number of cows per farmer in Japan, an automatic cow monitoring system is being introduced. One important aspect of such a system is the ability to identify individual cows and estimate their feeding time. In this study, we propose a method for achieving this goal through facial region analysis. We used a YOLO detector to extract the cow head region from video images captured during feeding with the head region cropped as a face region image. The face region image was used for cow identification and transfer learning was employed for identification. In the context of cow identification, transfer learning can be used to train a pre-existing deep neural network to recognize individual cows based on their unique physical characteristics, such as their head shape, markings, or ear tags. To estimate the time of feeding, we divided the feeding area into vertical strips for each cow and established a horizontal line just above the feeding materials to determine whether a cow was feeding or not by using Hough transform techniques. We tested our method using real-life data from a large farm, and the experimental results showed promise in achieving our objectives. This approach has the potential to diagnose diseases and movement disorders in cows and could provide valuable insights for farmers.

Effective livestock management is critical for cattle farms in today’s competitive era of smart modern farming. To ensure farm management solutions are efficient, affordable, and scalable, the manual identification and detection of cattle are not feasible in today’s farming systems. Fortunately, automatic tracking and identification systems have greatly improved in recent years. Moreover, correctly identifying individual cows is an integral part of predicting behavior during estrus. By doing so, we can monitor a cow’s behavior, and pinpoint the right time for artificial insemination. However, most previous techniques have relied on direct observation, increasing the human workload. To overcome this problem, this paper proposes the use of state-of-the-art deep learning-based Multi-Object Tracking (MOT) algorithms for a complete system that can automatically and continuously detect and track cattle using an RGB camera. This study compares state-of-the-art MOTs, such as Deep-SORT, Strong-SORT, and customized light-weight tracking algorithms. To improve the tracking accuracy of these deep learning methods, this paper presents an enhanced re-identification approach for a black cattle dataset in Strong-SORT. For evaluating MOT by detection, the system used the YOLO v5 and v7, as a comparison with the instance segmentation model Detectron-2, to detect and classify the cattle. The high cattle-tracking accuracy with a Multi-Object Tracking Accuracy (MOTA) was 96.88%. Using these methods, the findings demonstrate a highly accurate and robust cattle tracking system, which can be applied to innovative monitoring systems for agricultural applications. The effectiveness and efficiency of the proposed system were demonstrated by analyzing a sample of video footage. The proposed method was developed to balance the trade-off between costs and management, thereby improving the productivity and profitability of dairy farms; however, this method can be adapted to other domestic species.

Abnormal behavioral changes in the regular daily mobility routine of a pregnant dairy cow can be an indicator or early sign to recognize when a calving event is imminent. Image processing technology and statistical approaches can be effectively used to achieve a more accurate result in predicting the time of calving. We hypothesize that data collected using a 360-degree camera to monitor cows before and during calving can be used to establish the daily activities of individual pregnant cows and to detect changes in their routine. In this study, we develop an augmented Markov chain model to predict calving time and better understand associated behavior. The objective of this study is to determine the feasibility of this calving time prediction system by adapting a simple Markov model for use on a typical dairy cow dataset. This augmented absorbing Markov chain model is based on a behavior embedded transient Markov chain model for characterizing cow behavior patterns during the 48 h before calving and to predict the expected time of calving. In developing the model, we started with an embedded four-state Markov chain model, and then augmented that model by adding calving as both a transient state, and an absorbing state. Then, using this model, we derive (1) the probability of calving at 2 h intervals after a reference point, and (2) the expected time of calving, using their motions between the different transient states. Finally, we present some experimental results for the performance of this model on the dairy farm compared with other machine learning techniques, showing that the proposed method is promising.

Smart technologies are necessary for ambient assisted living (AAL) to help family members, caregivers, and health-care professionals in providing care for elderly people independently. Among these technologies, the current work is proposed as a computer vision-based solution that can monitor the elderly by recognizing actions using a stereo depth camera. In this work, we introduce a system that fuses together feature extraction methods from previous works in a novel combination of action recognition. Using depth frame sequences provided by the depth camera, the system localizes people by extracting different regions of interest (ROI) from UV-disparity maps. As for feature vectors, the spatial-temporal features of two action representation maps (depth motion appearance (DMA) and depth motion history (DMH) with a histogram of oriented gradients (HOG) descriptor) are used in combination with the distance-based features, and fused together with the automatic rounding method for action recognition of continuous long frame sequences. The experimental results are tested using random frame sequences from a dataset that was collected at an elder care center, demonstrating that the proposed system can detect various actions in real-time with reasonable recognition rates, regardless of the length of the image sequences.

Accurately predicting when calving will occur can provide great value in managing a dairy farm since it provides personnel with the ability to determine whether assistance is necessary. Not providing such assistance when necessary could prolong the calving process, negatively affecting the health of both mother cow and calf. Such prolongation could lead to multiple illnesses. Calving is one of the most critical situations for cows during the production cycle. A precise video-monitoring system for cows can provide early detection of difficulties or health problems, and facilitates timely and appropriate human intervention. In this paper, we propose an integrated approach for predicting when calving will occur by combining behavioral activities extracted from recorded video sequences with a Hidden Markov Model. Specifically, two sub-systems comprise our proposed system: (i) Behaviors extraction such as lying, standing, number of changing positions between lying down and standing up, and other significant activities, such as holding up the tail, and turning the head to the side; and, (ii) using an integrated Hidden Markov Model to predict when calving will occur. The experiments using our proposed system were conducted at a large dairy farm in Oita Prefecture in Japan. Experimental results show that the proposed method has promise in practical applications. In particular, we found that the high frequency of posture changes has played a central role in accurately predicting the time of calving.

In this paper, we introduce a simple method based on image analysis and deep learning that can be used in the objective assessment and measurement of tremors. A tremor is a neurological disorder that causes involuntary and rhythmic movements in a human body part or parts. There are many types of tremors, depending on their amplitude and frequency type. Appropriate treatment is only possible when there is an accurate diagnosis. Thus, a need exists for a technique to analyze tremors. In this paper, we propose a hybrid approach using imaging technology and machine learning techniques for quantification and extraction of the parameters associated with tremors. These extracted parameters are used to classify the tremor for subsequent identification of the disease. In particular, we focus on essential tremor and cerebellar disorders by monitoring the finger–nose–finger test. First of all, test results obtained from both patients and healthy individuals are analyzed using image processing techniques. Next, data were grouped in order to determine classes of typical responses. A machine learning method using a support vector machine is used to perform an unsupervised clustering. Experimental results showed the highest internal evaluation for distribution into three clusters, which could be used to differentiate the responses of healthy subjects, patients with essential tremor and patients with cerebellar disorders.

The Body Condition Score (BCS) for cows indicates their energy reserves, the scoring for which ranges from very thin to overweight. These measurements are especially useful during calving, as well as early lactation. Achieving a correct BCS helps avoid calving difficulties, losses and other health problems. Although BCS can be rated by experts, it is time-consuming and often inconsistent when performed by different experts. Therefore, the aim of our system is to develop a computerized system to reduce inconsistencies and to provide a time-saving solution. In our proposed system, the automatic body condition scoring system is introduced by using a 3D camera, image processing techniques and regression models. The experimental data were collected on a rotary parlor milking station on a large-scale dairy farm in Japan. The system includes an application platform for automatic image selection as a primary step, which was developed for smart monitoring of individual cows on large-scale farms. Moreover, two analytical models are proposed in two regions of interest (ROI) by extracting 3D surface roughness parameters. By applying the extracted parameters in mathematical equations, the BCS is automatically evaluated based on measurements of model accuracy, with one of the two models achieving a mean absolute percentage error (MAPE) of 3.9%, and a mean absolute error (MAE) of 0.13.

Advances in image processing technologies have provided more precise views in medical and health care management systems. Among many other topics, this paper focuses on several aspects of video-based monitoring systems for elderly people living independently. Major concerns are patients with chronic diseases and adults with a decline in physical fitness, as well as falling among elderly people, which is a source of life-threatening injuries and a leading cause of death. Therefore, in this paper, we propose a video-vision-based monitoring system using image processing technology and a Hidden Markov Model for differentiating falls from normal states for people. Specifically, the proposed system is composed of four modules: (1) object detection; (2) feature extraction; (3) analysis for differentiating normal states from falls; and (4) a decision-making process using a Hidden Markov Model for sequential states of abnormal and normal. In the object detection module, background and foreground segmentation is performed by applying the Mixture of Gaussians model, and graph cut is applied for foreground refinement. In the feature extraction module, the postures and positions of detected objects are estimated by applying the hybrid features of the virtual grounding point, inclusive of its related area and the aspect ratio of the object. In the analysis module, for differentiating normal, abnormal, or falling states, statistical computations called the moving average and modified difference are conducted, both of which are employed to estimate the points and periods of falls. Then, the local maximum or local minimum and the half width value are determined in the observed modified difference to more precisely estimate the period of a falling state. Finally, the decision-making process is conducted by developing a Hidden Markov Model. The experimental results used the Le2i fall detection dataset, and showed that our proposed system is robust and reliable and has a high detection rate.

Nowadays, for numerous reasons, smart farming systems focus on the use of image processing technologies and 5G communications. In this paper, we propose a tracking system for individual cows using an ear tag visual analysis. By using ear tags, the farmers can track specific data for individual cows such as body condition score, genetic abnormalities, etc. Specifically, a four-digit identification number is used, so that a farm can accommodate up to 9999 cows. In our proposed system, we develop an individual cow tracker to provide effective management with real-time upgrading enforcement. For this purpose, head detection is first carried out to determine the cow’s position in its related camera view. The head detection process incorporates an object detector called You Only Look Once (YOLO) and is then followed by ear tag detection. The steps involved in ear tag recognition are (1) finding the four-digit area, (2) digit segmentation using an image processing technique, and (3) ear tag recognition using a convolutional neural network (CNN) classifier. Finally, a location searching system for an individual cow is established by entering the ID numbers through the application’s user interface. The proposed searching system was confirmed by performing real-time experiments at a feeding station on a farm at Hokkaido prefecture, Japan. In combination with our decision-making process, the proposed system achieved an accuracy of 100% for head detection, and 92.5% for ear tag digit recognition. The results of using our system are very promising in terms of effectiveness.

In this paper, an innovative home care video monitoring system for detecting abnormal and normal events is proposed by introducing a virtual grounding point (VGP) concept. To be specific, the proposed system is composed of four main image processing components: (1) visual object detection, (2) feature extraction, (3) abnormal and normal event analysis, and (4) the decision-making process. In the object detection component, background subtraction is first achieved using a specific mixture of Gaussians (MoG) to model the foreground in the form of a low-rank matrix factorization. Then, a theory of graph cut is applied to refine the foreground. In the feature extraction component, the position and posture of the detected person is estimated by using a combination of the virtual grounding point, along with its related centroid, area, and aspect ratios. In analyzing the abnormal and normal events, the moving averages (MA) for the extracted features are calculated. After that, a new curve analysis is computed, specifically using the modified difference (MD). The local maximum (lmax), local minimum (lmin), and half width value (vhw) are determined on the observed curve of the modified difference. In the decision-making component, the support vector machine (SVM) method is applied to detect abnormal and normal events. In addition, a new concept called period detection (PD) is proposed to robustly detect the abnormal events. The experimental results were obtained using the Le2i fall detection dataset to confirm the reliability of the proposed method, and that it achieved a high detection rate.

Nowadays, with the emergence of sophisticated electronic devices, human daily activities are becoming more and more complex. On the other hand, research has begun on the use of reliable, cost-effective sensors, patient monitoring systems, and other systems that make daily life more comfortable for the elderly. Moreover, in the field of computer vision, human action recognition (HAR) has drawn much attention as a subject of research because of its potential for numerous cost-effective applications. Although much research has investigated the use of HAR, most has dealt with simple basic actions in a simplified environment; not much work has been done in more complex, real-world environments. Therefore, a need exists for a system that can recognize complex daily activities in a variety of realistic environments. In this paper, we propose a system for recognizing such activities, in which humans interact with various objects, taking into consideration object-oriented activity information, the use of deep convolutional neural networks, and a multi-class support vector machine (multi-class SVM). The experiments are performed on a publicly available cornell activity dataset: CAD-120 which is a dataset of human−object interactions featuring ten high-level daily activities. The outcome results show that the proposed system achieves an accuracy of 93.33%, which is higher than other state-of-the-art methods, and has great potential for applications recognizing complex daily activities.