Search

We examine the current state of the art and the related research on the automated detection and tracking of small objects—or persons—in the context of a person-overboard (POB) scenario and present the associated governing relationship between different technologies, platforms, and approaches as a system of systems. A novel phase model, structuring a POB scenario, comprises three phases: (1) detection, (2) search and track, and (3) rescue. Within these phases, we identify the central areas of responsibility and describe in detail the phases (1) and (2). We emphasize the importance of a high-level representation of different systems and their interactions to comprehensively represent the complexity and dynamics of POB scenarios. Our systematic classification and detailed description of the technologies and methods used provide valuable insights to support future regulatory and research activities. Our primary aim is to advance the development of corresponding technologies and standards. [ABSTRACT FROM AUTHOR]

The existing foreign object detection methods for belt conveyors have problems such as weak capability to extract object semantic information, poor detection precision, and only recognizing and detecting foreign objects. The methods cannot accurately calculate the number of foreign objects. In order to solve the above problems, a foreign object detection and counting method for belt conveyors based on improved YOLOv8n+DeepSORT has been designed. The method improves the YOLOv8n model and then uses the improved YOLOv8n model to recognize foreign objects in belt conveyors. The method uses the foreign object detection results of the improved YOLOv8n model as input for the DeepSORT algorithm to achieve foreign object tracking and counting on belt conveyors. YOLOv8n improvement method is replacing the C2f module in the backbone network with the C2f_MLCA module to improve the network's information extraction capability in a single color information environment. The method improves the head section using the separated and enhancement attention module (SEAM) to enhance the detection precision of foreign objects when they are obstructed. The method uses Focaler IoU optimization loss function to solve the problem of large differences in the shape of detection objects. The performance verification experiment results of MSF-YOLOv8n model show that the mAP50 of MSF-YOLOv8n model reaches 93.2%, which is 2.1% higher than the basic model. The parameter count is only 2.82×106, which is 0.19×106 less than the basic model, making it more suitable for deployment in edge devices such as inspection robots. The detection precision is 2.2%, 1.3%, and 0.3% higher than YOLOv5s, YOLOv7, and YOLOv8s algorithms, respectively. Although its frame rate is lower than YOLOv8s and YOLOv8n, it still meets the requirements of real-time video detection. The results of foreign object detection and counting experiments show that the DeepSORT algorithm has an accuracy rate of 80% and can accurately track occluded anchor rods and objects with significant shape differences.

Copyright of Przegląd Elektrotechniczny is the property of Przeglad Elektrotechniczny and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Mine Automation is the property of Industry & Mine Automation Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The concept of Image Phase Congruency (IPC) is deeply rooted in the way the human visual system interprets and processes spatial frequency information. It plays an important role in visual perception, influencing our capacity to identify objects, recognize textures, and decipher spatial relationships in our environments. IPC is robust to changes in lighting, contrast, and other variables that might modify the amplitude of light waves yet leave their relative phase unchanged. This characteristic is vital for perceptual tasks as it ensures the consistent detection of features regardless of fluctuations in illumination or other environmental factors. It can also impact cognitive and emotional responses; cohesive phase information across elements fosters a perception of unity or harmony, while inconsistencies can engender a sense of discord or tension. In this survey, we begin by examining the evidence from biological vision studies suggesting that IPC is employed by the human perceptual system. We proceed to outline the typical mathematical representation and different computational approaches to IPC. We then summarize the extensive applications of IPC in computer vision, including denoise, image quality assessment, feature detection and description, image segmentation, image registration, image fusion, and object detection, among other uses, and illustrate its advantages with a number of examples. Finally, we discuss the current challenges associated with the practical applications of IPC and potential avenues for enhancement. [ABSTRACT FROM AUTHOR]

Accurately and reliably perceiving the environment is a major challenge in autonomous driving and robotics research. Traditional vision-based methods often suffer from varying lighting conditions, occlusions, and complex environments. This paper addresses these challenges by combining a deep learning-based object detection algorithm, YOLOv8, with LiDAR data fusion technology. The principle of this combination is to merge the advantages of these technologies: YOLOv8 excels in real-time object detection and classification through RGB images, while LiDAR provides accurate distance measurement and 3D spatial information, regardless of lighting conditions. The integration aims to apply the high accuracy and robustness of YOLOv8 in identifying and classifying objects, as well as the depth data provided by LiDAR. This combination enhances the overall environmental perception, which is critical for the reliability and safety of autonomous systems. However, this fusion brings some research challenges, including data calibration between different sensors, filtering ground points from LiDAR point clouds, and managing the computational complexity of processing large datasets. This paper presents a comprehensive approach to address these challenges. Firstly, a simple algorithm is introduced to filter out ground points from LiDAR point clouds, which are essential for accurate object detection, by setting different threshold heights based on the terrain. Secondly, YOLOv8, trained on a customized dataset, is utilized for object detection in images, generating 2D bounding boxes around detected objects. Thirdly, a calibration algorithm is developed to transform 3D LiDAR coordinates to image pixel coordinates, which are vital for correlating LiDAR data with image-based object detection results. Fourthly, a method for clustering different objects based on the fused data is proposed, followed by an object tracking algorithm to compute the 3D poses of objects and their relative distances from a robot. The Agilex Scout Mini robot, equipped with Velodyne 16-channel LiDAR and an Intel D435 camera, is employed for data collection and experimentation. Finally, the experimental results validate the effectiveness of the proposed algorithms and methods. [ABSTRACT FROM AUTHOR]

Sustainable forest management systems require operational measures to preserve the functional design of forest roads. Frequent road data collection and analysis are essential to support target-oriented and efficient maintenance planning and operations. This study demonstrates an automated solution for monitoring forest road surface deterioration using consumer-grade optical sensors. A YOLOv5 model with StrongSORT tracking was adapted and trained to detect and track potholes in the videos captured by vehicle-mounted cameras. For model training, datasets recorded in diverse geographical regions under different weather conditions were used. The model shows a detection and tracking performance of up to a precision and recall level of 0.79 and 0.58, respectively, with 0.70 mean average precision at an intersection over union (IoU) of at least 0.5. We applied the trained model to a forest road in southern Norway, recorded with a Global Navigation Satellite System (GNSS)−fitted dashcam. GNSS-delivered geographical coordinates at 10 Hz rate were used to geolocate the detected potholes. The geolocation performance over this exemple road stretch of 1 km exhibited a root mean square deviation of about 9.7 m compared to OpenStreetMap. Finally, an exemple road deterioration map was compiled, which can be used for scheduling road maintenance operations. [ABSTRACT FROM AUTHOR]

We examine the current state of the art and the related research on the automated detection and tracking of small objects—or persons—in the context of a person-overboard (POB) scenario and present the associated governing relationship between different technologies, platforms, and approaches as a system of systems. A novel phase model, structuring a POB scenario, comprises three phases: (1) detection, (2) search and track, and (3) rescue. Within these phases, we identify the central areas of responsibility and describe in detail the phases (1) and (2). We emphasize the importance of a high-level representation of different systems and their interactions to comprehensively represent the complexity and dynamics of POB scenarios. Our systematic classification and detailed description of the technologies and methods used provide valuable insights to support future regulatory and research activities. Our primary aim is to advance the development of corresponding technologies and standards.

Object Detection and Tracking have provided a valuable tool for many tasks, mostly time-consuming and prone-to-error jobs, including fruit counting while in the field, among others. Fruit counting can be a challenging assignment for humans due to the large quantity of fruit available, which turns it into a mentally-taxing operation. Hence, it is relevant to use technology to ease the task of farmers by implementing Object Detection and Tracking algorithms to facilitate fruit counting. However, those algorithms suffer undercounting due to occlusion, which means that the fruit is hidden behind a leaf or a branch, complicating the detection task. Consequently, gathering the datasets from multiple viewing angles is essential to boost the likelihood of recording the images and videos from the most visible point of view. Furthermore, the most critical open-source datasets do not include labels for certain fruits, such as grape bunches. This study aims to unravel the scarcity of public datasets, including labels, to train algorithms for grape bunch Detection and Tracking by considering multiple angles acquired with a UAV to overcome fruit occlusion challenges.

This study presents a novel approach for object detection and tracking in aerial images using a multi-scale Northern Goshawk Pyramid Generative Adversarial Network (NGPGAN). The research evaluates different algorithms and features to identify people, trees, cars, and buildings in real-world drone videos, addressing challenges in pinpointing specific objects among multiple entities. Object detection and tracking are crucial tasks in various industries, prompting increased exploration of machine learning, particularly deep learning techniques. The proposed NGPGAN model integrates object detection and tracking stages, leveraging the Kalman filter with Northern Goshawk Optimization (NGO) for tracking and employing NGPGAN for detection. To enhance training stability, Northern Goshawk Optimization is utilized to optimize the generator’s cost and loss functions, mitigating issues like non-convergence and mode collapse. The study evaluates the proposed architecture’s performance using aerial drone data, focusing on efficiency and accuracy compared to existing methods.

The paper comprehensively addresses the integration of advanced technologies to enhance defense capabilities, with a particular focus on critical tasks such as object detection, tracking, and distance measurement. To this end, the integration of IMX219-77 cameras and Nvidia Jetson Nano is proposed, emphasizing the utilization of their respective features. Commonly used tools like open source computer vision (OpenCV) and GStreamer are preferred for ensuring cohesive integration between hardware and software components. On the software front, tools such as OpenCV and GStreamer are preferred for tasks related to computer vision and multimedia processing. The MOSSE algorithm is selected for object tracking due to its speed, efficiency, and resilience to changes in lighting conditions. Additionally, distance measurement is achieved through the use of Stereo Vision techniques. The results of the study demonstrate the effectiveness and accuracy of the proposed integration. It is found that accurate distance measurements with a margin of error ranging from 0 to 2 mm, falling within acceptable limits mentioned in relevant literature, can be achieved. This underscores the efficacy of the proposed technologies for tasks such as object detection, tracking, and distance measurement. The aim of the study is to conduct an in-depth examination of the integration of advanced tools such as IMX219-77 cameras and Nvidia Jetson Nano for use in defense operations. It seeks to showcase how this integration can strengthen defense strategies and provide protection against potential threats. Additionally, the study aims to lay the groundwork for ongoing innovation and development in defense technologies. In conclusion, the integration of electrooptical systems and computer vision technologies has the potential to significantly enhance defense capabilities and contribute to national security efforts. The advantages provided by this integration can serve as a valuable resource for researchers seeking to develop new solutions in defense and security domains. [ABSTRACT FROM AUTHOR]

Abstract Point cloud is an important expression form of three‐dimensional (3D) data. It has enjoyed continuous development and attracted increasing attention due to its wide applications in many areas, such as artificial intelligence, deep learning, autonomous driving and tracking. Recently, there is a large number of end‐to‐end point cloud‐based deep learning methods being proposed which are successful in the 3D domain. In order to better use point cloud data for analysis and to explore future research directions, this paper presents a comprehensive review of existing methods and publicly available datasets, with a focus on the methods and research status of using point cloud data as direct input. The background of point cloud is first introduced, including data acquisition methods, basic concepts, and challenges. Following that, the deep learning methods based on point cloud data are investigated and analysed according to classification, detection and tracking, and segmentation. Furthermore, the existing public datasets and evaluation metrics are introduced. Finally, promising research directions are proposed in conjunction with existing methods.

Point cloud is an important expression form of three‐dimensional (3D) data. It has enjoyed continuous development and attracted increasing attention due to its wide applications in many areas, such as artificial intelligence, deep learning, autonomous driving and tracking. Recently, there is a large number of end‐to‐end point cloud‐based deep learning methods being proposed which are successful in the 3D domain. In order to better use point cloud data for analysis and to explore future research directions, this paper presents a comprehensive review of existing methods and publicly available datasets, with a focus on the methods and research status of using point cloud data as direct input. The background of point cloud is first introduced, including data acquisition methods, basic concepts, and challenges. Following that, the deep learning methods based on point cloud data are investigated and analysed according to classification, detection and tracking, and segmentation. Furthermore, the existing public datasets and evaluation metrics are introduced. Finally, promising research directions are proposed in conjunction with existing methods. [ABSTRACT FROM AUTHOR]

Nowadays, deep learning (DL)-based video surveillance services are widely used in smart cities because of their ability to accurately identify and track objects, such as vehicles and pedestrians, in real time. This allows a more efficient traffic management and improved public safety. However, DL-based video surveillance services that require object movement and motion tracking (e.g., for detecting abnormal object behaviors) can consume a substantial amount of computing and memory capacity, such as (i) GPU computing resources for model inference and (ii) GPU memory resources for model loading. This paper presents a novel cognitive video surveillance management with long short-term memory (LSTM) model, denoted as the CogVSM framework. We consider DL-based video surveillance services in a hierarchical edge computing system. The proposed CogVSM forecasts object appearance patterns and smooths out the forecast results needed for an adaptive model release. Here, we aim to reduce standby GPU memory by model release while avoiding unnecessary model reloads for a sudden object appearance. CogVSM hinges on an LSTM-based deep learning architecture explicitly designed for future object appearance pattern prediction by training previous time-series patterns to achieve these objectives. By referring to the result of the LSTM-based prediction, the proposed framework controls the threshold time value in a dynamic manner by using an exponential weighted moving average (EWMA) technique. Comparative evaluations on both simulated and real-world measurement data on the commercial edge devices prove that the LSTM-based model in the CogVSM can achieve a high predictive accuracy, i.e., a root-mean-square error metric of 0.795. In addition, the suggested framework utilizes up to 32.1% less GPU memory than the baseline and 8.9% less than previous work. [ABSTRACT FROM AUTHOR]

This paper proposes a deep learning-based mmWave radar and RGB camera sensor early fusion method for object detection and tracking and its embedded system realization for ADAS applications. The proposed system can be used not only in ADAS systems but also to be applied to smart Road Side Units (RSU) in transportation systems to monitor real-time traffic flow and warn road users of probable dangerous situations. As the signals of mmWave radar are less affected by bad weather and lighting such as cloudy, sunny, snowy, night-light, and rainy days, it can work efficiently in both normal and adverse conditions. Compared to using an RGB camera alone for object detection and tracking, the early fusion of the mmWave radar and RGB camera technology can make up for the poor performance of the RGB camera when it fails due to bad weather and/or lighting conditions. The proposed method combines the features of radar and RGB cameras and directly outputs the results from an end-to-end trained deep neural network. Additionally, the complexity of the overall system is also reduced such that the proposed method can be implemented on PCs as well as on embedded systems like NVIDIA Jetson Xavier at 17.39 fps. [ABSTRACT FROM AUTHOR]

We propose HMRN, a deep heat map regression network to detect and track small moving objects in wide-area motion imagery (WAMI) by modifying a deep multi-object tracker. Object detection in WAMI images is challenging, because they cover large geographical areas and contain many small vehicles that do not have sufficient appearance-based cues for effective detection. Typically, background subtraction is applied to detect changed regions in WAMI image sequences. However, these methods suffer from high number of false detections. In this paper, we represent objects in WAMI images as heat maps and develop a deep regression network that predicts the object heat maps from current image, previous image and the predicted heat map of the previous image. Experiments are performed on Wright–Patterson Air Force Base (WPAFB) 2009 dataset and results show that the proposed method is almost ten times faster than its competitors while achieving state-of-the-art detection and tracking accuracy as well. We achieve significant reduction in false positives leading to an increase in average precision and F1 scores. [ABSTRACT FROM AUTHOR]

This work approaches an important computer vision area that is video object detection and tracking. Variational and non-variational partial differential equation (PDE)-based models for image and video object detection and tracking are surveyed here. Detection and tracking techniques based on Geometric Active Contour models, representing energy-based segmentation schemes, are presented first. The PDE-based detection and tracking geometric models using level-sets are then disscused. Moving object tracking approaches based on the optical flow estimated using PDEs are described next. Histogram-based PDE models for video tracking are then presented. Object detection techniques using PDE-based edge and contour extraction are also discussed. Our own contributions in this field, representing diffusion-based detection and tracking methods for certain object classes, are briefly presented. [ABSTRACT FROM AUTHOR]

Multi-object tracking (MOT) is an important field in computer vision that provides a critical understanding of video analysis in various applications, such as vehicle tracking in intelligent transportation systems (ITS). Several deep learning-based approaches have been introduced to basic motion and IoU trackers by extracting appearance features to assist in challenging situations such as lossy detection and occlusion. This study proposes a portable appearance extension (PAE) for single-stage object detection to jointly detect and extract appearance embeddings using a shared model. Furthermore, a novel training framework with a single image and without re-identification annotations is presented using an augmentation module, saving a tremendous amount of human labeling effort and increasing the real-world application adoption rate. Using UA-DETRAC dataset, RetinaNet-PAE and SSD-PAE achieve comparable results with current state-of-the-art models, where RetinaNet-PAE prioritizes detection and tracking performance with a 58.0% HOTA score and 4 FPS. In contrast, SSD-PAE prioritizes latency performance with a 47.3% HOTA score and 40 FPS.

Abstract Accurate and prompt traffic data are necessary for the successful management of major events. Computer vision techniques, such as convolutional neural network (CNN) applied on video monitoring data, can provide a cost-efficient and timely alternative to traditional data collection and analysis methods. This paper presents a framework designed to take videos as input and output traffic volume counts and intersection turning patterns. This framework comprises a CNN model and an object tracking algorithm to detect and track vehicles in the camera’s pixel view first. Homographic projection then maps vehicle spatial-temporal information (including unique ID, location, and timestamp) onto an orthogonal real-scale map, from which the traffic counts and turns are computed. Several video data are manually labeled and compared with the framework output. The following results show a robust traffic volume count accuracy up to 96.91%. Moreover, this work investigates the performance influencing factors including lighting condition (over a 24-h-period), pixel size, and camera angle. Based on the analysis, it is suggested to place cameras such that detection pixel size is above 2343 and the view angle is below 22°, for more accurate counts. Next, previous and current traffic reports after Texas A&M home football games are compared with the framework output. Results suggest that the proposed framework is able to reproduce traffic volume change trends for different traffic directions. Lastly, this work also contributes a new intersection turning pattern, i.e., counts for each ingress-egress edge pair, with its optimization technique which result in an accuracy between 43% and 72%.

Event-based vision is an emerging field of computer vision that offers unique properties, such as asynchronous visual output, high temporal resolutions, and dependence on brightness changes, to generate data. These properties can enable robust high-temporal-resolution object detection and tracking when combined with frame-based vision. In this paper, we present a hybrid, high-temporal-resolution object detection and tracking approach that combines learned and classical methods using synchronized images and event data. Off-the-shelf frame-based object detectors are used for initial object detection and classification. Then, event masks, generated per detection, are used to enable inter-frame tracking at varying temporal resolutions using the event data. Detections are associated across time using a simple, low-cost association metric. Moreover, we collect and label a traffic dataset using the hybrid sensor DAVIS 240c. This dataset is utilized for quantitative evaluation using state-of-the-art detection and tracking metrics. We provide ground truth bounding boxes and object IDs for each vehicle annotation. Further, we generate high-temporal-resolution ground truth data to analyze tracking performance at different temporal rates. Our approach shows promising results, with minimal performance deterioration at higher temporal resolutions (48–384 Hz) when compared with the baseline frame-based performance at 24 Hz. [ABSTRACT FROM AUTHOR]

Abstract Background With recent advances in microscopy, recordings of cell behaviour can result in terabyte-size datasets. The lattice light sheet microscope (LLSM) images cells at high speed and high 3D resolution, accumulating data at 100 frames/second over hours, presenting a major challenge for interrogating these datasets. The surfaces of vertebrate cells can rapidly deform to create projections that interact with the microenvironment. Such surface projections include spike-like filopodia and wave-like ruffles on the surface of macrophages as they engage in immune surveillance. LLSM imaging has provided new insights into the complex surface behaviours of immune cells, including revealing new types of ruffles. However, full use of these data requires systematic and quantitative analysis of thousands of projections over hundreds of time steps, and an effective system for analysis of individual structures at this scale requires efficient and robust methods with minimal user intervention. Results We present LLAMA, a platform to enable systematic analysis of terabyte-scale 4D microscopy datasets. We use a machine learning method for semantic segmentation, followed by a robust and configurable object separation and tracking algorithm, generating detailed object level statistics. Our system is designed to run on high-performance computing to achieve high throughput, with outputs suitable for visualisation and statistical analysis. Advanced visualisation is a key element of LLAMA: we provide a specialised tool which supports interactive quality control, optimisation, and output visualisation processes to complement the processing pipeline. LLAMA is demonstrated in an analysis of macrophage surface projections, in which it is used to i) discriminate ruffles induced by lipopolysaccharide (LPS) and macrophage colony stimulating factor (CSF-1) and ii) determine the autonomy of ruffle morphologies. Conclusions LLAMA provides an effective open source tool for running a cell microscopy analysis pipeline based on semantic segmentation, object analysis and tracking. Detailed numerical and visual outputs enable effective statistical analysis, identifying distinct patterns of increased activity under the two interventions considered in our example analysis. Our system provides the capacity to screen large datasets for specific structural configurations. LLAMA identified distinct features of LPS and CSF-1 induced ruffles and it identified a continuity of behaviour between tent pole ruffling, wave-like ruffling and filopodia deployment.

This paper proposes a new 3D multi-object tracker to more robustly track objects that are temporarily missed by detectors. Our tracker can better leverage object features for 3D Multi-Object Tracking (MOT) in point clouds. The proposed tracker is based on a novel data association scheme guided by prediction confidence, and it consists of two key parts. First, we design a new predictor that employs a constant acceleration (CA) motion model to estimate future positions, and outputs a prediction confidence to guide data association through increased awareness of detection quality. Second, we introduce a new aggregated pairwise cost to exploit features of objects in point clouds for faster and more accurate data association. The proposed cost consists of geometry, appearance and motion components. Specifically, we formulate the geometry cost using resolutions (lengths, widths and heights), centroids, and orientations of 3D bounding boxes (BBs), the appearance cost using appearance features from the deep learning-based detector backbone network, and the motion cost by associating different motion vectors. Extensive multi-object tracking experiments on the KITTI tracking benchmark demonstrated that our method outperforms, by a large margin, the state-of-the-art methods in both tracking accuracy and speed. [ABSTRACT FROM AUTHOR]

In this paper, a real-time road-object detection and tracking (LR_ODT) method for autonomous driving is proposed. The method is based on the fusion of lidar and radar measurement data, where they are installed on the ego car, and a customized unscented Kalman filter (UKF) is employed for their data fusion. The merits of both devices are combined using the proposed fusion approach to precisely provide both pose and velocity information for objects moving in roads around the ego car. Unlike other detection and tracking approaches, the balanced treatment of both pose estimation accuracy and its real-time performance is the main contribution in this work. The proposed technique is implemented using the high-performance language C++ and utilizes highly optimized math and optimization libraries for best real-time performance. Simulation studies have been carried out to evaluate the performance of the LR_ODT for tracking bicycles, cars, and pedestrians. Moreover, the performance of the UKF fusion is compared to that of the extended Kalman filter fusion (EKF) showing its superiority. The UKF has outperformed the EKF on all test cases and all the state variable levels (-24% average RMSE). The employed fusion technique shows how outstanding is the improvement in tracking performance compared to the use of a single device (-29% RMES with lidar and -38% RMSE with radar). [ABSTRACT FROM AUTHOR]

Small-size object detection (SOD) is one of the challenging problems in computer vision applications. SOD is highly useful in defense, military, surveillance, medical, industrial and analysis in sports applications. Various algorithms were developed in the past to solve the problem of SOD. However, the algorithms developed are not suitable for real-time applications. In this work, a convolutional neural network architecture based on YOLO is proposed to enhance small objects' detection performance. The proposed network is inspired by the ideas of Residual blocks, Densenet, Feature Pyramidal Network, Cross stage partial connections, and 1 × 1 convolutions. The Receptive field and the reuse of feature maps are the main factors in the design of the architecture and is hence referred to as RFSOD. It is developed as a lightweight network to suit real-time applications and can run smoothly on single-board computers such as Jetson Nano, Tx2, Raspberry Pi and the like. The proposed model is evaluated on various public datasets such as VHR10, BCCD dataset and few small-size objects from the MS COCO dataset. This work is motivated by the need to develop a vision system for a badminton-playing robot. Therefore, the proposed model is also tested on a custom-made shuttlecock dataset. The model's performance is compared with the state-of-the-art deep learning models that are suitable for real-time applications. The hardware implementation of the proposed model was carried out on Jetson Nano, Raspberry Pi4 and a Laptop with an i5 processor. Improved Detection accuracy was observed on small objects. More than 2 × detection speed was obtained on Raspberry Pi, and i5 processor while 30% improvement was observed on Jetson Nano with real-time videos. [ABSTRACT FROM AUTHOR]

The location information of entities in construction sites, such as workers and construction machines, is valuable in project management and safety. Therefore, as nonintrusive and accurate solutions, various vision-based methods have been proposed to track entities in construction sites and obtain their three-dimensional (3D) coordinates. However, most existing vision-based methods realize 3D localizations by basing entity matching on the epipolar line, which brings instability in entity matching due to the calculation error of the epipolar line or failure to match entities when multiple entities are located on the same epipolar. To solve this problem, a novel framework based on reidentification is proposed to automatically match workers across two camera views, thereby obtaining their 3D coordinates in construction sites. In this framework, deep-learning-based computer vision algorithms are firstly used to detect and track workers in two camera views. Then, the reidentification (ReID) algorithm is applied to utilize tracked workers' visual features to match the workers across both two camera views and different frames. As a result, for every matched pair, the worker's pixel locations in two camera views can be obtained to calculate the 3D coordinates through triangulation. The implementation of videos recorded from a real construction project proves the feasibility and accuracy of this framework. Specifically, through utilizing the ReID algorithm to match workers, the framework achieves competitive results on workers matching with precision, recall, and accuracy of more than 99%, 93%, and 93%. Furthermore, it also effectively addresses the practical problems of ID repetition and ID switching. Meanwhile, this paper extends the application scenarios of reidentification algorithms in construction sites, thereby contributing to the future application of multiple-camera vision-based methods in construction sites. [ABSTRACT FROM AUTHOR]

Background: With recent advances in microscopy, recordings of cell behaviour can result in terabyte-size datasets. The lattice light sheet microscope (LLSM) images cells at high speed and high 3D resolution, accumulating data at 100 frames/second over hours, presenting a major challenge for interrogating these datasets. The surfaces of vertebrate cells can rapidly deform to create projections that interact with the microenvironment. Such surface projections include spike-like filopodia and wave-like ruffles on the surface of macrophages as they engage in immune surveillance. LLSM imaging has provided new insights into the complex surface behaviours of immune cells, including revealing new types of ruffles. However, full use of these data requires systematic and quantitative analysis of thousands of projections over hundreds of time steps, and an effective system for analysis of individual structures at this scale requires efficient and robust methods with minimal user intervention. Results: We present LLAMA, a platform to enable systematic analysis of terabyte-scale 4D microscopy datasets. We use a machine learning method for semantic segmentation, followed by a robust and configurable object separation and tracking algorithm, generating detailed object level statistics. Our system is designed to run on high-performance computing to achieve high throughput, with outputs suitable for visualisation and statistical analysis. Advanced visualisation is a key element of LLAMA: we provide a specialised tool which supports interactive quality control, optimisation, and output visualisation processes to complement the processing pipeline. LLAMA is demonstrated in an analysis of macrophage surface projections, in which it is used to i) discriminate ruffles induced by lipopolysaccharide (LPS) and macrophage colony stimulating factor (CSF-1) and ii) determine the autonomy of ruffle morphologies. Conclusions: LLAMA provides an effective open source tool for running a cell microscopy analysis pipeline based on semantic segmentation, object analysis and tracking. Detailed numerical and visual outputs enable effective statistical analysis, identifying distinct patterns of increased activity under the two interventions considered in our example analysis. Our system provides the capacity to screen large datasets for specific structural configurations. LLAMA identified distinct features of LPS and CSF-1 induced ruffles and it identified a continuity of behaviour between tent pole ruffling, wave-like ruffling and filopodia deployment. [ABSTRACT FROM AUTHOR]

In this article, a real-time road-Object Detection and Tracking (LR_ODT) method for autonomous driving is proposed. This method is based on the fusion of lidar and radar measurement data, where they are installed on the ego car, and a customized Unscented Kalman Filter is employed for their data fusion. The merits of both devices are combined using the proposed fusion approach to precisely provide both pose and velocity information for objects moving in roads around the ego car. Unlike other detection and tracking approaches, the balanced treatment of both pose estimation accuracy and its real-time performance is the main contribution in this work. The proposed technique is implemented using the high-performance language C++ and utilizes highly optimized math and optimization libraries for best real-time performance. Simulation studies have been carried out to evaluate the performance of the LR_ODT for tracking bicycles, cars, and pedestrians. Moreover, the performance of the Unscented Kalman Filter fusion is compared to that of the Extended Kalman Filter fusion showing its superiority. The Unscented Kalman Filter has outperformed the Extended Kalman Filter on all test cases and all the state variable levels (−24% average Root Mean Squared Error). The employed fusion technique shows how outstanding is the improvement in tracking performance compared to the use of a single device (−29% Root Mean Squared Error with lidar and −38% Root Mean Squared Error with radar). [ABSTRACT FROM AUTHOR]

• A state-of-the-art AI-based method was proposed for the detection and tracking of condensing bubbles within subcooled flow boiling. • Approximately 90% of condensing bubbles were successfully detected by the proposed AI model, demonstrating its remarkable accuracy in capturing bubble dynamics. • Critical thermal-hydraulic parameters were effectively extracted and validated against empirical correlations, showcasing the AI model's accuracy and potential for broader application in complex fluid dynamics analysis. Subcooled flow boiling is a pivotal process prevalent in a myriad of scientific investigations and engineering applications, particularly in the realm of heat transfer system design and the foundational study of phase transition dynamics. The life cycle of bubbles, from nucleation and growth to departure and coalescence, along with their interaction with heat and mass transfer processes, critically influence the overall heat transfer efficiency. Nonetheless, the drastic transformations that bubbles undergo from inception to disappearance in subcooled flow boiling pose significant challenges for conventional bubble detection methods, particularly concerning condensing bubbles. In light of this, a cutting-edge AI-based method for condensing bubble detection and tracking in subcooled flow boiling is developed and validated in the present study. The present approach first identifies bubbles using object detection technique and subsequently tracks them across sequential frames. The proposed method demonstrates a robust capability of detecting approximately 90% of condensing bubbles within subcooled flow boiling. Furthermore, key thermal-hydraulic parameters in subcooled flow boiling such as aspect ratio, Sauter mean diameter, departure diameter, growth time, and bubble lifetime, were successfully extracted using the proposed AI-based model. Its results are compared with empirical correlations, and show a commendable consistency, demonstrating the viability and accuracy of the advanced AI-based model in analyzing the complex dynamics of subcooled flow boiling. The advantage of the newly developed method is preliminarily verified in the present study, and further validation is underway to corroborate its boarded application. [ABSTRACT FROM AUTHOR]

The usage of transportation systems is inevitable; any assistance module which can catalyze the flow involved in transportation systems, parallelly improving the reliability of processes involved is a boon for day-to-day human lives. This paper introduces a novel, cost-effective, and highly responsive Post-active Driving Assistance System, which is "Adaptive-Mask-Modelling Driving Assistance System" with intuitive wide field-of-view modeling architecture. The proposed system is a vision-based approach, which processes a panoramic-front view (stitched from temporal synchronous left, right stereo camera feed) & simple monocular-rear view to generate robust & reliable proximity triggers along with co-relative navigation suggestions. The proposed system generates robust objects, adaptive field-of-view masks using FRCNN+Resnet-101_FPN, DSED neural-networks, and are later processed and mutually analyzed at respective stages to trigger proximity alerts and frame reliable navigation suggestions. The proposed DSED network is an Encoder-Decoder-Convolutional-Neural-Network to estimate lane-offset parameters which are responsible for adaptive modeling of field-of-view range (157o-210o) during live inference. Proposed stages, deep-neural-networks, and implemented algorithms, modules are state-of-the-art and achieved outstanding performance with minimal loss(L{p, t}, Lδ, LTotal) values during benchmarking analysis on our custombuilt, KITTI, MS-COCO, Pascal-VOC, Make-3D datasets. The proposed assistance-system is tested on our custom-built, multiple public datasets to generalize its reliability and robustness under multiple wild conditions, input traffic scenarios & locations.

These days surveillance cameras are spreading very fast for security issues; however reviewing, retrieving and analyzing all these surveillance videos consume a lot of time. Video synopsis technology solved this problem by extracting all the active objects tubes that occurred at different times and relocating these objects simultaneously in a video for fast reviewing. Rearranging objects tubes in the video is considered the main challenge in creating video synopsis. Conventional methods proposed different approaches to optimize the energy function for relocating the object, but it suffers from high computational complexity and time-consuming. Moreover, some of these methods could not save the chronological order of moving objects. In this paper, the particle swarm algorithm is proposed for the first time to solve the energy minimization function and rearrange the objects to generate a condensed synopsis video with less collision and in chronological order. The experiments are applied to a benchmark dataset (VIRAT), and the preliminary simulation results demonstrate that the proposed method outperforms the genetic algorithm. [ABSTRACT FROM AUTHOR]

Event-based vision is an emerging field of computer vision that offers unique properties, such as asynchronous visual output, high temporal resolutions, and dependence on brightness changes, to generate data. These properties can enable robust high-temporal-resolution object detection and tracking when combined with frame-based vision. In this paper, we present a hybrid, high-temporal-resolution object detection and tracking approach that combines learned and classical methods using synchronized images and event data. Off-the-shelf frame-based object detectors are used for initial object detection and classification. Then, event masks, generated per detection, are used to enable inter-frame tracking at varying temporal resolutions using the event data. Detections are associated across time using a simple, low-cost association metric. Moreover, we collect and label a traffic dataset using the hybrid sensor DAVIS 240c. This dataset is utilized for quantitative evaluation using state-of-the-art detection and tracking metrics. We provide ground truth bounding boxes and object IDs for each vehicle annotation. Further, we generate high-temporal-resolution ground truth data to analyze tracking performance at different temporal rates. Our approach shows promising results, with minimal performance deterioration at higher temporal resolutions (48–384 Hz) when compared with the baseline frame-based performance at 24 Hz.

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Most traffic surveillance systems are based on videos which captured by fixed cameras on bridges, intersections, etc. However, many traffic collisions may occur in many places without such surveillance systems, e.g., in rural highway. Researchers have developed a set of techniques to improve safety on these places, while it is still not enough to reduce collision risk. Based on a novel concept, this paper proposes a traffic collisions early warning scheme aided by small unmanned aerial vehicle (UAV) companion. Basically, it is a vision-based driver assistance system, and the difference in comparison with the available schemes lies in the camera is flying along with the host vehicle. In particular, the system's framework and the vision-based vehicle collision detection algorithm are proposed. The small UAV works in two switchable modes, i.e., high speed flight or low speed motion. The high speed flight corresponds to the host vehicle moving in highway, while the low speed motion includes hover, vertical takeoff and landing. In addition, as the on-line machine learning is applied, the detection procedure can be implemented in real-time, which is critical in practical applications. Extensive experimental results and examples demonstrate the effectiveness of the proposed method, and its real-time performance outperforms typical tracking methods such as that based on Gaussian mixture model. Moreover, this scheme can be easily extended for some other similar application scenarios. [ABSTRACT FROM AUTHOR]

Object detection and tracking is one of the most relevant computer technologies related to computer vision and image processing. It may mean the detection of an object within a frame and classify it (human, animal, vehicle, building, etc) by the use of some algorithms. It may also be the detection of a reference object within different frames (under different angles, different scales, etc.). The applications of the object detection and tracking are numerous; most of them are in the security field. It is also used in our daily life applications, especially in developing and enhancing business management. Inventory or stock management is one of these applications. It is considered to be an important process in warehousing and storage business because it allows for stock in and stock out products control. The stock-out situation, however, is a very serious issue that can be detrimental to the bottom line of any business. It causes an increased risk of lost sales as well as it leads to reduced customer satisfaction and lowered loyalty levels. On this note, a smart solution for stockout detection in warehouses is proposed in this paper, to automate the process using inventory management software. The proposed method is a machine learning based real-time notification system using the exciting Scale Invariant Feature Transform feature detector (SIFT) and Random Sample Consensus (RANSAC) algorithms. Consequently, the comparative study shows the overall good performance of the system achieving 100% detection accuracy with features' rich model and 90% detection accuracy with features' poor model, indicating the viability of the proposed solution. [ABSTRACT FROM AUTHOR]