Search

Deep learning-based object detection (DLOD) methods are crucial in assisting dynamic construction site management (DCSM) but have not been systematically studied. This paper investigates DLOD methods through a mixed-methods systematic review concerning concept, methodology, and value dimensions. This review analyses 110 DLOD cases identified from 103 journal papers published during 2014–2023 by examining "what are the main targets and focuses", "what algorithms and techniques are adopted", and "how do DLOD methods perform in practical application scenarios of DCSM." Threefold knowledge gaps are identified as multifaceted concept with singularity and oversimplification, progressive methodology with a weak data foundation, and deterministic value with subjective evaluation and limited scenarios. Three directions are proposed for developing advanced DLOD methods: develop large-scale, high-quality, open-source construction datasets, adopt advanced algorithms, and explore integrated DCSM scenarios. This paper marks a milestone of DLOD research and sheds light on its future development for DCSM. [Display omitted] • Deep learning-based object detection (DLOD) for onsite construction is systematically examined. • 110 cases of DLOD methods are analysed in the dimensions of concept, methodology, and value. • Threefold knowledge gaps in the status quo of DLOD methods are identified. • Three research directions are proposed for developing advanced DLOD methods. • Developing large-scale, high-quality, open-source construction datasets is vital and urgent. [ABSTRACT FROM AUTHOR]

Machine learning algorithms especially deep neural networks have seen tremendous growth in their real-world deployment. While these algorithms have to yield high performances for the task at hand, it is of equal importance that they are robust against the different kinds of singularities or adversarial anomalies. Researchers have generally addressed singularity detection independently for each anomaly, however, for the algorithms to be effective in the real world, the singularity detection algorithm must be able to handle a wide variety of attacks whether trained individually on them or not seen at the time of training. With this objective, in this paper, we propose a unique end-to-end transformation domain network to detect a variety of well-known attacks. The proposed architecture utilizes a combination of two different wavelet transformations to simultaneously learn low-level and high-level image features in a deep stacked layer fashion. The proposed method is generalized to handle a broad set of singularities of several computer vision algorithms whether operating in the object recognition domain or biometrics recognition. We showcase the results on a wide range of singularity points including (i) adversarial perturbations which are learned using deep learning networks, (ii) physical presentation attacks on face recognition which aim to produce fake data for the sensor for acquisition, and identification, (iii) synthetic images, and (iv) digital retouching of the images. Even with a zero-day or open-world attack setting, the results of the proposed algorithm show improvement over state-of-the-art results. The proposed architecture is agnostic to parameters, computationally efficient to provide a sustainable detector to resource-constrained institutions, provides robustness to attacks, and supports green computing. • A novel parameter-agnostic architecture is proposed to defend anomalies. • The proposed algorithm can handle adversaries in seen, unseen, and open-set settings. • The proposed algorithm is domain agnostic i.e., can handle vision and face datasets. • Our unified algorithm surpasses the SOTA works developed to handle specific attacks. • Bring a computationally feasible algorithm, it can be deployed on mobile devices. [ABSTRACT FROM AUTHOR]

Recently, super-resolution (SR) techniques based on deep learning have taken more and more attention, aiming to improve the images and videos resolutions. Most of the SR methods are related to other fields of computer vision such as image classification, image segmentation, and object detection. Based on the success of the image SR task, many image SR surveys are introduced to summarize the recent work in the image SR domains. However, there is no survey to summarize the SR models for the lightweight image SR domain. In this paper, we present a comprehensive survey of the state-of-the-art lightweight SR models based on deep learning. The SR techniques are grouped into six major categories: include convolution, residual, dense, distillation, attention, and extremely lightweight based models. Also, we cover some other issues related to the SR task, such as benchmark datasets and metrics for performance evaluation. Finally, we discuss some future directions and open problems, that may help other community researchers in the future. [Display omitted] • Lightweight image super-resolution based on deep learning models are reviewed. • The network design, loss function, framework, and training datasets are considered. • The network design structure of the reviewed models are categorized into six types. • The future research directions of lightweight image super-resolution are indicated. [ABSTRACT FROM AUTHOR]

In the realm of video surveillance systems, pedestrian detection (PD) is a crucial task that enables the identification and tracking of individuals, crowd monitoring, and people counting. However, accurate PD is a challenging task due to variations in the appearance and pose of pedestrians. With the advent of deep learning (DL) techniques, particularly convolutional neural networks (CNNs), computer vision (CV) methods for object classification and detection have been revolutionized. Effective PD depends on supervised learning, which requires large labeled datasets. This paper, we present pedestrian detection (PD) using a Robust Multi-modal Pedestrian Detection using a Deep Convolutional Neural Network with an Ensemble Learning (RMPD-DCNN-EL) model. This model effects computer vision and ensemble learning to accomplish precise pedestrian detection. The SimAM EfficientNet model with a stochastic gradient descent (SGD) optimizer is used for feature extraction, while ensemble classification is performed using three DL models are Nested Long Short-Term Memory (NLSTM), deep belief networks (DBNs), and extreme learning machine (ELM). To evaluate the effectiveness of tettapproach, we use the INRIA dataset, and the experimental results prove the greater performance of RMPD-DCNN-EL over other DL methods. The proposed method enables robust pedestrian detection despite modifications in arrival and pose, making it an operative solution for video surveillance systems in smart transport environments. Experimental results show that the detection accuracy is 99.30% of the proposed algorithm is considerably higher than that of the traditional convolutional neural network algorithm. [ABSTRACT FROM AUTHOR]

In this paper, machine learning (ML)/deep learning (DL) has been employed to predict the thermal runaway (TR) occurrence in Li-ion batteries (LIBs). State-of-the-art convolutional neural networks (CNNs) are adopted to forecast the evolution of the TR phenomenon with a classification approach of three distinct stages, namely, safe operation, critical condition of triggered the TR, and the actual TR occurrence. In addition, the "you look only once" (YOLO) object detection ML model is used to identify the location of the TR heat sources within the battery. The neural networks are trained on simulated thermal images, computed with the multiphysics modeling of LIBs. The multiphysics modeling approach comprises a coupled thermal, electrochemical (P2D model), and degradation sub-models. The degradation phenomenon leading to the TR considered in this paper is the solid electrolyte interface (SEI) formation/decomposition on the negative electrode. The proposed ML model exhibits high accuracy in predicting the TR evolution stages and heat source locations. The combined multiphysics and ML modeling approach developed in this work provides qualitative insights for 'on-the-fly' prediction of the TR, as well as a framework for extending data-driven methodologies to broad applications in electrochemistry. • Multiphysics modeling of thermal runaway (TR). • Simulated battery thermal imaging for training machine learning models for TR detection. • Multiclass classification of TR evolution: safe, critical, and TR stages. • Real-time detection of TR heat sources using object detection. [ABSTRACT FROM AUTHOR]

The long-term accumulation of ammonia nitrogen in aquaculture seriously affects the life of fish and even causes large-scale death. Moreover, when the concentration of ammonia nitrogen starts to accumulate, it is a judgment standard to provide early warning through the changes in fish behavior to prevent excessive ammonia nitrogen in water. Therefore, this paper proposes a novel approach to monitoring water quality for aquaculture based on deep learning and three-dimensional movement trajectory. The improved YOLOv8 model was used as the object detection approach to obtain three-dimensional position information of fish by combining Kalman filter, Kuhn Munkres (KM) algorithm, and Kernelized Correlation Filters (KCF) algorithm. The proposed approach was evaluated in the recovery experiment of acute ammonia nitrogen stress of sturgeon, bass, and crucian. The experimental results show that the precision, recall, mAP@0.5 , and mAP@0.5:0.95 of the improved YOLOv8 model are 0.964, 0.914, 0.979, and 0.602, respectively. In addition, the proposed three-dimensional positioning approach can qualitatively and quantitatively analyze the fish behavior in different stages and further explores the fish behavior changes through behavior trajectories, volumes of exercise, spatial distribution, and movement velocity. This research provides a new method and idea for studying the abnormal behavior of aquatic animals under ammonia nitrogen stress and has theoretical and practical significance. • An object detection method combining three-dimensional position information is proposed. • This paper proposes improved YOLOv8 approach combined with multi head self-attention and Wise-IoU. • Quantitative analysis is conducted on the behavioral changes of fish under ammonia nitrogen stress. • This research provides a new method and idea for studying the abnormal behavior of aquatic animals. [ABSTRACT FROM AUTHOR]

Outdoor point-cloud object localization is an essential processing step for urban scene analysis and modeling in numerous applications, especially in land surveying and site analysis. Given the increasingly use of Terrestrial Laser Scanning (TLS) and LiDAR 3D data acquisition, numerous annotated point-cloud datasets are available and can be used to evaluate computer vision and machine learning based algorithms. Nevertheless, current point-cloud datasets mainly focus on object detection and classification in autonomous driving or urban planning type of applications, and share redundant object classes, e.g., trees, vehicles and pedestrians, which limit their usefulness for land surveying and site analysis. This paper introduces a novel 3D benchmark dataset LiSurveying, which is a large-scale point-cloud dataset with over a billion points and uncommon urban object categories in complex outdoor environments. Our dataset incorporates more urban object classes than existing datasets. Its instances have diverse point densities, shapes and dimensions, which also impose a challenge for point-cloud detection and classification algorithms. We conducted baselines experiments for point-cloud classification using machine learning classifiers and deep learning methods on different subsets of the LiSurveying dataset, and we are able to demonstrate that the various types of object classes, number of instances per class, distribution of object points, and variety of complex scenes, make this LiSurveying benchmark dataset suitable for evaluating 3D point-cloud classification, semantic segmentation, and object detection algorithms. [Display omitted] • Large-scale (point-wise and 3D bounding boxes) TLS-LiDAR dataset for Land Surveying. • 3D Urban benchmark for object classification y detection. • Learning-based methods applied on complex urban environments. • Comparative performance of existing algorithms for LiDAR classification. [ABSTRACT FROM AUTHOR]

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work. [Display omitted] [ABSTRACT FROM AUTHOR]

Local feature matching enjoys wide-ranging applications in the realm of computer vision, encompassing domains such as image retrieval, 3D reconstruction, and object recognition. However, challenges persist in improving the accuracy and robustness of matching due to factors like viewpoint and lighting variations. In recent years, the introduction of deep learning models has sparked widespread exploration into local feature matching techniques. The objective of this endeavor is to furnish a comprehensive overview of local feature matching methods. These methods are categorized into two key segments based on the presence of detectors. The Detector-based category encompasses models inclusive of Detect-then-Describe, Joint Detection and Description, Describe-then-Detect, as well as Graph Based techniques. In contrast, the Detector-free category comprises CNN Based, Transformer Based, and Patch Based methods. Our study extends beyond methodological analysis, incorporating evaluations of prevalent datasets and metrics to facilitate a quantitative comparison of state-of-the-art techniques. The paper also explores the practical application of local feature matching in diverse domains such as Structure from Motion, Remote Sensing Image Registration, and Medical Image Registration, underscoring its versatility and significance across various fields. Ultimately, we endeavor to outline the current challenges faced in this domain and furnish future research directions, thereby serving as a reference for researchers involved in local feature matching and its interconnected domains. A comprehensive list of studies in this survey is available at https://github.com/vignywang/Awesome-Local-Feature-Matching. • This endeavor is to furnish a comprehensive overview of local feature matching methods. • We analyze various metrics used for performance evaluation of key local feature matching methods. • We endeavor to outline the current challenges faced in this domain and furnish future research directions. [ABSTRACT FROM AUTHOR]

Object detection as a subfield within computer vision has achieved remarkable progress, which aims to accurately identify and locate a specific object from images or videos. Such methods rely on large-scale labeled training samples for each object category to ensure accurate detection, but obtaining extensive annotated data is a labor-intensive and expensive process in many real-world scenarios. To tackle this challenge, researchers have explored few-shot object detection (FSOD) that combines few-shot learning and object detection techniques to rapidly adapt to novel objects with limited annotated samples. This paper presents a comprehensive survey to review the significant advancements in the field of FSOD in recent years and summarize the existing challenges and solutions. Specifically, we first introduce the background and definition of FSOD to emphasize potential value in advancing the field of computer vision. We then propose a novel FSOD taxonomy method and survey the plentifully remarkable FSOD algorithms based on this fact to report a comprehensive overview that facilitates a deeper understanding of the FSOD problem and the development of innovative solutions. Finally, we discuss the advantages and limitations of these algorithms to summarize the challenges, potential research direction, and development trend of object detection in the data scarcity scenario. • A novel taxonomy scheme for the existing few-shot object detection techniques. • A comprehensive review of few-shot object detection techniques for a deeper understanding. • Quantitative analysis of selected methods for challenges and potential research direction. [ABSTRACT FROM AUTHOR]

Rising global fire incidents necessitate effective solutions, with forest surveillance emerging as a crucial strategy. This paper proposes a complete solution using technology that integrates visible and infrared spectrum images through Unmanned Aerial Vehicles (UAVs) for enhanced detection of people and vehicles in forest environments. Unlike existing computer vision models relying on single-sensor imagery, this approach overcomes limitations posed by limited spectrum coverage, particularly addressing challenges in low-light conditions, fog, or smoke. The developed 4-channel model uses both types of images to take advantage of the strengths of each one simultaneously. This article presents the development and implementation of a solution for forest monitoring ranging from the transmission of images captured by a UAV to their analysis with an object detection model without human intervention. This model consists of a new version of the YOLOv5 (You Only Look Once) architecture. After the model analyzes the images, the results can be observed on a web platform on any device, anywhere in the world. For the model training, a dataset with thermal and visible images from the aerial perspective was captured with a UAV. From the development of this proposal, a new 4-channel model was created, presenting a substantial increase in precision and mAP (Mean Average Precision) metrics compared to traditional SOTA (state-of-the-art) models that only make use of red, green, and blue (RGB) images. Allied with the increase in precision, we confirmed the hypothesis that our model would perform better in conditions unfavorable to RGB images, identifying objects in situations with low light and reduced visibility with partial occlusions. With the model's training using our dataset, we observed a significant increase in the model's performance for images in the aerial perspective. This study introduces a modular system architecture featuring key modules: multisensor image capture, transmission, processing, analysis, and results presentation. Powered by an innovative object detection deep-learning model, these components collaborate to enable real-time, efficient, and distributed forest monitoring across diverse environments. [Display omitted] • Real-time detection using UAVs for people and cars in forests based on deep learning. • 4-channel object detection model with RGB and IR, for low-visibility environments. • A new annotated and aligned image dataset with four channels (RGB and IR). • A system architecture for transmitting images between client (UAV) and server. • Web platform for real-time detection visualization across multiple devices. [ABSTRACT FROM AUTHOR]

Recent advancements in computer vision and augmented reality (AR) technology have created a pathway for enhanced human–computer interaction. Despite the potential benefits, the integration of these technologies is not yet common in the industry. Object detection is the foundation of AR systems for inspection, and an efficient object detection model will enable accurate object localization in AR devices. However, the limited computing capabilities and memory usage of wearable AR devices pose a significant challenge to the deployment of state-of-the-art detectors on these devices in real work scenarios. Moreover, as the person wearing the AR device keeps moving, the constantly changing scale of the captured image often leads to significant information loss when feature maps of different scales are fused. To address these challenges, this paper proposes a new compatible detector YOLO-Master based on improved YOLOv5 through deep learning and AR. The paper introduces several improvements, such as using MobileNetv3 as the backbone to reduce memory overhead and computational costs and designing an attention module MCA based on coordinate to reduce information loss. Besides, 20 hyperparameters of YOLOv5 are optimized by the genetic algorithm(GA) before model training, which can help reduces the loss value generated. YOLO-Master was trained and tested on hydropower device images collected from a power plant in the China Southern Power Grid, and the experiments on a self-made dataset demonstrated that the mAP of the proposed method can reach 70.1%, which is 2.9% higher than the baseline of 67.2%. Additionally, the number of parameters of the model was reduced by 66.7%, enabling the model to ensure accuracy while being compatible with real-time performance, meeting the urgent needs of engineering deployment. • A Compatible detector is proposed for device detection in AR inspection system. • Computation cost is improved by a lightweight backbone network. • A coordinate-based attention module is designed for effective feature fusion. • Optimizing hyperparameters of the new architecture to make the detector robust. [ABSTRACT FROM AUTHOR]

Object detection studies aim to solve safety problems at industrial sites; however, improving object detection performance and ensuring real-time capabilities simultaneously remains challenging in multi-complex industrial environments. This paper proposes an unconditionally protected detector to address this problem. The detector's backbone network uses a residual block with a bottleneck structure to reduce computation and enhance real-time performance. Additionally, a selective attention network enhances object detection by extracting important features based on the morphological characteristics of the feature map in the neck structure. The method includes a whole-body estimation to locate individuals obstructed by obstacles, thereby preventing collisions with vehicles and avoiding hazardous situations in restricted areas. The proposed method enables real-time object detection and risk assessment for nearby objects, enhancing safety for industrial vehicle drivers and workers. Moreover, it can lead to further improvements in object detection performance in research fields such as autonomous driving and AI CCTV utilizing cameras. • Build dataset of industrial sites and improve the object detection performance on various complex backgrounds • Deep learning-based object detection method for accident prevention in industrial and construction environments • Unconditionally protected detector based on a selective attention network to extract and strengthen important features • Application of various model optimization techniques to shorten learning time and achieve efficient learning • Whole-body estimation method to detect location of an actual person to prevent dangerous situations at industrial sites [ABSTRACT FROM AUTHOR]

Detecting objects in hospital indoor environments is critical for scene understanding and can have various applications in healthcare. Deep learning algorithms have proven to be effective in object recognition from images or videos, but the availability of annotated datasets plays a crucial role in their successful application. However, there is a shortage of datasets for object detection in hospital settings, hindering the advancement of hospital indoor object detection algorithms. In this paper, we present the Hospital Indoor Object Detection (HIOD) dataset, consisting of 4,417 images covering 56 object categories. The HIOD dataset represents the frequently encountered objects in hospitals and comprises 51,869 annotated objects. The dataset is characterized by dense annotation, with an average of 11.7 objects and 6.8 object categories per image. An object detection benchmark was established using the HIOD dataset and eight state-of-the-art object detectors. The benchmark provides a comprehensive evaluation of the performance of the selected object detectors on a large and diverse set of images of objects commonly seen in hospital environments. The results of the benchmark can be used to compare and analyze the performance of different object detectors and identify their strengths and weaknesses for use in hospital environments. In the benchmark, one-stage detectors have shown superior performance compared to two-stage detectors of similar parameter sizes. In particular, YOLOv6-L was able to attain a mean Average Precision (mAP) of 51.7% while operating at a detection speed of 255 FPS. The benchmark and dataset can serve as a valuable resource for researchers and practitioners in the field of computer vision and robotics, helping to advance the development of more effective and efficient object detection algorithms for developing automated operations in hospitals such as robotic disinfection and patient assistance. • A new object detection dataset for artificial intelligence applications in hospitals. • A benchmark to compare and analyze the performance of different object detectors. • Improve object detection via hyperparameter optimization and transfer learning. • Evaluate the effects of image size and backbone on object detection performance. [ABSTRACT FROM AUTHOR]

• An automated system for railgun launcher damage detection is proposed. • A new data set of railgun inner bore damage is obtained by the designed device. • Adaptive data augmentation and focal loss are used to balance the uneven data set. • YOLOv5 and SOLOv2 are used for the detection and shape extraction of damage. • Results on the data set show the effectiveness of the proposed methods. The inner bore damage affects the launch performance and service life of electromagnetic railgun launcher. Detection and observation of railgun inner bore damage contribute to the study on mechanism and development rules of railgun damage. This paper analyzes five types of typical railgun inner bore damage. Based on the detection requirement for the damages, this paper proposes an automated damage detection system for the inner bore of electromagnetic railgun launcher consisting of data acquisition device and detection algorithms. The proposed device can step inside the inner bore of the railgun launcher to take photos of the inner bore surface automatically. We use the images obtained by the proposed device to build a data set for the training and verification of the detection algorithms. The object detection algorithm You Only Look Once v5 (YOLOv5) is utilized to achieve the rapid detection of railgun inner bore damages. We introduce the adaptive data augmentation and the focal loss to balance out the uneven category distribution of our data set. The result proves that our YOLOv5 model reaches the state-of-the-art level in the railgun inner bore damage detection task, with its mean Average Precision (mAP) of 0.659 and detection speed of 47.6 frames per second (fps). We choose Segmenting Objects by Locations v2 (SOLOv2) to extract the shape of the damage, with the Average Precision of 0.631. We further achieve damage statistics and the model visualization of damage distribution. The experimental results show that the proposed detection system meets the requirements of rapid detection and accurate feature extraction. It provides researchers with an approach for the study of railgun inner bore damage mechanism. [ABSTRACT FROM AUTHOR]

Modern businesses benefit significantly from advances in computer vision technology, one of the important sectors of artificially intelligent and computer science research. Advanced computer vision issues like image processing, object recognition, and biometric authentication can benefit from using deep learning methods. As smart devices and facilities advance rapidly, current networks such as 4 G and the forthcoming 5 G networks may not adapt to the rapidly increasing demand. Classification of images, object classification, and facial recognition software are some of the most difficult computer vision problems that can be solved using deep learning methods. As a new paradigm for 6Core network design and analysis, artificial intelligence (AI) has recently been used. Therefore, in this paper, the 6 G wireless network is used along with Deep Learning to solve the above challenges by introducing a new methodology named Optimizing Computer Vision with AI-enabled technology (OCV-AI). This research uses deep learning – efficiency algorithms (DL-EA) for computer vision to address the issues mentioned and improve the system's outcome. Therefore, deep learning 6 G proposed frameworks (Dl-6 G) are suggested in this paper to recognize pattern recognition and intelligent management systems and provide driven methodology planned to be provisioned automatically. For Advanced analytics wise, 6 G networks can summarize the significant areas for future research and potential solutions, including image enhancement, machine vision, and access control. [ABSTRACT FROM AUTHOR]

Automatic Welding Helmet Use (WHU) detection technology is of great significance for the safety management of construction site, then, this paper proposes a lightweight face-assisted model using YOLOv5s for the detection of WHU (WHU-YOLO). First, the Ghost module is introduced into YOLOv5s to optimize feature extraction parts of backbone and neck, reducing model complexity. Then, the neck of YOLOv5s is reconstructed based on Bi-directional Feature Pyramid Network (Bi-FPN). The experimental results implemented on established Welding helmet and Human face Detection (WHD) dataset indicate that the false positives have been greatly decreased with the assistance of face data and the mean average precision (mAP) reaches 83.65%. Meanwhile, under the environment of NVIDIA GeForce GTX 1070 and 640 × 640 input size, WHU-YOLO with inference time up to 5.7 ms achieves model compression with 35.7%, 34.4% and 30.1% reductions in parameters, weight size and Floating Point Operations (FLOPs) compared with YOLOv5s, respectively, which has no decline on detection performance. [ABSTRACT FROM AUTHOR]

Real-time shield tail clearance measurement and monitoring is a key task during shield tunneling construction. The shield tail clearance measurement and monitoring technology development is still in its infancy, the current methods are mainly designed manually based on intuition. In order to fill the gap between the requirement of shield tail clearance measurement and monitoring and the limitations of the current methods, this paper systematically studies the existing mechanisms related to shield tail clearance measurement and monitoring, and develops a high-precision intelligent monitoring system for shield tail clearance. The proposed monitoring system includes four components: 1) two types of shield tail clearance calculation models, 2) the integrated hardware of the monitoring system which is composed of a data acquisition unit, a signal transmission unit and a control unit, 3) the region of interest (ROI) extraction method based on deep neural network, and the image processing algorithms for image enhancement and feature extraction, 4) the custom-developed software built on mature integrated development environment (IDE). After the calculation model of shield tail clearance is established, the system uses monitoring devices equipped with industrial cameras to obtain the on-site image, and then applies image processing technologies along with deep learning approach to extract the key features, which are brought into the model to calculate the values of shield tail clearance, finally displays these values and simulates the current tunneling attitude of the shield machine in real time. The experimental results show that the system proposed in this paper achieves the goal of high precision measuring and real-time monitoring of the shield tail clearance. • A high-precision intelligent monitoring system for shield tail clearance is proposed and invented. • Two types of shield tail clearance calculation models are presented. • The ROI extraction is taken as an object detection problem and solved by an end-to-end solution. • Comprehensive algorithms of image processing are used for image enhancement and feature extraction. • The high accuracy, stability and real-time performance of the system have been confirmed through experimental testing. [ABSTRACT FROM AUTHOR]

• Real-time vision-based methods are proposed for structural vibration measurement. • The YOLOv3-tiny-KLT method achieves higher accuracy over the YOLOv3-tiny method. • The YOLOv3-tiny-KLT method achieves higher robustness over the KLT method. • The proposed methods address illumination changes and background noise. Structural vibration measurement is crucial in structural health monitoring and structural laboratory tests. Traditional contact type sensors are usually required to be attached to the test specimens, which may be difficult to install, and may affect the structural properties and response. Non-contact type wireless sensors are usually expensive and require specialized workers to install and operate. In recent years, vision-based tracking methods for structural vibration measurement have gained increasing interests due to their high accuracy, non-contact feature and low cost. However, traditional vision-based tracking algorithms are susceptible to external environmental conditions such as illumination and background noise. In this paper, two real-time methods, YOLOv3-tiny and YOLOv3-tiny-KLT, are proposed to track structural motions. In the first method, YOLOv3-tiny is established based on the YOLOv3 architecture to localize customized markers where structural displacements are directly determined from the bounding boxes generated. The second method, YOLOv3-tiny-KLT, is a more advanced method which combines the YOLOv3-tiny detector and the traditional KLT tracking algorithm. The pretrained YOLOv3-tiny is deployed to localize the targets automatically, which will then be tracked by Kanade‐Lucas‐Tomasi algorithm. YOLOv3-tiny is intended to provide baseline vibration measurement when the KLT tracking gets lost. The proposed methods were implemented for the videos of shake table tests on a two-storey steel structure. Parametric studies were conducted for the YOLOv3-tiny-KLT method to examine its sensitivity to the tracking parameters. The results show that the proposed method is capable of achieving real-time speed and high accuracy, when compared with the traditional displacement sensors including linear variable differential transducer (LVDT) and String Pots. It is also found that the combined YOLOv3-tiny-KLT approach achieves higher accuracy than YOLOv3-tiny only method, and higher robustness than KLT only method against illumination changes and background noise. [ABSTRACT FROM AUTHOR]

Transformer models have recently become the dominant architecture in many computer vision tasks, including image classification, object detection, and image segmentation. The main reason behind their success is the ability to incorporate global context information into the learning process. By utilising self-attention, recent advancements in the Transformer architecture design enable models to consider long-range dependencies. In this paper, we propose a novel transformer, named Swin Transformer with Cascaded UPsampling (SwinCup) model for the segmentation of histopathology images. We use a hierarchical Swin Transformer with shifted windows as an encoder to extract global context features. The multi-scale feature extraction in a Swin transformer enables the model to attend to different areas in the image at different scales. A cascaded up-sampling decoder is used with an encoder to improve its feature aggregation. Experiments on GLAS and CRAG histopathology colorectal cancer datasets were used to validate the model, achieving an average 0.90 (F1 score) and surpassing the state-of-the-art by (23%). • Hierarchical transformer-based models can extract global context features. • Cascaded up samplers to extract features without consuming much computational cost. • Globally aware models generate better segmentation results in microscopy images. • In contrastive self-supervised learning, adding labels improves results. [ABSTRACT FROM AUTHOR]

• An underwater image dataset of sea cucumber was created and annotated with rectangular boxes. • A detection network S-FPN, introducing shortcut connections to the feature pyramid, was proposed. • A new Piecewise Focal Loss function was created to alleviate the problem of class imbalance in object detection. Convolutional neural network is a prominent innovation in computer vision but is often troubled by problems such as dark light, turbidity, blur and high similarity to the background when applied to underwater object detection. Underwater object detection is one of the basic techniques of underwater grasping automation which plays a very important role in ocean detection and fishery of aquatic products. This paper presented an automatic detection method of underwater sea cucumber based on deep learning, which will provide effective technical support for the automated breeding and harvesting of sea cucumber. The Shortcut Feature Pyramid Network (S-FPN) proposed in this paper improves the existing multi-scale feature fusion strategy through shortcut connection. The ablation experimental results show that the mean average precision (mAP) of S-FPN reaches 91.5% which outperforms the baseline Feature Pyramid Network (88.6%), YOLO v3 (83.7%) and SVM-HOG (61.6%). To resolve the problem of complex environmental background interference of ocean floor, we proposed a Piecewise Focal Loss (PFL) function for balancing the positive and negative samples such that the algorithm can focus on the training difficulty of hard (i.e., positive) samples. And the ablation experimental results show that the mAP of PFL reaches 92.3% which outperforms the baseline Cross Entropy (91.5%) and Focal Loss (91.8%). Also, we chose Exponential Linear Unit as the optimization strategy, and Adaptive Moment Estimation as the activation function by ablation research, finally the mAP reached 94%. [ABSTRACT FROM AUTHOR]

This paper presents a system for detecting damages in concrete dams that combines the proposed YOLOv5s-HSC algorithm and a three-dimensional (3D) photogrammetric reconstruction method to accurately identify and locate objects. Since the damages usually have complex background and blurred boundaries, Swin transformer blocks and coordinate attention modules were introduced to improve the ability of feature extraction. The mean average precision (mAP) value of the improved algorithm increased by 3.8% and exhibited a reasonably robust performance for both small objects and a considerable detection effect. Subsequently, to realize the localization of the detected damages and mapped to the corresponding positions, the projecting method was proposed by calculating the intersection of the ray from camera center and 3D photogrammetric reconstruction model generated from the same images as for detection. The results confirmed that the proposed method is appropriate for detecting damages and recording locations for intuitively exhibiting concrete dam damages. • An improved object detection algorithm called YOLOv5s-HSC is proposed. • Combination of attention modules is designed and achieves well performances. • Detection results are mapped on UAV based photogrammetric 3D reconstruction model. • The proposed method can achieve concrete dam damages detection and localisation. [ABSTRACT FROM AUTHOR]

Comprehensive image datasets can benefit the construction industry in terms of serving as the basis for generating deep-learning-based object detection models and testing the performance of object detection algorithms, but building such datasets is complex and requires vast professional knowledge. This paper develops and publicly releases a new large-scale image dataset specifically collected and annotated for the construction site, called Site Object Detection Dataset (SODA), which contains 15 object classes categorized by the worker, material, machine, and layout. >20,000 images were collected from multiple construction sites in different situations, weather conditions, and construction phases, covering different angles and perspectives. Statistical analysis shows that the dataset is well developed in terms of diversity and volume. Further evaluation with two widely-adopted deep learning-based object detection algorithms also illustrates the feasibility of the dataset, achieving a maximum mAP of 81.47%. This research contributes a large-scale open image dataset for the construction industry and sets up a performance benchmark for further evaluation of relevant algorithms. • A high-quality image dataset is publicly released for objects on construction site. • SODA has more images and 4M1E categories than the existing datasets. • SODA provides production experience, data analysis, and comparison of the others. • The benchmark provides the basis for selecting deep learning algorithms. [ABSTRACT FROM AUTHOR]

• The network cyber security has to be enhanced and the proposed technique implemented with cyber security control system. • This research has also performed multiple moving objects tracking using Kernel's convoluted moving window with Kalman filter (KCMW_KF). • Foreground object recognition in video is critical in many computer vision applications and automated video surveillance systems. • Object tracking establishes the correlation between objects in a video sequence's succeeding frames. Foreground object recognition in video is critical in many computer vision applications and automated video surveillance systems. Object detection and tracking are critical steps in navigation, object recognition and surveillance schemes. Object detection is process of separating foreground and background items in photographs. In this paper, we proposes a framework for achieving these tasks with the enhanced cyber security control facility. This proposed algorithmperformed in 2 stages: multi-object detection utilizing the Cyber secure Probabilistic Gaussian Mixture Model (Cy_SPGMM) and background suppression and another stage is multiple moving objects tracking utilizing Kernel convoluted moving window with Kalman filter (KCMW_KF). It can, however, deal with a variety of video sequences in the MOT 20 dataset. The experimental findings reveal that the proposed algorithm detects and tracks foreground objects in complex and dynamic scenarios with high accuracy, robustness, and efficiency. This method also produces smoothened images without noise. [Display omitted] [ABSTRACT FROM AUTHOR]

In the desert region of northwest China, the frequency of wind-sand disasters is high. All types of concrete buildings built in this area face severe wind erosion due to high wind speed, resulting in varying degrees of wind-erosion damage to concrete. To accomplish intelligent identification of concrete wind-erosion damage, a concrete wind erosion experiment was conducted in the laboratory, and a concrete wind-erosion damage dataset was generated under the interference of water stains, scratches, shooting distance, and background noise. This paper combined with transformer theory to improve YOLO-v4 and proposed an object detection algorithm called MHSA-YOLOv4 suitable for wind-erosion damage of concrete. The results demonstrate that MHSA-YOLOv4 exhibits improved object detection performance than YOLO-v3, improved YOLO-v3, and YOLO-v4. On the test set, ACC, Precision, Recall, and mAP of MHSA-YOLOv4 are 91.30%, 91.52%, 92.31%, and 0.89, respectively. MHSA-YOLOv4 can accurately identify wind-erosion damage of concrete images under different test conditions, which reflects strong robustness. The applicability of computer vision technology to the intelligent identification of wind-erosion damage on concrete has been verified. • The dataset of wind-erosion of concrete was established to simulate the spalling of concrete. • A object detection algorithm called MHSA-YOLOv4 was proposed based on Transformer. • MHSA-YOLOv4 can detect concrete wind-erosion damage with high-precision. [ABSTRACT FROM AUTHOR]

Using computer vision and deep learning (e.g., Convolutional Neural Networks) to automatically recognise unsafe behaviour from digital images can help managers identify and respond quickly to such actions and mitigate an adverse event. However, there has been a tendency for computer vision studies in construction to focus solely on detecting unsafe behaviour (i.e., object detection) or the regions of interest with pre-defined labels. Moreover, such approaches have been unable to consider rich semantic information among multiple unsafe actions in a digital image. The research we present in this paper uses a safety rule query to determine and locate several unsafe behaviours in a digital image by employing a visual grounding approach. Our approach consists of: (1) visual and text feature extraction, (2) recursive sub-query, and (3) generation of the bounding box. We validate our approach by conducting an experiment to demonstrate it is effectiveness. The results from an experimental study demonstrate an average precision, recall, and F1-score were 0.55, 0.85, and 0.65, respectively, suggesting our approach can accurately identify and locate different types of unsafe behaviours from digital images acquired from a construction site. [ABSTRACT FROM AUTHOR]

Waste sorting is highly labor intensive because the wide variety of waste items prohibits automation. More recently, deep learning (DL) and computer vision technology has presented an opportunity to streamline the sorting process, but many important developmental steps remain. If computer vision technology can increase the efficiency of automated waste sorting, this would be beneficial for society and the environment. Accordingly, this study used the You Only Look Once-v3 (Yolo-v3) detection model based on DL to enhance recognition performance of household waste products. TrashNet, a commonly used waste image database, was used to train an initial Yolo-v3 model, however each image used for training only had a single waste object, and this study found that the detection model trained with a single object dataset was not only unsuitable for sorting multiple waste objects, but that this has rarely been addressed in academic literature. It was also discovered that nations and regions will need to develop their own unique databases that reflect the types of waste products found. Samples images need to account for the various appearances and colors and be combined in multiple waste object images when training the system. This paper documents the training and testing of an object detection model suitable for detecting domestic waste specific to Taiwan; however, the approach taken would be of use to other countries seeking to automate waste sorting. To achieve this, it was necessary to compile the Taiwan Recycled Waste Database (TRWD). This was then used to train the Yolo-v3, and the efficiencies of this, versus the standard TrashNet model were compared. Results showed that the TRWD-trained Yolo-v3 achieved mAP @0.5 of 92.12% and could detect waste in real-time. Relative to the TrashNet-trained Yolo-v3, the TRWD counterpart performed better due to the multiple waste objects and more relevant image repository. Further studies are recommended to investigate the effect of combining additional sensors that would enable improved detection of specific wastes. Combining the TRWD-trained Yolo-v3 with a robot system for waste sorting would potentially be another rewarding avenue of research. [Display omitted] • Automatic waste detection improves waste recycling efficiency. • Different nations require customized datasets to train Yolo-v3 detection model. • Taiwan recycled waste dataset (TRWD) was expanded to improve detection rates. • Yolo-v3 trained on the TRWD outperformed the same system using TrashNet. [ABSTRACT FROM AUTHOR]

Cracks undermine the structural health of transportation infrastructure. Machine vision-based surface crack analysis is to process infrastructure inspection data collected by imaging devices for identifying the presence, location, and extent of cracks, classifying the corresponding severity levels, and eventually predicting their growth. Unlike the fragmented qualitative discussions on machine vision-based crack analysis methods in existing studies, this paper reviews the state of the art and practice of various machine vision solutions under different operating conditions in a fine-grained quantitative way, systematically describing the strengths and limitations of deep learning over other solutions. Moreover, the applicability assessment is implemented to describe the deployment and optimization of deep learning in five crack analysis tasks: image classification, object detection, pixel segmentation, geometric scale quantification, and growth prediction. At last, the challenges faced and corresponding breakthrough directions are summarized, respectively, driving further development of deep learning to assist more sophisticated maintenance decisions. • Fine-grained, quantitative, applicability-oriented reviews are implemented. • Assessments are conducted under four operating conditions and for five crack analysis tasks. • Systematically revealing the strengths and limitations of deep learning in crack analysis. • Future breakthrough directions are proposed to address three key challenges. • Driving a revolution in decision-making for infrastructure maintenance and management. [ABSTRACT FROM AUTHOR]

• There exist few surveys on this filed, covering only existing methods in a specific domain, and since 3D object detection is growing rapidly, it lacks the idea of the state-of-the-art methods that could provide some new solutions and directions. • We provide the background of 3D object detection and details of RGB-D data. • We systematically and comprehensively review the proposed solutions in 3D object detection methods for RGBD data and focus on the contributions and limitation of these methods, so that readers can see the cutting-edge technology of this filed more easily. • We provide a comparison of methods for RGBD-based 3D object detection and discuss current research issues and future research directions. 3D object detection is a critical part of environmental perception systems and one of the most fundamental tasks in understanding the 3D visual world, which benefit a series of downstream real-world applications. RGB-D images include object texture and semantic information, as well as depth information describing spatial geometry. Recently, numerous 3D object detection models for RGB-D images have been proposed with excellent performance, but summaries in this area are still absent. To stimulate future research, this paper provides a detailed analysis of current developments in 3D object detection methods for RGB-D images to motivate future research. It covers three major parts, including background on 3D object detection, RGB-D data details, and comparative results of state-of-the-art methods on several publicly available datasets, with an emphasis on contributions, design ideas, and limitations, as well as insightful observations and inspiring future research directions. [ABSTRACT FROM AUTHOR]

• An integrated detection-based interaction scheme (IDIS) has been proposed for improving the accuracy and reliability of HCI systems. • The recurrent process is used to identify the variations in interaction patterns and time. • The proposed scheme's performance is verified using dataset sources for the following metrics: accuracy, delay, error, and interaction span. • The classified input is verified using matching patterns throughout the interaction span. The pre-classification instances are verified using a DBN through training based on previous instances. Human-computer interaction (HCI) and computer vision (CV) provide interesting communication features between machines and humans in different real-time applications. Visualization helps to improve the accuracy of detecting target communication objects for better interaction. This paper introduces an integrated detection-based interaction scheme (IDIS) for improving the accuracy and reliability of HCI systems. The input is fetched from the ranged object, and the interaction session is initiated after the classification and detection of the object. The process of robust, long-term interaction with the detected object is achieved through pre-classification. In this detection process, deep learning is used to identify the object and perform its intended interaction requirements. The recurrent process is used to identify the variations in interaction patterns and time. The allocation of interaction sessions is streamlined to improve the interaction span and detection accuracy. The proposed scheme's performance is verified using dataset sources for the following metrics: accuracy, delay, error, and interaction span. Integrated Detection-Based Interaction Scheme for Human-Computer Interaction through Reliable Detection [Display omitted] Figure: Integrated Detection-Based Interaction Scheme [ABSTRACT FROM AUTHOR]

The drainage system is an important part of civil infrastructure. However, the underground sewage pipe will gradually suffer from defects over time, such as tree roots, deposits, infiltrations and cracks, which heavily affect the performance of sewage pipes. Therefore, it is significant to timely inspect the condition of sewage pipes. Closed-circuit television (CCTV) inspection is a commonly employed underground infrastructure inspection technology requiring engineering experience that can be subjective and inefficient. Nowadays, object detection based on convolutional neural network (CNN) can automatically detect defects, showing high potential for improving inspection efficiency. This paper proposed an improved CNN-based You Only Look Once version 3 (YOLOv3) method for automatic detection of sewage pipe defects, where the improvements are mainly involved in loss function, data augmentation, bounding box prediction and network structure. Experiment results demonstrate that the improved model outperforms Faster R-CNN and YOLOv3, achieving a mean average precision (mAP) value of 92%, which is higher than the existing research on automatic detection of sewage pipe defects. • Proposed an improved YOLO-based method for automatic detection of sewage defects. • The proposed new data augmentation method improved defect detection accuracy. • Applied three different loss functions and compared their detection performance. • Our new architecture outperformed previous research for detecting sewer defects. [ABSTRACT FROM AUTHOR]