Search

Offshore HVDC converter stations for transportation of offshore wind energy to the coast need regular human inspection. Automated surveillance of such a platform by a mobile robot has high potential of improving the speed and quality of decision-making while reducing operating expenses and risk of unmanned operation of the platforms. However, the challenging environment of such platforms has prevented operators from making use of mobile robots to this date. Recent progress in legged robotics resulted in systems that are becoming feasible for such tasks nowadays. For this reason, the quadrupedal robot ANYmal was tested on a platform in the North Sea for automated inspection. This paper presents the results of the field tests and discusses the challenges of industrial inspection of offshore sites., Springer Proceedings in Advanced Robotics, 16, ISSN:2511-1256, Field and Service Robotics, ISBN:978-981-15-9459-5, ISBN:978-981-15-9460-1

In this paper we propose a new concept for a compact 3D sensor dedicated to industrial inspection, combining chromatic Depth From Defocus (DFD) and structured illumination. Depth is estimated from a single image using local estimation of the defocus blur. As industrial objects usually show poor texture information, which is crucial for DFD, we rely on structured illumination. In contrast with state of the art approaches for active DFD, which project sparse patterns on the scene, our method exploits a dense textured pattern and provides dense depth maps of the scene. Besides, to overcome depth ambiguity and dead zone of DFD with a classical camera, we use an unconventional lens with chromatic aberration, providing spectrally varying defocus blur in the camera color channels. We provide comparisons of depth estimation performance for several projected patterns at various scales based on simulation and real experiments. The proposed method is then qualitatively evaluated on a real industrial object. Finally we discuss the perspectives of this work especially in terms of co-design of an 3D active sensor using DFD.

Active 3D imaging systems use artificial illumination in order to capture and record digital representations of objects. The use of artificial illumination allows the acquisition of dense and accurate range images of textureless objects that are difficult to acquire using passive vision systems. An active 3D imaging system can be based on different measurement principles that include time-of-flight, triangulation, and interferometry. Here, an in-depth presentation of triangulation-based 3D imaging is provided, including both scanning laser and structured light projection systems, with a focus on industrial applications. The characterization of triangulation is discussed using both an error-propagation framework and experimental protocols.

A new methodology that mixes the already well-established classifications of Wall Climbing Robots (WCR) based on locomotion and attachment is proposed. The need for industrial inspections of processing assets, with vertical flat surfaces, like tanks and pressure vessels that are continuously exposed to fatigue, corrosion and erosion are identified. The types of industrial inspections tasks vary in a wide range from, Visual Inspections (VI) and Non-Destructive Testing (NDT) to cleaning and repair. Mobile robots are used to perform inspection tasks. There are few commercial WCR that are available for industrial applications. The classification method is based on locomotion mechanisms, and attachment techniques rather than classical classification assumed in all previous reviews of WCR. A general design framework is suggested to derive the custom-building design of the inspection WCR. The performance of the WCR is compared against the main features of maximum speed and payload capacity. The main locomotion types are wheeled (W), tracked (T), walking (L) and hybrid (H). The primary attachment methods are magnetic (G), pneumatic (P), mechanical (M), electrostatic (E), chemical (C) and hybrid (H). As another application, the classification is used to identify and differentiate WCR from other types of Climbing Robots (CR) to cover almost all kinds of WCR. In addition, the classification can be employed in the customisation of WCR that are built to perform specialised inspection tasks. The customisation of WCR results from the need for simplicity in their structure, adapting testing equipment and improving operations.

The signal processing of industrial big data (IBD) is a challenging task, owing to the complex working scenarios and the lack of annotations. Defect detection, which is an important subject of IBD research works, has shown its effectiveness in digital signal processing of industrial inspection applications in many previous studies. This article proposes a novel defect detection method based on deep learning for digital signal processing of industrial inspection applications. In our method, a module named feature collection and compression network is applied to merge multiscale feature information. Then, a new pooling method named Gaussian weighted pooling, which provides more precise location information, is used to replace region of interest (ROI) pooling. Experiment results show that our method gets improvements in both accuracy and efficiency, with mAP/ $\text{AP}_{50}$ of 41.8/80.2 at 33 fps on NEU-DET, which satisfies the requirement of real-time systems.

With the fast growth of intelligent manufacturing industry, developing advanced industrial inspection robots is becoming a research and application hotspot in the fields of both computer vision and robotics. This kind of industrial inspection robots is expected to automatically detect anomalous structures (e.g., defects, damages, rejects, etc.) from the images of the manufactured products. Generally, the existing visual anomaly detection (VAD) methods mainly focus on modeling the complex and high-dimensional distribution of normal data, while neglecting the specific visual properties of abnormal data since their frequency of occurrence is much less than that of the normal data. In this paper, inspired by the human cognition on extracting abstractly visual properties and to distinguish the anomaly patterns from the observed data, we propose a novel cognitive VAD method for industrial inspection robot. Specifically, we introduce a constrained latent space to mimic the cognitive ability of humans, where the abstraction learned from the observed normal and anomaly data are represented. We build our method based on a convolutional generative adversarial network and a denoising auto-encoder, where the adversarial learning mechanism is adopted to establish the boundary between the normal and anomaly data. In the experiment, we evaluate our method on a real-world dataset where the images are captured for the manufactured products. The comprehensive results comparing with several recent VAD methods show that the proposed method is effective to detect the anomaly images of different categories with a high accuracy.

The image of 2D single-shot interferometry based high-speed camera at a shutter speed of ~1μs, which crosses over the limit of conventional OCTs, was experimentally observed. The optical shapes were captured even under vibrational environment.

The traditional mode of recording faults in heavy factory equipment has been via handmarked inspection sheets, wherein a machine engineer manually marks the faulty machine regions on a paper outline of the machine. Over the years, millions of such inspection sheets have been recorded and the data within these sheets has remained inaccessible. However, with industries going digital and waking up to the potential value of fault data for machine health monitoring, there is an increased impetus towards digitization of these handmarked inspection records. To target this digitization, we propose a novel visual pipeline combining state of the art deep learning models, with domain knowledge and low level vision techniques, followed by inference of visual relationships. Our framework is robust to the presence of both static and non-static background in the document, variability in the machine template diagrams, unstructured shape of graphical objects to be identified and variability in the strokes of handwritten text. The proposed pipeline incorporates a capsule and spatial transformer network based classifier for accurate text reading, and a customized CTPN [15] network for text detection in addition to hybrid techniques for arrow detection and dialogue cloud removal. We have tested our approach on a real world dataset of 50 inspection sheets for large containers and boilers. The results are visually appealing and the pipeline achieved an accuracy of 87.1% for text detection and 94.6% for text reading.

Fast and reliable industrial inspection is a main challenge in manufacturing scenarios. However, the defect detection performance is heavily dependent on manually defined features for defect representation. In this contribution, we investigate a new paradigm from machine learning, namely deep machine learning by examining design configurations of deep Convolutional Neural Networks (CNN) and the impact of different hyper-parameter settings towards the accuracy of defect detection results. In contrast to manually designed image processing solutions, deep CNN automatically generate powerful features by hierarchical learning strategies from massive amounts of training data with a minimum of human interaction or expert process knowledge. An application of the proposed method demonstrates excellent defect detection results with low false alarm rates.

With the rise of Industry 4.0 core technologies, companies are aiming to reap the benefits of their integration, and are looking for transition steps, suggestions and tools. In this paper, we mainly focus on the transition of the Industrial Inspection. The goal is to project this task into advanced Industry 4.0 factories, after looking at the literature. As a result, the proposed approach is to use computer vision technologies such as image processing & Analysis and Human-Machine Interactions, along with a modularity aspect allowing agile and customized configuration, on top of the use of embedded cameras on robots.

This work addresses the problem of semi-automatic inspection and navigation in confined environments. A system that overcomes many challenges at the state of the art is presented. It comprises a multirotor able to inspect an industrial combustion chamber thus working in a GPS-denied environment with poor lighting conditions, in the presence of magnetic and communication disturbances, iron dust and repetitive patterns on the structure walls. The presented system is able to pass through narrow entrances but still capable of acquiring high resolution images and to allow operators to perform inspection of the structures. Starting from the captured data, the system is able to provide a 3D reconstruction of the inspected environment for offline analysis.

A large number of real-world machine learning and data mining problems have several crucial issues that need to be solved, such as class imbalance and multiple objectives; these issues cannot be easily overcome with many learning methods. This study proposes an efficient approach to address the issues and applies the approach in the design of a machine vision system for a real-world problem: inline inspection of surface defects on glass substrates of thin-film transistor liquid crystal displays (TFT-LCDs). The three major steps in the machine vision system are: (1) selective extraction of defect features from images using 2-dimemnsional wavelet decomposition, (2) training cost-sensitive classifiers to handle class imbalance, and (3) use of ensemble techniques to achieve multiple manufacturing objectives. When applied to an industrial case study, the achieved performance shows that using the proposed approach in defect inspection of TFT-LCD glass substrates is a viable alternative to manual inspection.

The weld defects are formed due to the incorrect welding patterns or wrong welding process. The defects in the weld may vary from size, shape and their projected quality. The most common weld defects occur during welding process is slag inclusions, porosity, lack of fusion and incomplete penetration. In this study, an effective method for weld defect classification using machine learning algorithm is presented. The system uses Speeded-up Robust Features (SURF) for feature extraction and one of the machine learning algorithms called Auto-Encoder Classifier (AEC) for classification. Initially, the features that distinguish weld defects and no defects in the weld image are extracted by SURF. Then, AEC is analyzed for weld image classification using different number of neurons in different hidden layers (2 and 3 hidden layers). The performance of the system is evaluated by GD X-ray weld image database. The results show that the weld images are correctly classified with 98% accuracy using SURF and AEC.

Recently, researchers have been studying methods to introduce deep learning into automated optical inspection (AOI) systems to reduce labor costs. However, the integration of deep learning in the industry may encounter major challenges such as sample imbalance (defective products that only account for a small proportion). Therefore, in this study, an anomaly detection neural network, dual auto-encoder generative adversarial network (DAGAN), was developed to solve the problem of sample imbalance. With skip-connection and dual auto-encoder architecture, the proposed method exhibited excellent image reconstruction ability and training stability. Three datasets, namely public industrial detection training set, MVTec AD, with mobile phone screen glass and wood defect detection datasets, were used to verify the inspection ability of DAGAN. In addition, training with a limited amount of data was proposed to verify its detection ability. The results demonstrated that the areas under the curve (AUCs) of DAGAN were better than previous generative adversarial network-based anomaly detection models in 13 out of 17 categories in these datasets, especially in categories with high variability or noise. The maximum AUC improvement was 0.250 (toothbrush). Moreover, the proposed method exhibited better detection ability than the U-Net auto-encoder, which indicates the function of discriminator in this application. Furthermore, the proposed method had a high level of AUCs when using only a small amount of training data. DAGAN can significantly reduce the time and cost of collecting and labeling data when it is applied to industrial detection.

An application of spectral domain optical coherence tomography (SD-OCT) was demonstrated for a fast industrial inspection of an optical thin film panel. An optical thin film sample similar to a liquid crystal display (LCD) panel was examined. Two identical SD-OCT systems were utilized for parallel scanning of a complete sample in half time. Dual OCT inspection heads were utilized for transverse (fast) scanning, while a stable linear motorized translational stage was used for lateral (slow) scanning. The cross-sectional and volumetric images of an optical thin film sample were acquired to detect the defects in glass and other layers that are difficult to observe using visual inspection methods. The rapid inspection enabled by this setup led to the early detection of product defects on the manufacturing line, resulting in a significant improvement in the quality assurance of industrial products.

The inspection of moving specular freeform surfaces is an industrial challenge so far largely unsolved, even for the qualitative case, i.e. the mere determination of the presence of surface defects as opposed to the quantitative reconstruction of a surface. Products produced in high quantities therefore still have to be inspected manually which is labour intensive, expensive, monotonous and subjective. We propose a novel hardware setup and methodology to overcome this shortfall. The reflection of a line laser from a moving surface is captured on a translucent screen; surface defects show as gaps or bulges. Two methods to extract the resulting information are proposed and ways for its interpretation are shown. The proposed method is very cost effective and easy to implement. While limitations to surface geometry exist and absolute precision is not achievable, it is shown that the system is able to reliably detect, characterise and localise a range of typical surface defects on moving glazed ceramic tiles, our example application. The method is however applicable to a wide range of hybrid and specular surfaces.

This paper presents a novel antenna having a beam focused to a small spot in the near-field zone which is intended for use in microwave industrial inspection. The array is implemented using broadband U-slotted microstrip patch-antenna elements and an integrated microstrip feeding network. Analogy with a dielectric lens is used to generate appropriate feeding coefficients for the elements of the array. The concept is demonstrated with two practical implementations having 300- and 126-mm focal distances. The measured results are presented and shown to be in good agreement with the predictions, validating the development concept.

PurposeThe purpose of this paper is to present improvements in X‐ray equipment, which are leading to wider use.Design/methodology/approachDevelopments in X‐ray sources and detectors are described. This is followed by a review of the more innovative equipment available for security and industrial applications.FindingsTechnological developments have produced smaller, lighter X‐ray systems and extended their applications to on‐site work. Multiple wavelength systems distinguish between different materials, and stereo systems remove ambiguities from X‐ray security imaging and allow 3D gauging of industrial components.Originality/valueThe paper highlights the availability of portable X‐ray systems and explains the underlying technology.

PurposeDescribes the design, construction, and performance of the OmniTread serpentine robot. Provides a review of other designs in this new area of mobile robotics. Presents innovative and unique mechanical and control solutions.Design/methodology/approachA theoretical analysis of key aspects of the mechanical design and their implications on the performance of the robot is presented. Extensive experimentation and testing helped optimize choices of materials for the critical components: tracks and pneumatic bellows. Performance was evaluated by an independent third party: the Southwest Research Institute.FindingsIt was found that pneumatic bellows are optimal joint actuators for serpentine robots. They can provide both strength and compliance, depending on the task, at minimal volume and weight.Research limitations/implicationsThe described prototype is tethered to external sources of electrical and pneumatic power. A smaller and fully self‐contained version of the OmniTread is currently under development.Practical implicationsA fully functional OmniTread serpentine robot will provide unprecedented mobility on rough terrain, such as the rubble of a collapsed building. The ability to climb over high obstacles and span large gaps, while still fitting through small openings suggests use of this robot in urban search and rescue, industrial inspection, and military reconnaissance tasks.Originality/valueThe OmniTread serpentine robot incorporates multiple original features, which resulted in three recent patents. Most notably are the Integrated pneumatic joint actuator with proportional position and stiffness control system and the “Tracks all Around” design. These features provide dramatic performance improvements in serpentine robots.

Due to their high spatial resolution, THz imaging systems (based on time domain spectrometers) have a broad variety of applications in scientific as well as industrial applications in the future—especially through the upcoming of reliable titanium sapphire lasers and fiber amplified femtosecond lasers. With such THz scanning highly sensitive measurements of materials as well as physical phenomena and simultaneously real-time imaging with real-time data extraction can be realized which can be used for production monitoring and/or more accurate and novel inline control systems. The article gives an overview on recent state of the art THz systems based on femtosecond lasers and their application scenarios, like detection of foreign bodies in foods, inline inspection of paper layer thickness and area mass as well as non-destroying impact or tensile tests or fiber orientation checks in Compound-Polymers.

In recent years, the light, thin, short, small and multi-function demand of consumer electronic products has driven SMT manufactures to make their IC package component sizes continue to shrink. For improving production yields of the tiny package components, many manufacturers have produced and tested the tiny IC package components in a clean room. This study aims the tiny chip size package (CSP) components to develop a Kinect-based inspection system. The proposed system applied Kinect sensor to perform human skeleton detection. With our developed software, the operator outside the clean room can use his hand to directly control the important inspection modules such as lighting module and translation module, which are located in the clean room, through the glass window by applying somatosensory technology.

SPIRIT aims to develop an “inspection skill” for robots that takes the step from programming of complex inspection tasks to configuring such tasks. Result of the project is a software framework that includes an “offline framework” with features such as model-based automatic coverage planning for complex parts, automatic robot program generation and an “inline framework” that deals with sensor data mapping to transfer sensor measurements to the 3D object model. At the heart of the project is an accurate process-specific model that represents the sensor data acquisition process. This representation is sufficiently accurate to allow automatic planning in off-line settings using simulated workcells and then reproducing the inspection procedure on the real one, with some adaptations and corrections. More in detail, the “offline framework” will include a generic interface to allow the easy exchange of process models (for different inspection technologies), of the CAD model of the part (for a different type of product to be inspected) or of the work-cell model (for a different robot kinematic structure). The generic “inline framework” will provide the backbone for the execution of the actual inspection process. Relying on such a proven frameworks will reduce the risks of implementing complex inspection tasks and thus help the deployment of inspection robots. In this work initial results from the two main use cases are presented, they consist of inspection tasks for the automotive and aerospace industry respectively.

This paper provides a review of commercially available lighting systems for industrial inspection applications, including the underlying technology and optical behaviour.

This paper describes a convolution-based approach to the analysis of images containing few texture classes. Segmentation of foreground and background textures, or detection of boundaries between similarly textured objects, is demonstrated. The application to industrial inspection applications is demonstrated. Near frame-rate performance on low-cost hardware is possible, since only convolution with small kernels is used. A new algorithm to optimize convolution kernels for the required texture analysis task is presented. A key feature of the paper is the industrial readiness of the techniques described.

The Industrial Metrology and Industrial Inspection systems make student conversant with the latest trends of measuring efficiently and build industrial inspection and metrology systems. Therefore, this lab manual has been developed based on the common inspection and metrology platform, combining real-time data processing into sophisticated measurement systems for industrial applications.

We analyze the performance of the Federal Industrial Inspection Program operated by the Federal Attorney for Environmental Protection (PROFEPA) in Mexico City. We seek to answer three questions: What drives the inspections? What determines non-compliance? And what drives imposition of fines? We use firm-level data that identify certain characteristics of the firms, PROFEPA's inspections, compliance results and fines for all air polluting firms under the Program during the period January 2000–October 2008. We obtain three main results. First, PROFEPA's inspections have focused on larger firms devoted to industrial activities located in poor and dense municipalities. Second, past inspections were significant and have a high marginal effect on compliance. Third, the severity of the infraction was not relevant to the decision to impose a fine but rather to its amount.