Search

We give an overview of AutoInspect, a ROS-based software system for robust and extensible mission-level autonomy. Over the past three years AutoInspect has been deployed in a variety of environments, including at a mine, a chemical plant, a mock oil rig, decommissioned nuclear power plants, and a fusion reactor for durations ranging from hours to weeks. The system combines robust mapping and localisation with graph-based autonomous navigation, mission execution, and scheduling to achieve a complete autonomous inspection system. The time from arrival at a new site to autonomous mission execution can be under an hour. It is deployed on a Boston Dynamics Spot robot using a custom sensing and compute payload called Frontier. In this work we go into detail of the system's performance in two long-term deployments of 49 days at a robotics test facility, and 35 days at the Joint European Torus (JET) fusion reactor in Oxfordshire, UK., Comment: Accepted to the IEEE ICRA Workshop on Field Robotics 2024

Within (semi-)automated visual industrial inspection, learning-based approaches for assessing visual defects, including deep neural networks, enable the processing of otherwise small defect patterns in pixel size on high-resolution imagery. The emergence of these often rarely occurring defect patterns explains the general need for labeled data corpora. To alleviate this issue and advance the current state of the art in unsupervised visual inspection, this work proposes a DifferNet-based solution enhanced with attention modules: AttentDifferNet. It improves image-level detection and classification capabilities on three visual anomaly detection datasets for industrial inspection: InsPLAD-fault, MVTec AD, and Semiconductor Wafer. In comparison to the state of the art, AttentDifferNet achieves improved results, which are, in turn, highlighted throughout our quali-quantitative study. Our quantitative evaluation shows an average improvement - compared to DifferNet - of 1.77 +/- 0.25 percentage points in overall AUROC considering all three datasets, reaching SOTA results in InsPLAD-fault, an industrial inspection in-the-wild dataset. As our variants to AttentDifferNet show great prospects in the context of currently investigated approaches, a baseline is formulated, emphasizing the importance of attention for industrial anomaly detection both in the wild and in controlled environments., Comment: Accepted at WACV 2024

Despite progress in vision-based inspection algorithms, real-world industrial challenges -- specifically in data availability, quality, and complex production requirements -- often remain under-addressed. We introduce the VISION Datasets, a diverse collection of 14 industrial inspection datasets, uniquely poised to meet these challenges. Unlike previous datasets, VISION brings versatility to defect detection, offering annotation masks across all splits and catering to various detection methodologies. Our datasets also feature instance-segmentation annotation, enabling precise defect identification. With a total of 18k images encompassing 44 defect types, VISION strives to mirror a wide range of real-world production scenarios. By supporting two ongoing challenge competitions on the VISION Datasets, we hope to foster further advancements in vision-based industrial inspection.

In recent years, the focus on anomaly detection and localization in industrial inspection tasks has intensified. While existing studies have demonstrated impressive outcomes, they often rely heavily on extensive training datasets or robust features extracted from pre-trained models trained on diverse datasets like ImageNet. In this work, we propose a novel framework leveraging the visual-linguistic CLIP model to adeptly train a backbone model tailored to the manufacturing domain. Our approach concurrently considers visual and text-aligned embedding spaces for normal and abnormal conditions. The resulting pre-trained backbone markedly enhances performance in industrial downstream tasks, particularly in anomaly detection and localization. Notably, this improvement is substantiated through experiments conducted on multiple datasets such as MVTecAD, BTAD, and KSDD2. Furthermore, using our pre-trained backbone weights allows previous works to achieve superior performance in few-shot scenarios with less training data. The proposed anomaly backbone provides a foundation model for more precise anomaly detection and localization.

Robustness and accuracy are two critical metrics for industrial inspection. In this paper, we propose benchmarks that can evaluate the structured light method's performance. Our evaluation metric was learning from a lot of inspection tasks from the factories. The metric we proposed consists of four detailed criteria such as flatness, length, height and sphericity. Then we can judge whether the structured light method/device can be applied to a specified inspection task by our evaluation metric quickly. A structured light device built for TypeC pin needles inspection performance is evaluated via our metrics in the final experimental section.

Industrial inspection automation in aerospace presents numerous challenges due to the dynamic, information-rich and regulated aspects of the domain. To diagnose the condition of an aircraft component, expert inspectors rely on a significant amount of procedural and tacit knowledge (know-how). As systems capabilities do not match high level human cognitive functions, the role of humans in future automated work systems will remain important. A Cyber-Physical-Social System (CPSS) is a suitable solution that envisions humans and agents in a joint activity to enhance cognitive/computational capabilities and produce better outcomes. This paper investigates how a work-centred approach can support and guide the engineering process of a CPSS with an industrial use case. We present a robust methodology that combines fieldwork inquiries and model-based engineering to elicit and formalize rich mental models into exploitable design patterns. Our results exhibit how inspectors process and apply knowledge to diagnose the component`s condition, how they deal with the institution`s rules and operational constraints (norms, safety policies, standard operating procedures). We suggest how these patterns can be incorporated in software modules or can conceptualize Human-Agent Teaming requirements. We argue that this framework can corroborate the right fit between a system`s technical and ecological validity (system fit with operating context) that enhances data reliability, productivity-related factors and system acceptance by end-users.

Cyber-Physical-Social Systems (CPSS) performance for industry 4.0 is highly context-dependent, where three design areas arise: the artifact itself, the human-agent collaboration, and the organizational settings. Current HF&E tools are limited to conceptualize and anticipate future human-agent work situations with a fine-grained perspective. This paper explores how rich in-sights from work analysis can be translated into formative design patterns that provide finer guidance in conceptualizing the human-agent collaboration and the organizational settings. The current manual work content elicited is disaggregated into functional requirements. Each function is then scrutinized by a multidisciplinary design team that decides its feasibility and nature (autonomy function, human function, or hybrid function). By doing so, we uncover the technical capabilities of the CPSS in comparison with subject-matter experts` work activity. We called this concept technological coverage. The framework thereof allows close collaboration with design stakeholders to define detailed HAT configurations. We then imagined joint activity scenarios based on end-users work activity, the technological capabilities, and the interaction requirements to perform the work. We use a study on technological innovation in the aircraft maintenance domain to illustrate the framework`s early phases, Comment: Inclusion in International Ergonomics Association 21th Triennial Congress, proceedings in Advances in Intelligent Systems and Computing

The traditional mode of recording faults in heavy factory equipment has been via hand marked inspection sheets, wherein a machine engineer manually marks the faulty machine regions on a paper outline of the machine. Over the years, millions of such inspection sheets have been recorded and the data within these sheets has remained inaccessible. However, with industries going digital and waking up to the potential value of fault data for machine health monitoring, there is an increased impetus towards digitization of these hand marked inspection records. To target this digitization, we propose a novel visual pipeline combining state of the art deep learning models, with domain knowledge and low level vision techniques, followed by inference of visual relationships. Our framework is robust to the presence of both static and non-static background in the document, variability in the machine template diagrams, unstructured shape of graphical objects to be identified and variability in the strokes of handwritten text. The proposed pipeline incorporates a capsule and spatial transformer network based classifier for accurate text reading, and a customized CTPN network for text detection in addition to hybrid techniques for arrow detection and dialogue cloud removal. We have tested our approach on a real world dataset of 50 inspection sheets for large containers and boilers. The results are visually appealing and the pipeline achieved an accuracy of 87.1% for text detection and 94.6% for text reading., Comment: Published in 3rd International Workshop on Robust Reading at Asian Conference on Computer Vision 2018

This paper presents an original method of modelling the colour distributions of images using 2D Gaussian functions and its application to flaw detection in industrial inspection. 2D Gaussian functions are used to model the colours that appear in the non-flawed images in an unsupervised manner. Pixels under test are compared to the colour distribution from training images. 140 images have been tested and the results are given. This method has a wide range of applications for detecting colour separable objects in images. It also has great potential in industrial inspection due to its speed, accuracy and unsupervised training.

This paper presents an original method of modelling the colour distributions of images using 2D Gaussian functions and its application to flaw detection in industrial inspection. 2D Gaussian functions are used to model the colours that appear in the non-flawed images in an unsupervised manner. Pixels under test are compared to the colour distribution from training images. 140 images have been tested and the results are given. This method has a wide range of applications for detecting colour separable objects in images. It also has great potential in industrial inspection due to its speed, accuracy and unsupervised training.

We propose an autonomous control systemfor Aerial Service Vehicles capable of performing inspection tasks in buildings and industrial plants. In this paper, we present the applicative domain, the high-level control architecture along with some empirical results. The system is assessed on real-world and simulated scenarios representing industrial environments

This thesis describes research in the area of automated industrial inspection using machine vision systems. It is anticipated that the algorithms described will contribute to the design of a machine vision system for the automatic surface inspection of cylindrical pellets. Firstly, the acquisition and segmentation of pellet tray images using area capture is described. Individual pellets are segmented from a pellet tray image by a novel system using the Radon transform coupled with data clustering. Subsequent to the segmentation, the linking of four pellet views depicting the entire circumferential area of the pellet is described along with a simple technique to compensate for intensity variations brought about by imaging the three-dimensional cylindrical surface of the pellet. The image processing techniques of filtering, edge detection, thresholding and morphology are used in the segmentation of grey level pellet defect images. The grey level pellet images are low-pass filtered and binary images formed using edge detection with thresholding. Binary morphology operators are then used in conjunction with a termination condition based on the number of objects in the image to ensure homogenous defect representations. The problem of overlapping defects is addressed, resulting in a second algorithm using the Radon transform coupled with data clustering. Prior to classification salient features are extracted from a set of synthetic binary defect images to form feature vectors. The novel idea of image object classification using 100% fuzzy inference is described, and results are shown to be superior to results obtained by feature space classifiers. The sub-classification of crack defects is carried out using a heuristic classifier, and the parameterisation of pellet defects is described.

This thesis describes research in the area of automated industrial inspection using machine vision systems. It is anticipated that the algorithms described will contribute to the design of a machine vision system for the automatic surface inspection of cylindrical pellets. Firstly, the acquisition and segmentation of pellet tray images using area capture is described. Individual pellets are segmented from a pellet tray image by a novel system using the Radon transform coupled with data clustering. Subsequent to the segmentation, the linking of four pellet views depicting the entire circumferential area of the pellet is described along with a simple technique to compensate for intensity variations brought about by imaging the three-dimensional cylindrical surface of the pellet. The image processing techniques of filtering, edge detection, thresholding and morphology are used in the segmentation of grey level pellet defect images. The grey level pellet images are low-pass filtered and binary images formed using edge detection with thresholding. Binary morphology operators are then used in conjunction with a termination condition based on the number of objects in the image to ensure homogenous defect representations. The problem of overlapping defects is addressed, resulting in a second algorithm using the Radon transform coupled with data clustering. Prior to classification salient features are extracted from a set of synthetic binary defect images to form feature vectors. The novel idea of image object classification using 100% fuzzy inference is described, and results are shown to be superior to results obtained by feature space classifiers. The sub-classification of crack defects is carried out using a heuristic classifier, and the parameterisation of pellet defects is described.

This thesis describes research in the area of automated industrial inspection using machine vision systems. It is anticipated that the algorithms described will contribute to the design of a machine vision system for the automatic surface inspection of cylindrical pellets. Firstly, the acquisition and segmentation of pellet tray images using area capture is described. Individual pellets are segmented from a pellet tray image by a novel system using the Radon transform coupled with data clustering. Subsequent to the segmentation, the linking of four pellet views depicting the entire circumferential area of the pellet is described along with a simple technique to compensate for intensity variations brought about by imaging the three-dimensional cylindrical surface of the pellet. The image processing techniques of filtering, edge detection, thresholding and morphology are used in the segmentation of grey level pellet defect images. The grey level pellet images are low-pass filtered and binary images formed using edge detection with thresholding. Binary morphology operators are then used in conjunction with a termination condition based on the number of objects in the image to ensure homogenous defect representations. The problem of overlapping defects is addressed, resulting in a second algorithm using the Radon transform coupled with data clustering. Prior to classification salient features are extracted from a set of synthetic binary defect images to form feature vectors. The novel idea of image object classification using 100% fuzzy inference is described, and results are shown to be superior to results obtained by feature space classifiers. The sub-classification of crack defects is carried out using a heuristic classifier, and the parameterisation of pellet defects is described.

This thesis describes research in the area of automated industrial inspection using machine vision systems. It is anticipated that the algorithms described will contribute to the design of a machine vision system for the automatic surface inspection of cylindrical pellets. Firstly, the acquisition and segmentation of pellet tray images using area capture is described. Individual pellets are segmented from a pellet tray image by a novel system using the Radon transform coupled with data clustering. Subsequent to the segmentation, the linking of four pellet views depicting the entire circumferential area of the pellet is described along with a simple technique to compensate for intensity variations brought about by imaging the three-dimensional cylindrical surface of the pellet. The image processing techniques of filtering, edge detection, thresholding and morphology are used in the segmentation of grey level pellet defect images. The grey level pellet images are low-pass filtered and binary images formed using edge detection with thresholding. Binary morphology operators are then used in conjunction with a termination condition based on the number of objects in the image to ensure homogenous defect representations. The problem of overlapping defects is addressed, resulting in a second algorithm using the Radon transform coupled with data clustering. Prior to classification salient features are extracted from a set of synthetic binary defect images to form feature vectors. The novel idea of image object classification using 100% fuzzy inference is described, and results are shown to be superior to results obtained by feature space classifiers. The sub-classification of crack defects is carried out using a heuristic classifier, and the parameterisation of pellet defects is described.

Idag genomför OptoNovas system en mängd avsyningar hos tillverkande företag där defekta produkter sorteras ut. Systemen samlar in data för analys och delar av resultatet sparas i textbaserade loggfiler. Här finns en möjlighet att utnyttja loggdatan till en betydligt högre grad för att få ut maximalt med information. Examensarbetet omfattar en utredning av vilka loggdata som är användbara och på vilket sätt de kan visualiseras för att uppnå så hög förståelse och användbarhet som möjligt, följt av en implementation. Arbetet resulterade i en standard för loggdata som utgörs av en databas samt en visualiseringsapplikation skriven i C++ och Qt. Det som krävs för att använda visualiseringsapplikationen är en databas på föreslaget format samt enkla bilder på aktuella artiklar som datan ska visualiseras för. Med hjälp av Visualiseringsapplikationen kan användaren se statistik över de olika artiklarna som avsyningssystemet avsynar. Applikationen visar information om var på artiklarna defekterna förekommer, vid vilka tidpunkter det uppstår flest defekter m.m. Visualiseringsapplikationen kommer att fungera som ett hjälpmedel för att förbättra felsökningen i tillverkningsindustrin bland de företag som valt att använda OptoNovas avsyningssystem., Today OptoNovas systems perform lots of visual inspections for production companies where defect products are sorted out. The systems collects data to analyze and part of the results are stored in text based log files. Here is an opportunity to make higher use of the data to maximize the amount of useful information. The thesis work comprise an inquiry of which log data that is useful and how it is possible to visualize them to attain as high understanding and usefulness as possible, followed by an implementation. The thesis work resulted in a standard for the log data, which is represented in a database, and a visualization application written in C++ and the Qt framework. To use the visualization application, database on the proposed format is needed and pictures for the articles the data will be visualized for. With the visualization application the user can provide statistics for the current articles that the inspection system inspects. The application shows information about the position for the defects, on which time there are most inspected defects and more. The visualization application is going to work as a tool to provide better troubleshooting in production companies that uses inspection systems from OptoNova AB.

Idag genomför OptoNovas system en mängd avsyningar hos tillverkande företag där defekta produkter sorteras ut. Systemen samlar in data för analys och delar av resultatet sparas i textbaserade loggfiler. Här finns en möjlighet att utnyttja loggdatan till en betydligt högre grad för att få ut maximalt med information. Examensarbetet omfattar en utredning av vilka loggdata som är användbara och på vilket sätt de kan visualiseras för att uppnå så hög förståelse och användbarhet som möjligt, följt av en implementation. Arbetet resulterade i en standard för loggdata som utgörs av en databas samt en visualiseringsapplikation skriven i C++ och Qt. Det som krävs för att använda visualiseringsapplikationen är en databas på föreslaget format samt enkla bilder på aktuella artiklar som datan ska visualiseras för. Med hjälp av Visualiseringsapplikationen kan användaren se statistik över de olika artiklarna som avsyningssystemet avsynar. Applikationen visar information om var på artiklarna defekterna förekommer, vid vilka tidpunkter det uppstår flest defekter m.m. Visualiseringsapplikationen kommer att fungera som ett hjälpmedel för att förbättra felsökningen i tillverkningsindustrin bland de företag som valt att använda OptoNovas avsyningssystem., Today OptoNovas systems perform lots of visual inspections for production companies where defect products are sorted out. The systems collects data to analyze and part of the results are stored in text based log files. Here is an opportunity to make higher use of the data to maximize the amount of useful information. The thesis work comprise an inquiry of which log data that is useful and how it is possible to visualize them to attain as high understanding and usefulness as possible, followed by an implementation. The thesis work resulted in a standard for the log data, which is represented in a database, and a visualization application written in C++ and the Qt framework. To use the visualization application, database on the proposed format is needed and pictures for the articles the data will be visualized for. With the visualization application the user can provide statistics for the current articles that the inspection system inspects. The application shows information about the position for the defects, on which time there are most inspected defects and more. The visualization application is going to work as a tool to provide better troubleshooting in production companies that uses inspection systems from OptoNova AB.

Postprint (published version)

Postprint (published version)

This thesis describes the development of a range sensing system. The goal was to create a range sensor that is robust and flexible so that a number of applications within the forest products manufacturing environment can be addressed. Features of the system include: the capability of producing spatially registered image pairs of range and intensity, the ability to generate both range and intensity very quickly, the applicability to a wide variety of industrial applications, the ability to handle large depth-of-field range sensing problems, the ability to do real-time data processing, and the capability to do extensive system diagnostics under complete software control. A triangulation based plane-of-light optical method is employed to extract range information. The research shows that this method suits range sensing applications where conveyor belts are involved. An in-depth study of the triangulation method is included. In the study it shows that this method also supports large depth-of-field range sensing. A dedicated signal processing hardware, built on the Micro Channel interface, performs pipelined image processing and generates range and intensity images in a spatially registered form. The hardware is designed to support several modes of operation, for the purpose of facilitating optical adjustments and calibrations. The hardware self-diagnostic facility is also included in the design. A memory management scheme is provided that facilitates real-time data processing of the range and intensity images. The experiments show that this scheme provides a real-time environment for software processing. This thesis also contains a theory exploring the limitations of the measurement accuracy of the range detection algorithm employed in the prototype system. The maximum data generation rate of the prototype system is 380 range/intensity lines per second at 128 range/intensity pixels per line. Several proposals toward future work are included that aim at improving the spe

The purpose of the study was to investigate the applicability of laboratory findings of vigilance behavior to an industrial inspection task. Vigilance behavior refers to the attentiveness of the human operator, or, more specifically, to human capabilities for detecting small changes in stimulus events over relatively long periods of sustained monitoring. (Author)

Supersedes AD-073 385.

Learning features for object detection and recognition with deep learning has received increasing attention in the past several years and recently attained widespread popularity. In this PhD thesis we investigate its applications to the automatic surface inspection system of our industrial partner ArcelorMittal, for classification and segmentation problems. Currently employed algorithms, in fact, use fixed feature extractors which are hard to tune and require extensive prior-knowledge. Our work, instead, focuses on learnable systems that can be used to improve recognition and detection without requiring hard to obtain task-specific domain knowledge. For image classification we propose extensions to max-pooling convolutional networks, so that they can be applied to solve the general defect classification problem via a new pooling and feature encoding schemes. State-of-the-art deep learning algorithms for object detection/segmentation have reached outstanding performance given high-quality annotated data. Unfortunately, they do not meet the required processing speeds of steel industry. We propose an architecture that does not suffer the same computational bottleneck (1500-fold speed-up) while retaining equal performance. To further advance the field we study the learning of morphological operators, largely used in industry. Only few attempts have been proposed in the literature, but no approach has ever considered the problem in its generality because of its hard formulation. We tackle it from a different perspective and introduce a learnable framework which seamlessly integrates morphological operators; hence bringing these powerful tools to deep learning for the first time. Re-engineering an industrial system requires time. In order to deliver an immediate return we investigate metric learning problems to boost performance of currently used features. Our multimodal similarity sensitive hashing model scales well to web-scale datasets and, thanks to the binary represe

Learning features for object detection and recognition with deep learning has received increasing attention in the past several years and recently attained widespread popularity. In this PhD thesis we investigate its applications to the automatic surface inspection system of our industrial partner ArcelorMittal, for classification and segmentation problems. Currently employed algorithms, in fact, use fixed feature extractors which are hard to tune and require extensive prior-knowledge. Our work, instead, focuses on learnable systems that can be used to improve recognition and detection without requiring hard to obtain task-specific domain knowledge. For image classification we propose extensions to max-pooling convolutional networks, so that they can be applied to solve the general defect classification problem via a new pooling and feature encoding schemes. State-of-the-art deep learning algorithms for object detection/segmentation have reached outstanding performance given high-quality annotated data. Unfortunately, they do not meet the required processing speeds of steel industry. We propose an architecture that does not suffer the same computational bottleneck (1500-fold speed-up) while retaining equal performance. To further advance the field we study the learning of morphological operators, largely used in industry. Only few attempts have been proposed in the literature, but no approach has ever considered the problem in its generality because of its hard formulation. We tackle it from a different perspective and introduce a learnable framework which seamlessly integrates morphological operators; hence bringing these powerful tools to deep learning for the first time. Re-engineering an industrial system requires time. In order to deliver an immediate return we investigate metric learning problems to boost performance of currently used features. Our multimodal similarity sensitive hashing model scales well to web-scale datasets and, thanks to the binary represe

In the industrial production process, unsupervised visual inspection methods have obvious advantages over supervised visual inspection methods due to the scarcity of defect samples, annotation costs and the uncertainty of defect generation. Currently, unsupervised defect detection and localization methods have demonstrated significant improvements in detection accuracy to find numerous applications in industrial inspection. Nonetheless, the complexity of these methods limits their practical application. In this paper, we integrate the FastFlow model plugin as a probability distribution by introducing a simpler and lightweight CNN pre-trained backbone. Concurrently, various training strategies are employed to optimize the 2D Flow module within the Lightweight unsupervised flow model (LightFlow). Notably, the number of model parameters in the LightFlow model is only 1/4 of the original model size of the typical Vision Transformer (ViT) model CaiT. Thereby, this offers heightened training efficiency and speed. Therefore, extensive experimental results on three challenging anomaly detection datasets (MVTec AD, VisA, and BTAD) using various CNN backbones and multiple current state-of-the-art vision algorithms demonstrate the effectiveness of our approach. Specifically, the existing method can achieve 99.1% and 95.2% image-level AUROC (area under the receiver operating characteristic) in MVTec AD and VisA, respectively. IEEE

Remote control technology for machines and robots has experienced significant advancement in many domains where visual information delivery is essential. Safety management at airports is one field that benefits from remote control systems, enabling operators to scan the airstrip for obstacles and debris. Depth perception and proper view position are critical in remote control systems, where operators need to accurately perceive 3D coordinates and maintain an appropriate perspective for performing tasks from a distance. This Demo paper presents a laboratory platform for an experimental study on human interaction with remote control systems using visual interfaces for inspection tasks to enhance airport safety management. The platform can evaluate the user experience of interfaces provided by First Person View, Third Person View, and Augmentation technologies. This platform enables exploration through controlled experiments and by user tests. These provide an avenue for assessing how these interfaces may affect human performance, depth perception, and user experience when conducting inspection tasks remotely. The findings will shed light on the strengths and limitations of each interface type, offering insights into their potential applications in various domains such as industrial inspection, surveillance, and remote exploration.

Remote control technology for machines and robots has experienced significant advancement in many domains where visual information delivery is essential. Safety management at airports is one field that benefits from remote control systems, enabling operators to scan the airstrip for obstacles and debris. Depth perception and proper view position are critical in remote control systems, where operators need to accurately perceive 3D coordinates and maintain an appropriate perspective for performing tasks from a distance. This Demo paper presents a laboratory platform for an experimental study on human interaction with remote control systems using visual interfaces for inspection tasks to enhance airport safety management. The platform can evaluate the user experience of interfaces provided by First Person View, Third Person View, and Augmentation technologies. This platform enables exploration through controlled experiments and by user tests. These provide an avenue for assessing how these interfaces may affect human performance, depth perception, and user experience when conducting inspection tasks remotely. The findings will shed light on the strengths and limitations of each interface type, offering insights into their potential applications in various domains such as industrial inspection, surveillance, and remote exploration., Part of ISBN 9798350361582QC 20240827

This paper introduces an innovative approach to simulating Electrical Impedance Tomography (EIT) through MATLAB, aimed at advancing the accuracy and reliability of internal imaging using electrodes. We address the critical challenge of reconstructing interior images with high fidelity by simulating inhomogeneous mediums. Our methodology involves the generation of synthetic datasets, encompassing various inhomogeneity scenarios, followed by applying forward solutions to ascertain voltage measurements indicative of interior conductivity variations. The research emphasizes the creation of an adaptive framework capable of simulating real-world scenarios within a controlled digital environment, thereby enhancing the predictive capabilities of EIT systems in diverse applications ranging from medical imaging to industrial inspection.

This paper introduces an innovative approach to simulating Electrical Impedance Tomography (EIT) through MATLAB, aimed at advancing the accuracy and reliability of internal imaging using electrodes. We address the critical challenge of reconstructing interior images with high fidelity by simulating inhomogeneous mediums. Our methodology involves the generation of synthetic datasets, encompassing various inhomogeneity scenarios, followed by applying forward solutions to ascertain voltage measurements indicative of interior conductivity variations. The research emphasizes the creation of an adaptive framework capable of simulating real-world scenarios within a controlled digital environment, thereby enhancing the predictive capabilities of EIT systems in diverse applications ranging from medical imaging to industrial inspection.

The estimation of 6D object poses is a fundamental task in many computer vision applications. Particularly, in high risk scenarios such as human-robot interaction, industrial inspection, and automation, reliable pose estimates are crucial. In the last years, increasingly accurate and robust deep-learning-based approaches for 6D object pose estimation have been proposed. Many top-performing methods are not end-to-end trainable but consist of multiple stages. In the context of deep uncertainty quantification, deep ensembles are considered as state of the art since they have been proven to produce well-calibrated and robust uncertainty estimates. However, deep ensembles can only be applied to methods that can be trained end-to-end. In this work, we propose a method to quantify the uncertainty of multi-stage 6D object pose estimation approaches with deep ensembles. For the implementation, we choose SurfEmb as representative, since it is one of the top-performing 6D object pose estimation approaches in the BOP Challenge 2022. We apply established metrics and concepts for deep uncertainty quantification to evaluate the results. Furthermore, we propose a novel uncertainty calibration score for regression tasks to quantify the quality of the estimated uncertainty., Comment: 8 pages

Three-dimensional point cloud anomaly detection that aims to detect anomaly data points from a training set serves as the foundation for a variety of applications, including industrial inspection and autonomous driving. However, existing point cloud anomaly detection methods often incorporate multiple feature memory banks to fully preserve local and global representations, which comes at the high cost of computational complexity and mismatches between features. To address that, we propose an unsupervised point cloud anomaly detection framework based on joint local-global features, termed PointCore. To be specific, PointCore only requires a single memory bank to store local (coordinate) and global (PointMAE) representations and different priorities are assigned to these local-global features, thereby reducing the computational cost and mismatching disturbance in inference. Furthermore, to robust against the outliers, a normalization ranking method is introduced to not only adjust values of different scales to a notionally common scale, but also transform densely-distributed data into a uniform distribution. Extensive experiments on Real3D-AD dataset demonstrate that PointCore achieves competitive inference time and the best performance in both detection and localization as compared to the state-of-the-art Reg3D-AD approach and several competitors.

Automated and autonomous industrial inspection is a longstanding research field, driven by the necessity to enhance safety and efficiency within industrial settings. In addressing this need, we introduce an autonomously navigating robotic system designed for comprehensive plant inspection. This innovative system comprises a robotic platform equipped with a diverse array of sensors integrated to facilitate the detection of various process and infrastructure parameters. These sensors encompass optical (LiDAR, Stereo, UV/IR/RGB cameras), olfactory (electronic nose), and acoustic (microphone array) capabilities, enabling the identification of factors such as methane leaks, flow rates, and infrastructural anomalies. The proposed system underwent individual evaluation at a wastewater treatment site within a chemical plant, providing a practical and challenging environment for testing. The evaluation process encompassed key aspects such as object detection, 3D localization, and path planning. Furthermore, specific evaluations were conducted for optical methane leak detection and localization, as well as acoustic assessments focusing on pump equipment and gas leak localization., Comment: Submitted for publication in IEEE Sensors Journal

Within (semi-)automated visual inspection, learning-based approaches for assessing visual defects, including deep neural networks, enable the processing of otherwise small defect patterns in pixel size on high-resolution imagery. The emergence of these often rarely occurring defect patterns explains the general need for labeled data corpora. To not only alleviate this issue but to furthermore advance the current state of the art in unsupervised visual inspection, this contribution proposes a DifferNet-based solution enhanced with attention modules utilizing SENet and CBAM as backbone - AttentDifferNet - to improve the detection and classification capabilities on three different visual inspection and anomaly detection datasets: MVTec AD, InsPLAD-fault, and Semiconductor Wafer. In comparison to the current state of the art, it is shown that AttentDifferNet achieves improved results, which are, in turn, highlighted throughout our quantitative as well as qualitative evaluation, indicated by a general improvement in AUC of 94.34 vs. 92.46, 96.67 vs. 94.69, and 90.20 vs. 88.74%. As our variants to AttentDifferNet show great prospects in the context of currently investigated approaches, a baseline is formulated, emphasizing the importance of attention for anomaly detection., Comment: Accepted to CVPRW 2023: VISION'23 - 1st workshop on Vision-based InduStrial InspectiON (Extended Abstract). arXiv admin note: substantial text overlap with arXiv:2311.02747

Visual anomaly detection aims to identify anomalous regions in images through unsupervised learning paradigms, with increasing application demand and value in fields such as industrial inspection and medical lesion detection. Despite significant progress in recent years, there is a lack of comprehensive benchmarks to adequately evaluate the performance of various mainstream methods across different datasets under the practical multi-class setting. The absence of standardized experimental setups can lead to potential biases in training epochs, resolution, and metric results, resulting in erroneous conclusions. This paper addresses this issue by proposing a comprehensive visual anomaly detection benchmark, \textbf{\textit{ADer}}, which is a modular framework that is highly extensible for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. Additionally, we have open-sourced the GPU-assisted \href{https://pypi.org/project/ADEval}{ADEval} package to address the slow evaluation problem of metrics like time-consuming mAU-PRO on large-scale data, significantly reducing evaluation time by more than \textit{1000-fold}. Through extensive experimental results, we objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection. We hope that \textbf{\textit{ADer}} will become a valuable resource for researchers and practitioners in the field, promoting the development of more robust and generalizable anomaly detection systems. Full codes have been attached in Appendix and open-sourced at \url{https://github.com/zhangzjn/ader}.

Accurate and fast 3D imaging of specular surfaces still poses major challenges for state-of-the-art optical measurement principles. Frequently used methods, such as phase-measuring deflectometry (PMD) or shape-from-polarization (SfP), rely on strong assumptions about the measured objects, limiting their generalizability in broader application areas like medical imaging, industrial inspection, virtual reality, or cultural heritage analysis. In this paper, we introduce a measurement principle that utilizes a novel technique to effectively encode and decode the information contained in a light field reflected off a specular surface. We combine polarization cues from SfP with geometric information obtained from PMD to resolve all arising ambiguities in the 3D measurement. Moreover, our approach removes the unrealistic orthographic imaging assumption for SfP, which significantly improves the respective results. We showcase our new technique by demonstrating single-shot and multi-shot measurements on complex-shaped specular surfaces, displaying an evaluated accuracy of surface normals below $0.6^\circ$.

During the spring term of 2021 a thesis was carried out at Volvo Trucks GTO. Volvo Trucks is one of the largest and most successful actors in the automotive industry and delivers trucks all over the world. The company offers trucks that can be customized and adapted according to the customer wishes. Offering highly customizable trucks puts a lot of stresses on the system which requires that CAD-drawings, component documentation and all databases are synchronized and updated according to the changes done on the truck. It has been observed within the company that the actual dimensions of the trucks do not always coincide with the system dimensions. The deviations in dimensions of the truck causes problems during transports as the trucks either exceeds or fall under the system dimensions. The objective, having the above in mind, has been to achieve a better understanding of how different measurement techniques can be used as an automated system for industrial inspection. Using the system, the aim was to collect data to enable a root-cause analysis on the deviations. In the project, different sensors have been tested in order to identify a capable sensor for a future automated system. During the project it became apparent that the objective of using the sensor for collecting data could not be achieved. There were several factors that led to this conclusion. But mainly it was due to the lack of time, but also due to the difficulties faced when calibrating and programming the sensor to achieve reliable and accurate measurements. However, the struggle of trying to use the sensor provided a lot of knowledge and data which was considered to be of great value for Volvo. The collection of data and analysis was still accomplished, using a manual ruler stick which worked better than expected. Some pre stated factors and causes, and consequences were justified by the measurements and analyzation of data, while others were rejected. A recommendation for the one who will continue i, Under vårterminen 2021 genomfördes en avhandling på Volvo GTO. Volvo Lastvagnar är en av de största aktörerna inom fordonsindustrin och levererar lastbilar över hela världen. Företaget erbjuder lastbilar som kan anpassas och modiferas efter kundens önskemål. Att erbjuda lastbilar med hög anpassnings- och förändringsgrad innebär en hel del påfrestningar på systemet vilket kräver att CAD-ritningar, komponentdokumentation och alla databaser synkroniseras och uppdateras enligt de ändringar som gjorts på lastbilen. Det har observerats inom företaget att lastbilarnas faktiska dimensioner inte alltid sammanfaller med systemdimensionerna. Avvikelserna i lastbilens dimensioner orsakar problem vid transporter eftersom lastbilarna antingen överskrider eller faller under systemdimensionerna. Målet, med ovanstående i åtanke, har varit att uppnå en bättre förståelse för hur olika mättekniker kan användas vid ett automatiserat system för industriell inspektion. Med hjälp av systemet var syftet att samla in data för att möjliggöra en rotorsaksanalys av avvikelserna. I projektet har olika sensorer testats för att kunna användas i ett framtida automatiskt system för inspektion. Under projektets gång blev det uppenbart att målet att använda sensorn för insamling av data inte var möjlig. Det fanns flera faktorer som ledde till denna slutsats. Främst berodde det på tidsbrist, men också på de svårigheter som uppstod vid kalibrering och programmering av sensorerna för att uppnå tillförlitliga och exakta mätresultat. Strävan att försöka använda sensorerna gav dock mycket kunskap och data som ansågs vara av stort värde för Volvo. Trots problemen och hindren så genomfördes datainsamling och analys med hjälp av en manuell linjalsticka som fungerade bättre än förväntat. Vissa, på förhand givna faktorer och orsaker, och konsekvenser bekräftades av mätningarna och analysen av data, medan andra avvisades. En rekommendation för den som kommer tar över projektet är att fortsätta samla in data, både, During the spring term of 2021 a thesis was carried out at Volvo Trucks GTO. Volvo Trucks is one of the largest and most successful actors in the automotive industry and delivers trucks all over the world. The company offers trucks that can be customized and adapted according to the customer wishes. Offering highly customizable trucks puts a lot of stresses on the system which requires that CAD-drawings, component documentation and all databases are synchronized and updated according to the changes done on the truck. It has been observed within the company that the actual dimensions of the trucks do not always coincide with the system dimensions. The deviations in dimensions of the truck causes problems during transports as the trucks either exceeds or fall under the system dimensions. The objective, having the above in mind, has been to achieve a better understanding of how different measurement techniques can be used as an automated system for industrial inspection. Using the system, the aim was to collect data to enable a root-cause analysis on the deviations. In the project, different sensors have been tested in order to identify a capable sensor for a future automated system. During the project it became apparent that the objective of using the sensor for collecting data could not be achieved. There were several factors that led to this conclusion. But mainly it was due to the lack of time, but also due to the difficulties faced when calibrating and programming the sensor to achieve reliable and accurate measurements. However, the struggle of trying to use the sensor provided a lot of knowledge and data which was considered to be of great value for Volvo. The collection of data and analysis was still accomplished, using a manual ruler stick which worked better than expected. Some pre stated factors and causes, and consequences were justified by the measurements and analyzation of data, while others were rejected. A recommendation for the one who will continue i

The utilization of autonomous unmanned aerial vehicles (UAVs) has increased rapidly due to their ability to perform a variety of tasks, including industrial inspection. Conducting testing with actual flights within industrial facilities proves to be both expensive and hazardous, posing risks to the system, the facilities, and their personnel. This paper presents an innovative and reliable methodology for developing such applications, ensuring safety and efficiency throughout the process. It involves a staged transition from simulation to reality, wherein various components are validated at each stage. This iterative approach facilitates error identification and resolution, enabling subsequent real flights to be conducted with enhanced safety after validating the remainder of the system. Furthermore, this article showcases two use cases: wind turbine inspection and photovoltaic plant inspection. By implementing the suggested methodology, these applications were successfully developed in an efficient and secure manner.

[SPA] Las rápidas mejoras en la tecnología del Internet de las Cosas (IoT) han provocado transformaciones revolucionarias en varias industrias. El objetivo de esta tesis es estudiar la relevancia del IoT industrial y su influencia en las actividades industriales, a saber, mejoras en la eficiencia operativa, la calidad de los productos, el consumo de energía de los sensores, la latencia y la fiabilidad. La primera parte del estudio se dedica a las redes LoRaWAN de largo alcance y al uso de la inteligencia artificial, en particular los algoritmos de aprendizaje por refuerzo, para mejorar la fiabilidad de la transmisión y minimizar el consumo de energía de los nodos. Mediante estos algoritmos inteligentes, la red IoT podría ajustar los parámetros de transmisión de forma adaptativa. Gracias al aprovechamiento de los algoritmos emergentes de IA, como el aprendizaje profundo y la visión por ordenador, se puede automatizar y optimizar el proceso de inspección, garantizando una mayor precisión y una detección más rápida de los defectos en entornos industriales. Además, los robots colaborativos, que trabajan junto a operadores humanos, agilizan las tareas y optimizan la asignación de recursos. Además, la tesis expone la importancia de la IA para ofrecer soluciones de computación móvil en el borde (Mobile Edge Computing, MEC) en sectores industriales mediante el uso de drones. Los drones habilitados para IA proporcionan procesamiento de datos en tiempo real, eliminando los retrasos en la transmisión de datos y dando soporte a aplicaciones sensibles al tiempo, al tiempo que mejoran la eficiencia general de los procesos industriales. Los resultados de este estudio se suman al campo demostrando el enorme potencial del IoT industrial, la IA, las tecnologías de detección, los robots colaborativos y los drones para mejorar los procesos industriales., [ENG] Rapid improvements in Internet of Things (IoT) technology have resulted in disruptive transformations across several industries. The purpose of this thesis is to investigate the relevance of Industrial IoT and its influence on industrial activities, namely enhancing operational efficiency, product quality, sensor energy consumption, latency, and dependability. The first section of the study investigates the Long-Range Wide Area Networks LoRaWAN and the use of Artificial Intelligence particularly Reinforcement Learning algorithms to improve transmission reliability and minimize energy usage in small end nodes. Using these intelligent algorithms, the IoT network could adjust transmission settings adaptation, mitigating communication losses and improving the overall system. In addition, this thesis delves into enhancing industrial inspection quality for defect detection. By leveraging emerging AI algorithms, such as Deep Learning and Computer Vision, the inspection process can be automated and optimized, ensuring higher precision and faster defect detection in industrial settings. Furthermore, Collaborative robots, working alongside human operators, streamline tasks and optimize resource allocation, while energy AI models offer intelligent energy management solutions, further increasing productivity and cost-effectiveness. Moreover, the thesis emphasizes the importance of AI in offering mobile Edge Computing solutions in industrial sectors using drones. Drones with AI capabilities provide real-time data processing, eliminating data transmission delays and supporting time-sensitive applications while improving overall industrial process efficiency. This study's findings add to the area by demonstrating the enormous potential of Industrial IoT, AI, sensing technologies, collaborative robots, and drones in improving industrial processes., [FRE] [FRE] Les améliorations rapides dans la technologie de l'Internet des objets (IoT) ont entraîné des transformations disruptives dans plusieurs industries. Le but de cette thèse est d'investiguer la pertinence de l'IoT industriel et son influence sur les activités industrielles, notamment l'amélioration de l'efficacité opérationnelle, de la qualité des produits, de la consommation d'énergie des capteurs, de la latence et de la fiabilité. La première section de l'étude examine les réseaux longue portée à large zone (LoRaWAN) et l'utilisation de l'intelligence artificielle, en particulier des algorithmes d'apprentissage par renforcement, pour améliorer la fiabilité de la transmission et minimiser la consommation d'énergie dans les petits nœuds de terminaison. En utilisant ces algorithmes intelligents, le réseau IoT pourrait ajuster les paramètres de transmission de manière adaptative, atténuant les pertes de communication et améliorant l'ensemble du système. De plus, cette thèse se penche sur l'amélioration de la qualité de l'inspection industrielle pour la détection des défauts. En exploitant les algorithmes émergents d'IA, tels que l'apprentissage profond et la vision par ordinateur, le processus d'inspection peut être automatisé et optimisé, garantissant une plus grande précision et une détection plus rapide des défauts dans les environnements industriels. En outre, les robots collaboratifs, travaillant aux côtés des opérateurs humains, rationalisent les tâches et optimisent l'allocation des ressources, tandis que les modèles d'IA énergétique offrent des solutions inteligentes de gestion de l'énergie, augmentant davantage la productivité et la rentabilité. De plus, la thèse souligne l'importance de l'IA dans la proposition de solutions de calcul en périphérie mobile dans les secteurs industriels en utilisant des drones. Les drones avec des capacités d'IA permettent un traitement des données en temps réel, éliminant les retards de transmission des données et so

Reconstruction of shapes and appearances of thin film objects can be applied to many fields such as industrial inspection, biological analysis, and archaeologic research. However, it comes with many challenging issues because the appearances of thin film can change dramatically depending on view and light directions. The appearance is deeply dependent on not only the shapes but also the optical parameters of thin film. In this paper, we propose a novel method to estimate shapes and film thickness. First, we narrow down candidates of zenith angle by degree of polarization and determine it by the intensity of thin film which increases monotonically along the zenith angle. Second, we determine azimuth angle from occluding boundaries. Finally, we estimate the film thickness by comparing a look-up table of color along the thickness and zenith angle with captured images. We experimentally evaluated the accuracy of estimated shapes and appearances and found that our proposed method is effective., 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),27-30 June 2016,Las Vegas, NV, USA

© Optical Society of America. We thank Agustín González-Cano, Juan C. Martínez-Antón, and José Alonso for kind and useful comments. Also we thank Manuel Fuentes, Centro de Estudios e Investigaciones Técnicas, for concern for the project and kindness., A lineal illuminating system based on an optical fiber array is studied. It can be applied to automatic industrial inspection systems, e.g., those used for chromatic classification of ceramic floor tiles. Improvement in the existing system’s performance is achieved by using a collimated beam, as shown by both photometric analysis and experimental results. This system has been implemented as part of a real-time controlling tool in a production line., Depto. de Óptica, Fac. de Ciencias Físicas, TRUE, pub

Autonomous robotic systems have seen extensive deployment across domains such as manufacturing, industrial inspection, transportation, and planetary surface exploration. A crucial requirement for these systems is navigating from an initial to a final position, while avoiding potential collisions with obstacles en route. This challenging task of devising collision-free trajectories, formally termed as motion planning, is of prime importance in robotics. Traditional motion planning approaches encounter scalability challenges when planning in higher-dimensional state-spaces. Moreover, they rarely consider robot dynamics during the planning process. To address these limitations, a class of probabilistic planning methods called Sampling-Based Motion Planning (SBMP) has gained prominence. SBMP strategies exploit probabilistic techniques to construct motion planning solutions. In this dissertation, our focus turns towards feasible SBMP algorithms that prioritize rapidly discovering solutions while considering robot kinematics and dynamics. These algorithms find utility in quickly solving complex problems (e.g., Alpha puzzle) where obtaining any feasible solution is considered as an achievement. Furthermore, they find practical use in computationally constrained systems and in seeding time-consuming optimal solutions. However, many existing feasible SBMP approaches assume the ability to find precise trajectories that exactly connect two states in a robot's state space. This challenge is framed as the Two-Point Boundary Value Problem (TPBVP). But finding closed-form solutions for the TPBVP is difficult, and numerical approaches are computationally expensive and prone to precision and stability issues. Given these complexities, this dissertation's primary focus resides in the development of SBMP algorithms for different scenarios where solving the TPBVP is challenging. Our work addresses four distinct scenarios -- two for single-agent systems and two for multi-agent systems.

Anomaly detection is the task of recognising novel samples which deviate significantly from pre-establishednormality. Abnormal classes are not present during training meaning that models must learn effective rep-resentations solely across normal class data samples. Deep Autoencoders (AE) have been widely used foranomaly detection tasks, but suffer from overfitting to a null identity function. To address this problem, weimplement a training scheme applied to a Denoising Autoencoder (DAE) which introduces an efficient methodof producing Adversarially Learned Continuous Noise (ALCN) to maximally globally corrupt the input priorto denoising. Prior methods have applied similar approaches of adversarial training to increase the robustnessof DAE, however they exhibit limitations such as slow inference speed reducing their real-world applicabilityor producing generalised obfuscation which is more trivial to denoise. We show through rigorous evaluationthat our ALCN method of regularisation during training improves AUC performance during inference whileremaining efficient over both classical, leave-one-out novelty detection tasks with the variations-: 9 (normal)vs. 1 (abnormal) & 1 (normal) vs. 9 (abnormal); MNIST - AUCavg: 0.890 & 0.989, CIFAR-10 - AUCavg: 0.670& 0.742, in addition to challenging real-world anomaly detection tasks: industrial inspection (MVTEC-AD -AUCavg: 0.780) and plant disease detection (Plant Village - AUC: 0.770) when compared to prior approaches., Comment: 18th International Conference on Computer Vision Theory and Applications

We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution across $>$110 cm$^2$ areas over multi-millimeter axial ranges. Our computational microscope, termed STARCAM (Scanning Topographic All-in-focus Reconstruction with a Computational Array Microscope), features a parallelized, 54-camera architecture with 3-axis translation to capture, for each sample of interest, a multi-dimensional, 2.1-terabyte (TB) dataset, consisting of a total of 224,640 9.4-megapixel images. We developed a self-supervised neural network-based algorithm for 3D reconstruction and stitching that jointly estimates an all-in-focus photometric composite and 3D height map across the entire field of view, using multi-view stereo information and image sharpness as a focal metric. The memory-efficient, compressed differentiable representation offered by the neural network effectively enables joint participation of the entire multi-TB dataset during the reconstruction process. To demonstrate the broad utility of our new computational microscope, we applied STARCAM to a variety of decimeter-scale objects, with applications ranging from cultural heritage to industrial inspection.

Most advanced unsupervised anomaly detection (UAD) methods rely on modeling feature representations of frozen encoder networks pre-trained on large-scale datasets, e.g. ImageNet. However, the features extracted from the encoders that are borrowed from natural image domains coincide little with the features required in the target UAD domain, such as industrial inspection and medical imaging. In this paper, we propose a novel epistemic UAD method, namely ReContrast, which optimizes the entire network to reduce biases towards the pre-trained image domain and orients the network in the target domain. We start with a feature reconstruction approach that detects anomalies from errors. Essentially, the elements of contrastive learning are elegantly embedded in feature reconstruction to prevent the network from training instability, pattern collapse, and identical shortcut, while simultaneously optimizing both the encoder and decoder on the target domain. To demonstrate our transfer ability on various image domains, we conduct extensive experiments across two popular industrial defect detection benchmarks and three medical image UAD tasks, which shows our superiority over current state-of-the-art methods., Comment: NeurIPS 2023 Poster