Search

A robust computer vision system is proposed to visualize subsurface barely visible impact damage (BVID) in composite structures through a simple image correlation technique together with a damage imaging condition. This system uses a digital camera to record a video of the surface motion, capturing micron-scale dynamic movement from guided waves propagating on the surface of the structure generated via a sweeping frequency excitation (chirp) up to the ultrasonic frequency range. As the excitation frequency changes during the chirp, waves become trapped within this damaged region, forming standing waves to generate local resonance at specific frequencies. This localized resonance accumulates high wave energy in the region, generating a higher transverse displacement at the damage site compared to the remaining part of the structure. In this work, a simple image correlation technique is proposed to correlate each filtered video frame with the temporal mean of the filtered wavefield video to highlight standing waves at localized damage. The proposed image correlation technique is distinct from digital image correlation (DIC) since it does not use correlation for subset matching to track the movement of surface patterns between two images (video frames). Instead, it uses correlation to directly quantify similarities between corresponding image pixels (windows). Utilizing a single camera greatly simplifies system complexity and hence enhances the practicality and potential for real-time performance over a recently developed technique that utilized 3D DIC with a stereo camera for vision-based BVID detection. To realize the aim, work was conducted in two sequential steps: (1) off-axis 2D DIC was employed rather than 3D DIC to examine the potential of employing a single camera to capture images and extract not only in-plane displacement but also a fraction of transverse displacement in which local resonance is dominant and (2) image correlation was then employed, supplanting the off-axis 2D DIC image processing involved in the first step, to highlight the damage with significantly less processing complexity. This proposed technique using a zero-mean normalized cross-correlation imaging condition, rather than the total wave energy used for DIC-based approaches, is efficient and effective for identifying regions of minute surface movement from local resonance within the damage region without the use of the computationally intensive interpolation to get the sub-pixel gray level information followed by subset matching employed in DIC-based image processing. Two geometrically identical CFRP composite honeycomb panels that had been subjected to low-velocity impacts were used for verification and validation, and two excitation location configurations were tested for each panel. Damaged images produced with correlation for a 100 mm × 100 mm field of view using a single 3-s video, or a total of 19,200 video frames, show accurate damage imaging capabilities regardless of excitation location that exceed that of DIC techniques and is comparable to benchmark damage images obtained from laser Doppler vibrometry, ultrasonic C-scans, and X-ray CT scans. This success of correlation-based imaging of subsurface BVID demonstrates substantial improvements in efficiency and practicality and shows high potential for in situ and real-time computer vision-based nondestructive inspection or structural health monitoring of subsurface BVID in composite aircraft and other critical structures. [ABSTRACT FROM AUTHOR]

In sports science, thermal imaging is applied to investigate various questions related to exercise-induced stress response, muscle fatigue, anomalies, and diseases. Infrared thermography monitors thermal radiation from the skin's surface over time. For further analysis, regions of interest are extracted and statistically analyzed. Although computer vision algorithms have grown in recent years due to data-driven approaches, this is not the case for detailed segmentation in thermal images. In a supervised manner, machine learning optimizations require a large amount of training data with input and ground truth output data. Unfortunately, obtaining annotated data are a costly problem that increases with the complexity of the task. For semantic segmentation, pixel-wise label masks must be created by experts. Few datasets meet the needs of sports scientists and physicians to perform advanced applications of thermal computer vision during physical activity and generate new insights in their fields. In this paper, a new method is introduced to transfer segmentation masks from the vision domain to the thermal domain with a stereo-calibrated time-of-flight camera and high-resolution mid-wave infrared camera. A post-processing procedure is then utilized to obtain dense pixel masks for the posterior legs during walking and running on a treadmill. The developed StereoThermoLegs dataset is based on 14 participants and includes 11 subjects for training with 12,826 thermograms and the remaining three individuals for testing with 3433 images. A deep neural network was trained with the DeepLabv3+ architecture, the AdaBelief optimizer, and Dice loss as a benchmark. After 29 epochs, the test set achieved an average intersection over union of 0.66. The analysis of the posterior leg region, specifically the left and right calf, offered the most insights, with values of 0.83 and 0.83, respectively. The first multimodal stereo dataset containing synchronized visual and thermal images of a runner's back provides a starting point for data-driven segmentation tasks in sports science and medicine. Our technique allows for automatic production of customized datasets for deep learning, accelerating the implementation of baseline outcomes for newly identified areas of interest in thermal imaging, while bypassing the requirement for extensive manual annotation. The approach is not exclusive to stereo rig and segmentation tasks utilizing RGBD and thermal cameras, but can be applied to other imaging tasks and modalities. [ABSTRACT FROM AUTHOR]

Profilometry is proposed as a novel non-intrusive image-based technique to capture the profile of the air–water interface as a dense point cloud. It can be classified as an active stereo-vision method applied to the study of gravity-driven water waves and specifically developed to be used in large hydrodynamic laboratories. As an active vision technique, it relies on the use of light sources, and as a stereo technique, it requires one or more high-speed camera pairs for imaging the same scene synchronously. To enhance the visibility of the laser lights on the wave profile, the water surface is sprayed with water droplets. Profilometry, compared to standard wave probes, can be considered as an alternative source of information that can augment spatial resolution to the identification of the air–water interface to capture nonlinear wave-evolution mechanisms and violent wave–body interactions. Its feasibility and accuracy are examined preliminarily in a small-scale flume and then in a large-scale towing tank using long-crested wave scenarios, including regular, irregular, and focused gravity-driven waves, without the presence of a structure. The values of the wave steepness examined were various and included also quite steep cases with nearly vertical wave fronts. Role played by parameters of the technique, as well as of its setup in capturing the wave features are also analysed, with the aim to provide a useful guidance for other researchers that intent to use and develop further this approach. [ABSTRACT FROM AUTHOR]

In the process of scratch defect detection in textured materials, there are often problems of low efficiency in traditional manual detection, large errors in machine vision, and difficulty in distinguishing defective scratches from the background texture. In order to solve these problems, we developed an enhanced scratch defect detection system for textured materials based on optimized photometric stereo vision and FFT-Gabor filtering. We designed and optimized a novel hemispherical image acquisition device that allows for selective lighting angles. This device integrates images captured under multiple light sources to obtain richer surface gradient information for textured materials, overcoming issues caused by high reflections or dark shadows under a single light source angle. At the same time, for the textured material, scratches and a textured background are difficult to distinguish; therefore, we introduced a Gabor filter-based convolution kernel, leveraging the fast Fourier transform (FFT), to perform convolution operations and spatial domain phase subtraction. This process effectively enhances the defect information while suppressing the textured background. The effectiveness and superiority of the proposed method were validated through material applicability experiments and comparative method evaluations using a variety of textured material samples. The results demonstrated a stable scratch capture success rate of 100% and a recognition detection success rate of 98.43% ± 1.0%. [ABSTRACT FROM AUTHOR]

Corner detection is responsible for accurate camera calibration, which is an essential task for binocular three-dimensional (3D) reconstruction. In microscopic scenes, binocular 3D reconstruction has significant potential to achieve fast and accurate measurements. However, traditional corner detectors and calibration patterns (checkerboard) performed poorly in microscopic scenes due to the non-uniform illumination and the shallow depth of field of the microscope. In this paper, we present a novel method for detecting grid junction corners based on image segmentation, offering a robust alternative to the traditional checkerboard pattern. Model fitting was utilized to obtain the coordinates at a sub-pixel level. The procedures of the proposed method were elaborated, including image segmentation, corner prediction, and model fitting. The mathematical model was established to describe the grid junction. The experiment was conducted using both synthetic and real data and the experimental result shows that this method achieves high precision and is robust to image blurring, indicating this method is suitable for microscope camera calibration. [ABSTRACT FROM AUTHOR]

Line structured light (LSL) measurement systems can obtain high accuracy profiles, but the overall clarity relies greatly on the sampling interval of the scanning process. Photometric stereo (PS), on the other hand, is sensitive to tiny features but has poor geometrical accuracy. Cooperative measurement with these two methods is an effective way to ensure precision and clarity results. In this paper, an LSL-PS cooperative measurement system is brought out. The calibration methods used in the LSL and PS measurement system are given. Then, a data fusion algorithm with adaptive weights is proposed, where an error function that contains the 3D point cloud matching error and normal vector error is established. The weights, which are based on the angles of adjacent normal vectors, are also added to the error function. Afterward, the fusion results can be obtained by solving linear equations. From the experimental results, it can be seen that the proposed method has the advantages of both the LSL and PS methods. The 3D reconstruction results have the merits of high accuracy and high clarity. [ABSTRACT FROM AUTHOR]

3D digital-image correlation (3D-DIC) is a non-contact optical technique for full-field shape, displacement, and deformation measurement. Given the high experimental hardware costs associated with 3D-DIC, the development of high-fidelity 3D-DIC simulations holds significant value. However, existing research on 3D-DIC simulation was mainly carried out through the generation of random speckle images. This study innovatively proposes a complete 3D-DIC simulation method involving optical simulation and mechanical simulation and integrating 3D-DIC, virtual stereo vision, and image super-resolution reconstruction technology. Virtual stereo vision can reduce hardware costs and eliminate camera-synchronization errors. Image super-resolution reconstruction can compensate for the decrease in precision caused by image-resolution loss. An array of software tools such as ANSYS SPEOS 2024R1, ZEMAX 2024R1, MECHANICAL 2024R1, and MULTIDIC v1.1.0 are used to implement this simulation. Measurement systems based on stereo vision and virtual stereo vision were built and tested for use in 3D-DIC. The results of the simulation experiment show that when the synchronization error of the basic stereo-vision system (BSS) is within 10 − 3  time steps, the reconstruction error is within 0.005 mm and the accuracy of the virtual stereo-vision system is between the BSS's synchronization error of 10 − 7  and 10 − 6  time steps. In addition, after image super-resolution reconstruction technology is applied, the reconstruction error will be reduced to within 0.002 mm. The simulation method proposed in this study can provide a novel research path for existing researchers in the field while also offering the opportunity for researchers without access to costly hardware to participate in related research. [ABSTRACT FROM AUTHOR]

This work focuses on investigating the accuracy of 3D reconstructions from fixed stereo-photogrammetric monitoring systems through different camera calibration procedures. New reliable and effective calibration methodologies that require minimal effort and resources are presented. A full-format camera equipped with fixed 50 and 85 mm focal length optics is considered, but the methodologies are general and can be applied to other systems. Four different calibration strategies are considered: (i) full-field calibration (FF); (ii) multi-image on-the-job calibration (MI); (iii) point cloud-based calibration (PC); and (iv) self (on-the-job) calibration (SC). To evaluate the calibration strategies and assess their actual performance and practicality, two test sites are used. The full-field calibration, while very reliable, demands significant effort if it needs to be repeated. The multi-image strategy emerges as a favourable compromise, offering good results with minimal effort for its realisation. The point cloud-based method stands out as the optimal choice, balancing ease of implementation with quality results; however, it requires a reference 3D point cloud model. On-the-job calibration with monitoring images is the simplest but least reliable option, prone to uncertainty and potential inaccuracies, and should hence be avoided. Ultimately, prioritising result reliability over absolute accuracy is paramount in continuous monitoring systems. [ABSTRACT FROM AUTHOR]

The accuracy of the pointing error in the subreflector is of great importance to the large steerable radio telescope, so the measurement of the six degree-of- freedom (DOF) poses of the subreflector is crucial, which are measured by the detector and the laser array. The errors in the pose measurement system of the subreflector contain random measure errors and fitting iteration errors; all of them will add up to the pose measurement, which affects the accuracy of the six DOF parameters. This paper proposed the accuracy analysis method in the pose measurement of the subreflector of large antennas, which includes the error in the variable model in laser three-dimensional point space linear fitting, iterative accuracy analysis of spatial equations in Newton's downhill method, and the analysis of the adjustment theory of the Bursa model. The experimental results show that the measure time in six-DOF poses of the subreflector in a large- aperture antenna is at most 0.5 s, while the measurement accuracy error of translation is within 0.0131 mm and the error of rotation is within 0.2323°, which indicates that the pose measurement method of the subreflector in large antennas based on stereo structured light is efficient and applicable enough to analyze the measurement system accuracy. [ABSTRACT FROM AUTHOR]

Although amount of works are focused on monocular depth estimation, these works mainly study on the RGB spectrum, which has a poor performance on the case of nighttime, low light environment and even zero light environment. The images of other spectrum provide an opportunity to obtain depth without an active projector source. In this paper, we design a three-step architecture to realize monocular depth estimation by fusing cross-spectral stereo information. In the first step, we employ Spectral Translation Network to tackle with the problem that different spectral images have huge appearance differences and propose a disparity reservation loss to reserve disparity when translating. In the second step, we use Monocular Estimation Network to predict disparity of the principal spectrum, which is used for test. In the third step, we retrain the Spectral Translation Network with a generative optimization loss to improve the quality of image translation. Experiments show that our method achieves preeminent performance and reaches real-time speed. [ABSTRACT FROM AUTHOR]

The proliferation of digital technologies is substantially transforming inspection methodologies for construction activities. Although the implementation of a three-dimensional (3D) model has emerged as an advantageous, feasible inspection application, the selection of the most suitable 3D models is challenging due to multiple technology options. The primary objectives of this study were to investigate current trends and identify future technologies for 3D models in the construction industry. This study utilized systematic reviews by identifying and selecting quality journals, analyzing selected articles, and conducting content analysis and meta-analysis to identify dominant themes in 3D models. Results showed that the top technologies used to model construction projects are building information models, remote sensing, stereo vision system/photo processing programs, and augmented reality/virtual reality. The main benefits and challenges of these technologies for modeling were also determined. This study identified three areas with significant knowledge gaps for future research: (1) the amalgamation of two or more technologies to overcome project obstacles; (2) solution optimization for inspections in remote areas; and (3) the development of algorithm-based technologies. This research contributes to the body of knowledge by exploring current trends and future directions of 3D model technologies in the construction industry. [ABSTRACT FROM AUTHOR]

The use of visual rationality in planning and designing indoor 3D spaces can effectively enhance the quality of environmental art design. For the rational planning and design of 3D spaces, it is essential to assign attribute values to the rational evaluation index of 3D space planning, determine the weight vector of rational evaluation factors for indoor 3D space planning, and accomplish the visual rationality planning and design of indoor 3D spaces. 3D reconstruction is one of the primary objectives and focal points of machine vision research. The principle is to extract 3D information from the 2D image captured by the machine’s sensors in order to reconstruct a 3D scene model. The traditional method constructs the evaluation index system for indoor 3D space planning to determine the weight of the evaluation index for alternative points. However, it overlooks the weight vector of the evaluation factors, leading to low accuracy in rational planning. This paper optimizes the effect of structured light reconstruction using photometric stereo vision. Firstly, the text delves into the impact of light sources on the 3D reconstruction effect. Subsequently, a fusion algorithm combining 3D and structured light reconstruction methods is introduced, leveraging artificial intelligence (AI) technology. Experimental results show that this algorithm has a superior 3D reconstruction effect. [ABSTRACT FROM AUTHOR]

In the past decades, a variety of learning-based algorithms have been emerged to try to explore a better solution for stereo matching by leveraging various machine learning algorithms. For enriching learning-based stereo matching algorithm's methodologies, we cast the disparity estimation as a regression problem by leveraging Structured Support Vector Machine (SSVM) in this paper. There are three categories of features have been extracted on account of disparity cues for training the SSVM. Particularly, one of the three feature is named as 'Coarse-to-Fine PatchMatch Filtering', which effectively exploits region and pixel disparity cues. For attaining region disparity cues, we adopt MeshStereo and MeshStereo with Cross-Scale algorithms; for attaining pixel disparity cues, PatchMatch and Cross-Scale PatchMatch stereo matching algorithms are utilized. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed algorithm reveals comparable accuracy with other challenging learning-based ones. It is worth pointing out that our proposal performs over several orders of magnitude faster than others on training time. [ABSTRACT FROM AUTHOR]

Carotid pulse signal (CPS) is one of the most important biomarkers for evaluating cardiovascular risk. To realize easy‐to‐operate and accurate CPS detection, a single‐camera telecentric imaging stereo‐digital image correlation (stereo‐DIC) system is established. The system uses a specially designed colour separation device and a single colour camera attached with a telecentric lens to record images of blue and red colours from different optical paths. By processing the recorded two sub‐channel colour images using regular stereo‐DIC algorithm, full‐field 3D displacements of the skin on the neck directly over the carotid artery are obtained. The maximum fluctuations in the out‐of‐plane displacement direction are extracted as CPS, based on which velocity, acceleration and frequency can be further extracted. The performance of the established system was demonstrated by detecting the pulse signal of a volunteer before and after exercise. [ABSTRACT FROM AUTHOR]

In order to solve the problem of underwater in-situ measurement of fuel assembly bending and twisting deformation, an underwater in-situ deformation measurement method of fuel assembly based on line laser 3D vision is proposed. The calibration of single-layer refraction-based underwater camera and plane-constrained optical plane calibration algorithms are used to solve the calibration problem of single measurement unit; ten groups of measurement units based on three-dimensional calibration tooling are quickly calibrated in situ to solve the problem of global coordinate unification of the measurement system and the bending and torsion accuracy of the calibration tooling is measured by laser tracking instrument or the comparative experimental verification of the measurement accuracy. The study shows that the developed underwater n-situ deformation measurement system for fuel assemblies based on line laser 3D vision has an error limit of 0.15° for twist measurement and 0.3 mm for bending measurement, and the whole measurement time is about 1 s, which is better than that of the existing measurement system in terms of the measurement efficiency and measurement error limit. [ABSTRACT FROM AUTHOR]

In this paper, the influence of parallax and image shape type on stereo vision perception is quantitatively explored by using RDS stimuli based on AR equipment through subjective scoring. Results show that an angular parallax amplitude between 0.6 ° and 1.2 ° can bring the best stereo sense. When amplitude ranged between 0 ° and 1.2°, the best visual comfort and the fastest image fusion speed could be achieved. [ABSTRACT FROM AUTHOR]

Given the prevalent issues surrounding accuracy and efficiency in contemporary stereo matching algorithms, this research introduces an innovative image segmentation based approach. The proposed methodology integrates residual and Swim Transformer modules into the established 3D Unet framework, yielding the Res-Swim-UNet image segmentation model. The algorithm estimates the disparateness of segmented outputs by employing regression techniques, culminating in a comprehensive disparity map. Experimental findings underscore the superiority of the proposed algorithm across all evaluated metrics. Specifically, the proposed network demonstrates marked improvements, with IoU and mPA enhancements of 2.9% and 162%, respectively. Notably, the average matching error rate of the algorithm registers at 2.02%, underscoring its efficacy in achieving precise stereoscopic matching. Moreover, the model’s enhanced generalization capability and robustness underscore its potential for widespread applicability. [ABSTRACT FROM AUTHOR]

In response to the low matching accuracy of stereo matching algorithms in image regions with specular reflection, this paper proposes a multidimensional fusion stereo matching algorithm named MFANet. The algorithm embeds a multispectral attention module into the residual feature extraction network, utilizing two‐dimensional discrete cosine transforms to extract frequency features. In the pyramid pooling module, a coordinated attention mechanism is introduced to capture relevant positional information. In the cost aggregation part, the MFANet algorithm incorporates a three‐dimensional attention mechanism, focusing on the more important semantic information in high‐level features. By combining detailed information from low‐level features, semantic information from high‐level features, and contextual information, the algorithm generates features that are more conducive to disparity prediction. The MFANet algorithm is evaluated on three standard datasets (SceneFlow, KITTI2015, and KITTI2012). Experimental results demonstrate its robustness against specular reflection interference, accurate prediction of disparities in specular reflection pathological regions, and promising application prospects. [ABSTRACT FROM AUTHOR]

Large hoop truss antennas are locked into a stable structure during deployment in space. The key to characterizing the performance of each mechanism, and discovering problems in the deployment process, is the three-dimensional motion of the antenna. Therefore, it is important to measure antenna motion with high precision for antenna design. Contact measurement requires sensors to be attached to the object, affecting the motion of the antenna; thus, noncontact measurement is the preferred measurement method. For a large antenna, a large field of view and observation angle are required to measure the deployment process. Traditional binocular stereo vision measurement is not sufficiently accurate. In this paper, 36 cameras were combined into an array to measure antenna deployment. A high-precision triggering system was designed, a calibration method suitable for large-scale, multiangle measurement was developed, and a reconstruction method based on the geometric constraints was proposed, greatly improving measurement resolution. The camera array was used to measure the motion of a large hoop truss antenna with a diameter of 4 m during deployment. Based on the measured parameters, the angular velocity of each antenna mechanism was calculated, and the synchronization of the antenna structure was analyzed. [ABSTRACT FROM AUTHOR]

Rubber production is a labour-intensive process. In order to reduce the needed number of workers and the waste of material, the level of digitalisation should be increased. One part of the production is the extrusion to produce gaskets and similar objects. An automated observation of the continuous rubber extrudate enables an early intervention in the production process. In addition to chemical monitoring, the geometrical observation of the extrudate is an important aspect of the quality control. For this purpose, we use laser triangulation sensors (LTS) at the beginning and the end of the cooling phase of the extrudate after the extrusion. The LTS acquire two-dimensional profiles at a constant frequency. To combine these profiles into a three-dimensional model of the extrudate, the movement of the extrudate has to be tracked. Since the extrudate is moved over a conveyor belt, the conveyor belt can be tracked by a stereo camera system to deduce the movement of the extrudate. For the correct usage of the tracking, the orientation between the LTS and the stereo camera system needs to be known. A calibration object that considers the different data from the LTS and the camera system was developed to determine the orientation. Afterwards, the orientation can be used to combine arbitrary profiles. The measurement setup, consisting of the LTS, the stereo camera system and the conveyor belt, is explained. The development of the calibration object, the algorithm for evaluating the orientation data and the combination of the LTS profiles are described. Finally, experiments with real extrusion data are presented to validate the results and compare three variations of data evaluation. Two use the calculated orientation, but have different tracking approaches and one without any orientation necessary. [ABSTRACT FROM AUTHOR]

Along with the improvement of quality requirements industrial production, surface inspection of workpiece has gradually become an indispensable and important process in the production of the workpiece. Aiming at the traditional methods in textured paper inspection, there are problems of low efficiency and large error based on machine vision, we propose a "photometric stereo vision + fast Fourier enhancement + feature fusion" composite structure inspection method. First, as the traditional CCD camera produces obvious noise and scratches, which are difficult to distinguish from the background texture area, we propose combining the photometric stereo vision measurement algorithm to get the surface gradient information of the textured paper to obtain more gradient texture information; and then realize the secondary enhancement of the image through Fourier transform in spatial and frequency domains. Second, as the textured paper scratches are difficult to detect, the features are difficult to extract, and the threshold boundary is difficult to define, we propose dynamic threshold segmentation through multi-feature fusion to realize the surface scratch detection work of textured paper. We designed experiments using more than 300 different textured papers; and the results show that the composite structure detection method proposed in this paper is feasible and has advantages. [ABSTRACT FROM AUTHOR]

In modern industrial production, high-temperature environments are commonplace, posing significant challenges to equipment stability, safety, and production efficiency. Machine vision, as an effective automated inspection technology, has attracted extensive attention in high-temperature settings. However, the unique conditions of high temperatures, such as significant thermal noise and optical interference, demand enhanced performance from machine vision systems. The second law of thermodynamics provides a theoretical foundation for understanding these challenges, emphasizing the increase of entropy in energy transformation and transfer processes, and guides the design and optimization of machine vision systems in high-temperature environments. This paper aims to comprehensively explore the application of machine vision based on the second law of thermodynamics in high-temperature industrial inspection, focusing on two core issues: the impact of thermodynamic parameters on the performance of machine vision systems and the technology for analyzing high-temperature industrial infrared images using multiscale entropy. By thoroughly analyzing how thermodynamic parameters influence the design and implementation of machine vision systems, and by developing infrared image processing algorithms adapted to high temperatures, this study seeks to enhance the efficiency and accuracy of machine vision technology in high-temperature industrial applications, providing theoretical support and technical guidance for the advancement of intelligent manufacturing. [ABSTRACT FROM AUTHOR]

Background: The root of a plant is a fundamental organ for the multisensory perception of the environment. Investigating root growth dynamics as a mean of their interaction with the environment is of key importance for improving knowledge in plant behaviour, plant biology and agriculture. To date, it is difficult to study roots movements from a dynamic perspective given that available technologies for root imaging focus mostly on static characterizations, lacking temporal and three-dimensional (3D) spatial information. This paper describes a new system based on time-lapse for the 3D reconstruction and analysis of roots growing in hydroponics. Results: The system is based on infrared stereo-cameras acquiring time-lapse images of the roots for 3D reconstruction. The acquisition protocol guarantees the root growth in complete dark while the upper part of the plant grows in normal light conditions. The system extracts the 3D trajectory of the root tip and a set of descriptive features in both the temporal and frequency domains. The system has been used on Zea mays L. (B73) during the first week of growth and shows good inter-reliability between operators with an Intra Class Correlation Coefficient (ICC) > 0.9 for all features extracted. It also showed measurement accuracy with a median difference of < 1 mm between computed and manually measured root length. Conclusions: The system and the protocol presented in this study enable accurate 3D analysis of primary root growth in hydroponics. It can serve as a valuable tool for analysing real-time root responses to environmental stimuli thus improving knowledge on the processes contributing to roots physiological and phenotypic plasticity. [ABSTRACT FROM AUTHOR]

Monocular stereo vision has excellent application prospects in the field of microrobots. On the basis of the geometric model of bifocal imaging, this paper proposes a monocular depth perception method by liquid zoom imaging. Firstly, the configuration of a monocular liquid vision system for depth measurement is presented, and the working mechanism of the system is analyzed through theoretical derivation. Then, to eliminate the influence of optical axis drift induced by the liquid gravity factor on the measurement results, the target image area is used as the calculation feature instead of the image vector length. A target area calculation method based on chain code classification and strip segmentation is proposed. Furthermore, in response to the fluctuation problem of liquid lens focal power caused by factors such as temperature and object distance, a dynamic focal length model of the liquid zoom imaging system is constructed after precise calibration of the focal power function. Finally, a testing experiment is designed to validate the proposed method. The experimental results show that the average error of depth perception methods is 4.30%, and its measurement time is only on the millisecond scale. Meanwhile, the proposed method has good generalization performance. [ABSTRACT FROM AUTHOR]

In many cases, the Digital Surface Models (DSMs) and Digital Elevation Models (DEMs) are obtained with Light Detection and Ranging (LiDAR) or stereo matching. As an active method, LiDAR is very accurate but expensive, thus often limiting its use in small-scale acquisition. Stereo matching is suitable for large-scale acquisition of terrain information as the increase of satellite stereo sensors. However, underperformance of stereo matching easily occurs in textureless areas. Accordingly, this study proposed a Shading Aware DSM GEneration Method (SADGE) with high resolution multi-view satellite images. Considering the complementarity of stereo matching and Shape from Shading (SfS), SADGE combines the advantage of stereo matching and SfS technique. First, an improved Semi-Global Matching (SGM) technique is used to generate an initial surface expressed by a DSM; then, it is refined by optimizing the objective function which modeled the imaging process with the illumination, surface albedo, and normal object surface. Different from the existing shading-based DEM refinement or generation method, no information about the illumination or the viewing angle is needed while concave/convex ambiguity can be avoided as multi-view images are utilized. Experiments with ZiYuan-3 and GaoFen-7 images show that the proposed method can generate higher accuracy DSM (12.5–56.3% improvement) with sound overall shape and temporarily detailed surface compared with a software solution (SURE) for multi-view stereo. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Remote Sensing is the property of Editorial Office of Journal of Remote Sensing & Science Publishing Co. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The use of stereo camera systems to measure fish body size has gained attention in aquaculture systems management. However, the different correlations between swimming depth and size according to fish species can lead to measurement bias depending on camera range and position. This study accordingly applied a fish behavior model based on the Boid model to simulate the relationship among camera position, viewing range, and measurement results considering swimming depth behavior. A total of 722 simulated fish (yellowtail Seriola quinqueradiata) were placed into a rectangular aquaculture net pen model. Virtual cameras were located at different depths along the side, above, and below the pen to measure the total length of individual fish within view. Horizontally oriented cameras closer to the surface captured significantly larger/smaller individuals when the behavior model assumed that larger individuals swim at shallower/deeper depths; vertically oriented cameras facing down from the water surface or up from the bottom net provided mean total length measurements might close to the actual mean total length of the population when a wide viewing range was provided. The proposed simulation method can be applied to preliminarily evaluate the measurement biases inherent to fish behavior and thereby minimize their influence on camera measurements. [ABSTRACT FROM AUTHOR]

We present a near real-time solution for 3D reconstruction from aerial images captured by consumer UAVs. Our core idea is to simplify the multi-view stereo problem into a series of two-view stereo matching problems. Our method applies to UAVs equipped with only one camera and does not require special stereo-capturing setups. We found that the neighboring two video frames taken by UAVs flying at a mid-to-high cruising altitude can be approximated as left and right views from a virtual stereo camera. By leveraging GPU-accelerated real-time stereo estimation, efficient PnP correspondence solving algorithms, and an extended Kalman filter, our system simultaneously predicts scene geometry and camera position/orientation from the virtual stereo cameras. Also, this method allows for the user selection of varying baseline lengths, which provides more flexibility given the trade-off between camera resolution, effective measuring distance, flight altitude, and mapping accuracy. Our method outputs dense point clouds at a constant speed of 25 frames per second and is validated on a variety of real-world datasets with satisfactory results. [ABSTRACT FROM AUTHOR]

Recent years have witnessed increasing interest in adversarial attacks on images, while adversarial video attacks have seldom been explored. In this paper, we propose a sparse adversarial attack strategy on videos (DeepSAVA). Our model aims to add a small human-imperceptible perturbation to the key frame of the input video to fool the classifiers. To carry out an effective attack that mirrors real-world scenarios, our algorithm integrates spatial transformation perturbations into the frame. Instead of using the l p norm to gauge the disparity between the perturbed frame and the original frame, we employ the structural similarity index (SSIM), which has been established as a more suitable metric for quantifying image alterations resulting from spatial perturbations. We employ a unified optimisation framework to combine spatial transformation with additive perturbation, thereby attaining a more potent attack. We design an effective and novel optimisation scheme that alternatively utilises Bayesian Optimisation (BO) to identify the most critical frame in a video and stochastic gradient descent (SGD) based optimisation to produce both additive and spatial-transformed perturbations. Doing so enables DeepSAVA to perform a very sparse attack on videos for maintaining human imperceptibility while still achieving state-of-the-art performance in terms of both attack success rate and adversarial transferability. Furthermore, built upon the strong perturbations produced by DeepSAVA, we design a novel adversarial training framework to improve the robustness of video classification models. Our intensive experiments on various types of deep neural networks and video datasets confirm the superiority of DeepSAVA in terms of attacking performance and efficiency. When compared to the baseline techniques, DeepSAVA exhibits the highest level of performance in generating adversarial videos for three distinct video classifiers. Remarkably, it achieves an impressive fooling rate ranging from 99.5% to 100% for the I3D model, with the perturbation of just a single frame. Additionally, DeepSAVA demonstrates favourable transferability across various time series models. The proposed adversarial training strategy is also empirically demonstrated with better performance on training robust video classifiers compared with the state-of-the-art adversarial training with projected gradient descent (PGD) adversary. • Sparse attacks on video models: perturb fewer frames to gain high fooling rate. • Combining additive and spatial perturbations to enhance attacking performance. • Using SSIM instead of l p -norm to maintain the human perception. • Applying Bayesian Optimisation to identify the most critical frame to perturb. • A new adversarial training method based on combination of diverse perturbations. [ABSTRACT FROM AUTHOR]

Real-time stereo matching with high accuracy is a dynamic research topic; it is attractive in diverse computer vision applications. This paper presents a stereo-matching algorithm that produces high-quality disparity map while maintaining real-time performance. The proposed stereo-matching method is based on three per-pixel difference measurements with adjustment elements. The absolute differences and the gradient matching are combined with a colour-weighted extension of complete rank transform to reduce the effect of radiometric distortion. The disparity calculation is realized using improved dynamic programming that optimizes along and across all scanlines. It solves the inter-scanline inconsistency problem and increases the matching accuracy. The proposed algorithm is implemented on parallel high-performance graphic hardware using the Compute Unified Device Architecture to reach over 240 million disparity evaluations per second. The processing speed of our algorithm reaches 98 frames per second on 240 × 320-pixel images and 32 disparity levels. Our method ranks fourth in terms of accuracy and runtime for quarter-resolution images in the Middlebury stereo benchmark. [ABSTRACT FROM AUTHOR]

Display technologies have evolved over the years. It is critical to develop practical HDR capturing, processing, and display solutions to bring 3D technologies to the next level. Depth estimation of multi-exposure stereo image sequences is an essential task in the development of cost-effective 3D HDR video content. In this paper, we develop a deep architecture for multi-exposure stereo depth estimation. The proposed architecture has two novel components. First, the stereo matching technique used in traditional stereo depth estimation is revamped. For the stereo depth estimation component of our architecture, a mono-to-stereo transfer learning approach is deployed. The proposed formulation circumvents the cost volume construction requirement, which is replaced by a dual-encoder single-decoder CNN with different weights for feature fusion. EfficientNet-based blocks are used to learn the disparity. Secondly, we combine disparity maps obtained from the stereo images at different exposure levels using a robust disparity feature fusion approach. The disparity maps obtained at different exposures are merged using weight maps calculated for different quality measures. The final predicted disparity map obtained is more robust and retains best features that preserve the depth discontinuities. The proposed CNN offers flexibility to train using standard dynamic range stereo data or with multi-exposure low dynamic range stereo sequences. In terms of performance, the proposed model surpasses state-of-the-art monocular and stereo depth estimation methods, both quantitatively and qualitatively, on challenging Scene flow and differently exposed Middlebury stereo datasets. The architecture performs exceedingly well on complex natural scenes, demonstrating its usefulness for diverse 3D HDR applications. [ABSTRACT FROM AUTHOR]

At present, in end-to-end network combined with deep learning in stereo matching, how to perceive ample image details within an acceptable range of computational cost, so as to improve the final matching accuracy, has become a hot research topic. In this paper, we design a cascaded disparity correction network based on DispNet, which can refine the initial disparity image by combining the process output results in the trunk network. The method proposed in this paper is named ETE–DC–CNN (end to end–disparity correction–convolutional neural network). In the trunk network, the mixed dilated convolution module is employed to extract the features of the original image, and the feature image is constructed into a 3D disparity space volume through depth-separable convolution, after which the initial disparity image can be obtained through 2D encoder–decoder module. In the correction network, the gradient information of the initial feature image is combined to reconstruct the modified matching cost body guided by the initial disparity image and then a smaller encoder–decoder module is trained and processed to make the final result at the sub-pixel level. The algorithm in this paper has been verified in SceneFlow and KITTI 2012/2015 data sets. The method proposed in this article achieves 3.46, 2.12, 1.19, and 0.6 results in the non-occluded area and 4.04, 2.55, 1.58, and 0.7 results in the global area on the Er (> 2 px), Er (> 3 px), Er (> 5 px), and Epe indicators in the KITTI 2012 test set, which are higher than those of the backbone network DispNet. [ABSTRACT FROM AUTHOR]

The past decade has been a boom in lunar exploration. China, India, Japan and other countries have successfully landed landers or rovers on the lunar surface (Wu et al., 2014, 2018, 2020; Prasad et al., 2023). Future missions to explore the Moon are focusing on the lunar south pole (Peña-Asensio et al., 2024). The solar altitude angle at the lunar south pole is extremely low, resulting in low solar irradiance and large areas often in dim light or shadows. The permanently shadowed regions (PSRs) at the lunar south pole are also likely containing substantial amounts of water ice (Li et al., 2018). Future lunar robots exploring the lunar south pole will need to operate in low light or shadowed regions, making camera sensors sensitive to the dim-light environments necessary for these robots. Common night vision sensors usually use near-infrared cameras. However, sensors based on passive infrared technology have image resolution limited by several factors, including the intensity of infrared radiation emitted by the object, the sensitivity of the camera, and the performance of the optical system. For instance, thermal imagers typically have a resolution of 388*284 pixels only. We have developed a dim-light adaptive camera (DimCam) that is ultra-sensitive to the varying illumination conditions driven by AI to achieve high-definition imaging of 1080P or above, for future lunar robots operating in shadows or dim-light regions. The DimCam integrates two starlight-level ultra-sensitive imaging sensors connected by a rigid base to provide stereo vision in low illumination environment. An AI edge computing unit is embedded inside the DimCam to adaptively denoise and enhance image quality. The AI module uses an end-to-end image denoising network to identify and remove noises in the images more accurately by utilizing depth information from the stereo sensors. Compared with traditional monocular denoising algorithms, the denoising network based on stereo vision can significantly improve denoising effects and efficiency by enhancing the signal-to-noise ratio of the data input in the front end. The superposition of overlapping scenes can be regarded as a delayed exposure. Concurrently, the residual analysis of the aligned images aids in noise identification. In addition, for pixels obscured by noise, more accurate pixel values can be restored through interpolation or replacement using depth information obtained from the stereo sensors. Subsequently, a pre-trained lightweight deep network modified from Zero-DCE (Guo et al., 2020) is used for image quality enhancement in terms of brightness and contrast, providing high-quality images even in low-light environments for subsequent applications, such as positioning and navigation of robots, 3D mapping of the surrounding environment, and autonomous driven. We have tested the DimCam in a simulated environment in the laboratory, and the results show that the DimCam has promising performances and great potential for various applications. [ABSTRACT FROM AUTHOR]

This document is a guest editorial from the journal Friction, focusing on the special issue of Artificial Intelligence (AI) and emerging computational approaches for tribology. Tribology research, which studies friction and wear, has traditionally relied on extensive experimentation. However, AI and computational approaches offer new opportunities to explore complex processes in tribology and push the boundaries of research. The special issue includes 15 papers covering various aspects of AI and machine learning in tribology, such as wear assessment, lubrication performance, contact wear prediction, and composite materials design. These articles highlight the transformative impact of AI in tribology research and provide valuable insights for future innovations. [Extracted from the article]

The significance of medical image synthesis has exponentially grown due to constrained medical resources, making it a critical component in numerous clinical applications. This process facilitates the generation of high-quality, multi-modal medical images, ultimately enhancing medical image diagnostics. Currently, prevailing medical image synthesis methodologies primarily rely on voxel-based or GAN-based strategies to address substantial challenges arising from disparities in various imaging principles and significant noise. However, these methodologies rarely consider the intensity distribution difference among multi-modal or multi-parameters medical images, which generates unstable and unexplainable results. In response to these limitations, we propose a novel approach-a general variation-driven neural network for medical image synthesis that considers explicit data distribution. Within this method, we introduce the concept of a variation-based distance metric, providing a quantitative framework for capturing distribution disparities between medical images originating from both the source and target domains. Subsequently, guided by this variation-based distance metric, we introduce a robust end-to-end neural network architecture carefully designed to synthesize target medical images. Our proposed method has undergone extensive experimentation across various medical image synthesis tasks, including cross-modality transformations between CT and MRI, high-dose CT synthesis from low-dose CT, and the conversion of multi-parameters in MRI, including T1, T2, T1ce, and Flair sequences. In comparative assessments against existing methods, our approach consistently outperforms them across three publicly available datasets: Gold Atlas, LDCT, and BraTS2018. Additionally, we have successfully applied our model to generate high-quality Micro-CT images in dental clinics from CBCT data, significantly enhancing diagnostic capabilities in this clinical setting. [ABSTRACT FROM AUTHOR]

High-quality agricultural multi-view stereo reconstruction technology is the key to precision and informatization in agriculture. Multi-view stereo reconstruction methods are an important part of 3D vision technology. In the multi-view stereo 3D reconstruction method based on deep learning, the effect of feature extraction directly affects the accuracy of reconstruction. Aiming at the actual problems in orchard fruit tree reconstruction, this paper designs an improved multi-view stereo structure based on the combination of remote sensing and artificial intelligence to realize the accurate reconstruction of jujube tree trunks. Firstly, an automatic key frame extraction method is proposed for the DSST target tracking algorithm to quickly recognize and extract high-quality data. Secondly, a composite U-Net feature extraction network is designed to enhance the reconstruction accuracy, while the DRE-Net feature extraction enhancement network improved by the parallel self-attention mechanism enhances the reconstruction completeness. Comparison tests show different levels of improvement on the Technical University of Denmark (DTU) dataset compared to other deep learning-based methods. Ablation test on the self-constructed dataset, the MVSNet + Co U-Net + DRE-Net_SA method proposed in this paper improves 20.4% in Accuracy, 12.8% in Completion, and 16.8% in Overall compared to the base model, which verifies the real effectiveness of the scheme. [ABSTRACT FROM AUTHOR]

In recent years, the mining industry has encountered challenges, such as a shortage of human resources, an ongoing emphasis on safety enhancements, and increased ecological preservation requirements. Autonomous mining trucks have emerged as a novel solution to effectively address these issues within open-pit mining operations. To meet the demanding conditions of open-pit mines, characterized by intense vibrations and extreme temperature variations, hybrid solid-state LiDAR has emerged as the primary choice for perception sensors. Recognizing the distinct data structure and distribution disparities between point clouds obtained through nonrepetitive scanning methods of hybrid solid-state LiDAR and traditional mechanical LiDAR, this paper proposed an innovative LiDAR 3D object detection model, PointPillars-HSL (PointPillars-Hybrid Solid-state LiDAR). This approach harmonizes the unique characteristics of open-pit mining environments and hybrid solid-state LiDAR point clouds. It optimizes the model's preprocessing methodology, augments the dimensionality of pillar features, fine-tunes the loss function, and employs transfer learning techniques to reduce the reliance on specific datasets. The result is the effective deployment of a 3D object detection algorithm customized for hybrid solid-state LiDAR within the specific operational framework of open-pit mining. This achievement has yielded a noteworthy overall vehicle recognition rate of 89.72%. [ABSTRACT FROM AUTHOR]

In order to accurately estimate the disparity of ill-posed regions, such as weak texture and occlusion regions, we propose DSPANet, a stereo matching network that incorporates a dual-stream pyramid module and a channel and spatial attention module. The dual-stream pyramid module captures numerous complementary features from different layers by utilizing multi-resolution inputs and feature extraction blocks. This approach enables the learning of local detailed features at various scales. These features at various scales are then combined to calculate the stereo matching cost. By incorporating channel and spatial attention module into the feature extraction process to obtain richer and more concise contextual information, the matching cost can be constructed more accurately, which provides powerful conditions for subsequent cost aggregation. In the cost aggregation stage, we utilize the stacked hourglass module for both encoding and decoding. Additionally, we incorporate 3D global attention upsampling during the decoding stage, which enables high-level features to provide guidance information to low-level features in a simple way. We evaluate our proposed method on the Scene Flow dataset, as well as the KITTI2012 and KITTI2015 datasets. The experimental results demonstrate that our DSPANet achieves superior performance and effectively enhances the matching results in ill-posed regions. Our code has been implemented using PyTorch and will be released after paper publication at . [ABSTRACT FROM AUTHOR]

With the emergence of the Smart City concept, the rapid advancement of urban three-dimensional (3D) reconstruction becomes imperative. While current developments in the field of 3D reconstruction have enabled the generation of 3D products such as Digital Surface Models (DSM), challenges persist in accurately reconstructing shadows, handling occlusions, and addressing low-texture areas in very-high-resolution remote sensing images. These challenges often lead to difficulties in calculating satisfactory disparity maps using existing stereo matching methods, thereby reducing the accuracy of 3D reconstruction. This issue is particularly pronounced in urban scenes, which contain numerous super high-rise and densely distributed buildings, resulting in large disparity values and occluded regions in stereo image pairs, and further leading to a large number of mismatched points in the obtained disparity map. In response to these challenges, this paper proposes a method to refine the disparity in urban scenes based on open-source GIS data. First, we register the GIS data with the epipolar-rectified images since there always exists unignorable geolocation errors between them. Specifically, buildings with different heights present different offsets in GIS data registering; thus, we perform multi-modal matching for each building and merge them into the final building mask. Subsequently, a two-layer optimization process is applied to the initial disparity map based on the building mask, encompassing both global and local optimization. Finally, we perform a post-correction on the building facades to obtain the final refined disparity map that can be employed for high-precision 3D reconstruction. Experimental results on SuperView-1, GaoFen-7, and GeoEye satellite images show that the proposed method has the ability to correct the occluded and mismatched areas in the initial disparity map generated by both hand-crafted and deep-learning stereo matching methods. The DSM generated by the refined disparity reduces the average height error from 2.2 m to 1.6 m, which demonstrates superior performance compared with other disparity refinement methods. Furthermore, the proposed method is able to improve the integrity of the target structure and present steeper building facades and complete roofs, which are conducive to subsequent 3D model generation. [ABSTRACT FROM AUTHOR]

To enhance the real-time performance and reduce CPU usage in feature-based visual SLAM, this paper introduces a lightweight tightly coupled stereo-inertial SLAM with fisheye cameras, incorporating several key innovations. First, the stereo-fisheye camera is treated as two independent monocular cameras, and the SE(3) transformation is computed between them to minimize the CPU burden during stereo feature matching and eliminate the need for camera rectification. Another important innovation is the application of maximum-a-posteriori (MAP) estimation for the inertial measurement unit (IMU), which effectively reduces inertial bias and noise in a short time frame. By optimizing the system's parameters, the constant-velocity model is replaced from the beginning, resulting in improved tracking efficiency. Furthermore, the system incorporates the inertial data in the loop closure thread. The IMU data are employed to determine the translation direction relative to world coordinates and utilized as a necessary condition for loop detection. Experimental results demonstrate that the proposed system achieves superior real-time performance and lower CPU usage compared to the majority of other SLAM systems. [ABSTRACT FROM AUTHOR]

Headland boundary identification and ranging are the key supporting technologies for the automatic driving of intelligent agricultural machinery, and they are also the basis for controlling operational behaviors such as autonomous turning and machine lifting. The complex, unstructured environments of farmland headlands render traditional image feature extraction methods less accurate and adaptable. This study utilizes deep learning and binocular vision technologies to develop a headland boundary identification and ranging system built upon the existing automatic guided tractor test platform. A headland image annotation dataset was constructed, and the MobileNetV3 network, notable for its compact model structure, was employed to achieve binary classification recognition of farmland and headland images. An improved MV3-DeeplabV3+ image segmentation network model, leveraging an attention mechanism, was constructed, achieving a high mean intersection over union (MIoU) value of 92.08% and enabling fast and accurate detection of headland boundaries. Following the detection of headland boundaries, binocular stereo vision technology was employed to measure the boundary distances. Field experiment results indicate that the system's average relative errors of distance in ranging at distances of 25 m, 20 m, and 15 m are 6.72%, 4.80%, and 4.35%, respectively. This system is capable of meeting the real-time detection requirements for headland boundaries. [ABSTRACT FROM AUTHOR]

With the continuous evolution of autonomous driving and unmanned driving systems, traditional limitations such as a limited field-of-view, poor ranging accuracy, and real-time display are becoming inadequate to satisfy the requirements of binocular stereo-perception systems. Firstly, we designed a binocular stereo-imaging-perception system with a wide-field-of-view and infrared- and visible light-dual-band fusion. Secondly we proposed a binocular stereo-perception optical imaging system with a wide field-of-view of 120.3°, which solves the small field-of-view of current binocular stereo-perception systems. Thirdly, For image aberration caused by the wide-field-of-view system design, we propose an ellipsoidal-image-aberration algorithm with a low consumption of memory resources and no loss of field-of-view. This algorithm simultaneously solves visible light and infrared images with an aberration rate of 45% and 47%, respectively. Fourthly, a multi-scale infrared- and visible light-image-fusion algorithm is used, which improves the situational-awareness capabilities of a binocular stereo-sensing system in a scene and enhances image details to improve ranging accuracy. Furthermore, this paper is based on the Taylor model-calibration binocular stereo-sensing system of internal and external parameters for limit correction; the implemented algorithms are integrated into an NVIDIA Jetson TX2 + FPGA hardware framework, enabling near-distance ranging experiments. The fusion-ranging accuracy within 20 m achieved an error of 0.02 m, outperforming both visible light- and infrared-ranging methods. It generates the fusion-ranging-image output with a minimal delay of only 22.31 ms at a frame rate of 50 Hz. [ABSTRACT FROM AUTHOR]

With the development of remote sensing satellite technology for Earth observation, remote sensing stereo images have been used for three-dimensional reconstruction in various fields, such as urban planning and construction. However, remote sensing images often contain noise, occluded regions, untextured areas, and repeated textures, which can lead to reduced accuracy in stereo matching and affect the quality of 3D reconstruction results. To reduce the impact of complex scenes in remote sensing images on stereo matching and to ensure both speed and accuracy, we propose a new end-to-end stereo matching network based on convolutional neural networks (CNNs). The proposed stereo matching network can learn features at different scales from the original images and construct cost volumes with varying scales to obtain richer scale information. Additionally, when constructing the cost volume, we introduce negative disparity to adapt to the common occurrence of both negative and non-negative disparities in remote sensing stereo image pairs. For cost aggregation, we employ a 3D convolution-based encoder–decoder structure that allows the network to adaptively aggregate information. Before feature aggregation, we also introduce an attention module to retain more valuable feature information, enhance feature representation, and obtain a higher-quality disparity map. By training on the publicly available US3D dataset, we obtain an accuracy of 1.115 pixels in end-point error (EPE) and 5.32% in the error pixel ratio (D1) on the test dataset, and the inference speed is 92 ms. Comparing our model with existing state-of-the-art models, we achieve higher accuracy, and the network is beneficial for the three-dimensional reconstruction of remote sensing images. [ABSTRACT FROM AUTHOR]

Stereo cameras can capture the rich image textures of a scene, while LiDAR can obtain accurate 3D coordinates of point clouds of a scene. They complement each other and can achieve comprehensive and accurate environment perception through data fusion. The primary step in data fusion is to establish the relative positional relationship between the stereo cameras and the 3D LiDAR, known as extrinsic calibration. Existing methods establish the camera–LiDAR relationship by constraints of the correspondence between different planes in the images and point clouds. However, these methods depend on the planes and ignore the multipath-closure constraint among the camera–LiDAR–camera sensors, resulting in poor robustness and accuracy of the extrinsic calibration. This paper proposes a trihedron as the calibration object to effectively establish various coplanar and collinear constraints between stereo cameras and 3D LiDAR. With the various constraints, the multipath-closure constraint between the three sensors is further formulated for the extrinsic calibration. Firstly, the coplanar and collinear constraints between the camera–LiDAR–camera are built using the trihedron calibration object. Then, robust and accurate coplanar constraint information is extracted through iterative maximum a posteriori (MAP) estimation. Finally, a multipath-closure extrinsic calibration method for multi-sensor systems is developed with structurally mutual validation between the cameras and the LiDAR. Extensive experiments are conducted on simulation data with different noise levels and a large amount of real data to validate the accuracy and robustness of the proposed calibration algorithm. [ABSTRACT FROM AUTHOR]

Deep neural networks significantly enhance the accuracy of the stereo‐based disparity estimation. Some current methods suffer from inefficient use of the global context information, which will lead to the loss of structural details in ill‐posed areas. To this end, a novel stereo network GAMNet is designed, composed of three core components (GDA, MPF, DCA) for estimating the depth prediction in challenging real‐world environments. First, a lightweight attention module is presented, integrating the global semantic cues for every feature position across the channel and spatial dimensions. Next, the MPF module is constructed to fuse the diverse semantic and contextual information from different levels of the feature pyramid. Finally, cost volume is aggregated with a stacked encoder‐decoder composed of the DCA module and 3D convolutions, filtering the transmission of matching clues and capturing the rich global contexts. Substantial experiments conducted on KITTI 2012, KITTI 2015, SceneFlow, and Middlebury‐v3 datasets manifest that GAMNet surpasses preceding methods with contour‐preserving disparity predictions. In addition, the first 3D scene reconstruction linear evaluation strategy on spatial grasping points for the end‐to‐end stereo networks in an unsupervised mode is proposed, and it is deployed on the designed robot vision‐guided system. In application experiments, the method can produce densely high‐precision 3D reconstructions to implement the grasping task in complex real‐world scenes, and achieve excellent robust performance with competitive inference efficiency. [ABSTRACT FROM AUTHOR]

Future wireless power transmission will cover power levels up to kilowatts or more and transmission distances up to the scale of kilometers. With its narrow beam divergence angle, optical wireless power transmission (OWPT) is a promising candidate for such system implementations. In the operation of OWPT, it is necessary to estimate the position, direction (azimuth, elevation), and attitude of the target photovoltaic device before the power supply. The authors have proposed the detection of targets using differential absorption imaging and positioning with a combination of stereo imagery. In the positioning by stereo imagery, a condition regarding the consistency of the left and right images can be defined. This corresponds to the certain value of the exposure time of the image sensor, and this depends on the target's attitude angle. In this paper, we discuss target attitude estimation using this minimum exposure time at which the integrity measure converges. A physical model was derived under general conditions of target position and experimental configuration. Target attitudes were estimated within an error range of 10 to 15 degrees in approximately 60 degrees range. On the other hand, there is an attitude estimation method based on the apparent size of the target. When using this method to estimate the attitude angle, errors are significantly large for specular and diffuse mixed targets like the PV. The method proposed in this paper is a robust attitude estimation method for the photovoltaic device in OWPT. [ABSTRACT FROM AUTHOR]

RGB-D cameras provide depth and color information and are widely used in 3D reconstruction and computer vision. In the majority of existing RGB-D cameras, a considerable portion of depth values is often lost due to severe occlusion or limited camera coverage, thereby adversely impacting the precise localization and three-dimensional reconstruction of objects. In this paper, to address the issue of poor-quality in-depth images captured by RGB-D cameras, a depth image hole repair algorithm based on non-local means is proposed first, leveraging the structural similarities between grayscale and depth images. Second, while considering the cumbersome parameter tuning associated with the non-local means hole repair method for determining the size of structural blocks for depth image hole repair, an intelligent block factor is introduced, which automatically determines the optimal search and repair block sizes for various hole sizes, resulting in the development of an adaptive block-based non-local means algorithm for repairing depth image holes. Furthermore, the proposed algorithm's performance are evaluated using both the Middlebury stereo matching dataset and a self-constructed RGB-D dataset, with performance assessment being carried out by comparing the algorithm against other methods using five metrics: RMSE, SSIM, PSNR, DE, and ALME. Finally, experimental results unequivocally demonstrate the innovative resolution of the parameter tuning complexity inherent in-depth image hole repair, effectively filling the holes, suppressing noise within depth images, enhancing image quality, and achieving elevated precision and accuracy, as affirmed by the attained results. [ABSTRACT FROM AUTHOR]

Forest scene 3D reconstruction serves as the fundamental basis for crucial applications such as forest resource inventory, forestry 3D visualization, and the perceptual capabilities of intelligent forestry robots in operational environments. However, traditional 3D reconstruction methods like LiDAR present challenges primarily because of their lack of portability. Additionally, they encounter complexities related to feature point extraction and matching within multi-view stereo vision sensors. In this research, we propose a new method that not only reconstructs the forest environment but also performs a more detailed tree reconstruction in the scene using conditional generative adversarial networks (CGANs) based on a single RGB image. Firstly, we introduced a depth estimation network based on a CGAN. This network aims to reconstruct forest scenes from images and has demonstrated remarkable performance in accurately reconstructing intricate outdoor environments. Subsequently, we designed a new tree silhouette depth map to represent the tree's shape as derived from the tree prediction network. This network aims to accomplish a detailed 3D reconstruction of individual trees masked by instance segmentation. Our approach underwent validation using the Cityscapes and Make3D outdoor datasets and exhibited exceptional performance compared with state-of-the-art methods, such as GCNDepth. It achieved a relative error as low as 8% (with an absolute error of 1.76 cm) in estimating diameter at breast height (DBH). Remarkably, our method outperforms existing approaches for single-image reconstruction. It stands as a cost-effective and user-friendly alternative to conventional forest survey methods like LiDAR and SFM techniques. The significance of our method lies in its contribution to technical support, enabling the efficient and detailed utilization of 3D forest scene reconstruction for various applications. [ABSTRACT FROM AUTHOR]

This paper proposes and experimentally evaluates, a reconfigurable intelligent surface (RIS) control system for multiple users using stereo camera images. The aim is to enable fast and fair received signal strength (RSS) beamforming for multiple users. In the proposed system, a stereo camera is mounted on top of the RIS. Human bodies are recognized from the obtained images. Subsequently, the beam directions and the phase of the RIS are calculated for the estimated positions. The stereo camera‐based scheme enables fast beamforming because channel state information estimation is omitted. Moreover, identical RSS beamforming is enabled by calculating the weighted sums of the RIS phases for multiple users utilizing distance and angle information of users from the RIS obtained using the stereo camera. Experiments are conducted in an anechoic chamber using RIS operating at 28 GHz. The efficacy of the RIS control system using stereo camera images and its ability to increase the RSS fairly for multiple users is validated via simulations and experiments. [ABSTRACT FROM AUTHOR]