Search

Automatic fish length estimation is essential for modern aquaculture. Occlusion and body bended make accurate fish length estimation challenging in intensive aquaculture environments. Aiming at these issues, this study proposes a fish length estimation scheme based on cross-modal feature fusion Mask R-CNN (CMFF Mask R-CNN) and point cloud normalization segmentation transformation. To eliminate fish which are incomplete in binocular images due to occlusion and extract masks of fish which are complete in binocular images, a cross-modal feature fusion module is designed and embedded into Mask R-CNN to aggregate boundary features of fish from RGB and disparity into unified feature maps. The feature maps help remove incomplete fish and improve the accuracy of complete fish mask boundary. A fish length estimation algorithm based on point cloud normalization segmentation transformation is designed to reduce the length estimation error caused by bending. After plane and ellipse fitting transformation, the fish contour point cloud is then transformed into a unified space for K-means clustering segmentation. The sum of each segment is the fish length. Experimental results show that the mean relative error of the salmon length estimation is less than 5%. Moreover, the generalizability experiments show that the average relative error of the proposed method for the other freshwater fish species is less than 5%. This indicates that the proposed method for fish length estimation could be applied in aquaculture. [ABSTRACT FROM AUTHOR]

Biological specimens exhibit significant variations in size and shape, challenging autonomous robotic manipulation. We focus on the mouse skull window creation task to illustrate these challenges. The study introduces a microscopic stereo camera system (MSCS) enhanced by the linear model for depth perception. Alongside this, a precise registration scheme is developed for the partially exposed mouse cranial surface, employing a CNN-based constrained and colorized registration strategy. These methods are integrated with the MSCS for robotic micromanipulation tasks. The MSCS demonstrated a high precision of $ 0.10\,{\rm mm} \pm 0.02\,{\rm mm} $ 0.10mm±0.02mm measured in a step height experiment and real-time performance of 30 FPS in 3D reconstruction. The registration scheme proved its precision, with a translational error of $ 1.13\,{\rm mm} \pm 0.31\,{\rm mm} $ 1.13mm±0.31mm and a rotational error of $ 3.38^\circ \pm 0.89^\circ $ 3.38∘±0.89∘ tested on 105 continuous frames with an average speed of 1.60 FPS. This study presents the application of a MSCS and a novel registration scheme in enhancing the precision and accuracy of robotic micromanipulation in scientific and surgical settings. The innovations presented here offer automation methodology in handling the challenges of microscopic manipulation, paving the way for more accurate, efficient, and less invasive procedures in various fields of microsurgery and scientific research. [ABSTRACT FROM AUTHOR]

In 3D reconstruction the stereo technique is one of the most used, generating point clouds of acceptable quality. One way to improve its quality is by fusing it with active systems such as lasers. For this fusion, a registration process can be used. It is important to evaluate the quality of the reconstruction in terms of spatial structure accuracy and visual appearance. A method of evaluating the quality of an active 3D stereo reconstruction is proposed, which takes a stereo camera image and compares it with a reprojected image of the registered cloud. The typical way of comparing two images calculates the point-to-point error, but does not take into account other aspects. Several quality metrics are studied to select the one that contributes the most in a given 3D reconstruction context. To improve the computational cost, a process of partitioning the images by regions is performed. A Quadtree partition was chosen with a homogeneity criterion that is selected among several criteria. The experiments and results for selecting the quality metric and the homogeneity criterion for the Quadtree partition are presented. [ABSTRACT FROM AUTHOR]

When light passes through air–glass and glass–water interfaces, refraction occurs, which affects the accuracy of stereo vision three-dimensional measurements of underwater targets. To eliminate the impact of refraction, we developed a refractive stereo vision measurement model based on light propagation paths, utilizing the normalized coordinate of the underwater target. This model is rigorous in theory, and easy to understand and apply. Additionally, we established an underwater simulation imaging model based on the principle that light travels the shortest time between two points. Simulation experiments conducted using this imaging model verified the performance of the underwater stereo vision measurement model. The results demonstrate that the accuracy achieved by the new measurement model is comparable to that of the stereo vision measurement model in the air and significantly higher than that of the existing refractive measurement model. This is because the light rays from the camera's optical center to the refraction point at the air–glass interface do not always intersect. The experiments also indicate that the deviation in the refractive index of water lead to corresponding systematic errors in the measurement results. Therefore, in real underwater measurements, it is crucial to carefully calibrate the refractive index of water and maintain the validity of the calibration results. [ABSTRACT FROM AUTHOR]

A rigid-flexible dual-arm space robot offers promising potential for on-orbit operation due to complementary strengths of its rigid and flexible manipulators. The rigid manipulator has the advantage of large payload capacity and high motion accuracy. The segmented linkage flexible manipulator is ideal for maneuvering in narrow, unstructured environments like internal satellite inspections and maintenance, due to its flexibility and slender body. However, there are inevitable tracking errors due to the multiple cable-driven mechanisms in the flexible manipulator. It's difficult to get the configuration and end pose of flexible manipulator without adding external sensors. To address these issues, this paper presents a method for configuration reconstruction and all-joint synchronous measurement of the flexible manipulator. The flexible manipulator is captured by the stereovision system mounted on the rigid manipulator. The links' equivalent center points are recognized by central moment method and edge line extraction method based on the natural characteristics. Joint - to - link kinematic model of flexible manipulator is established to measure joint angles and reconstruct configuration. Several simulations and experiments are carried out to validate the proposed method. The experimental results showed links' average measurement position errors were less than 13 mm and joint angles reconstructed using the proposed method had an error of approximately 0.35 °. These results demonstrate the accuracy and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

The industrialization, high-density, and greener aquaculture requires a more precise and intelligent aquaculture management. Phenotypic and behavioral information of fish, which can reflect fish growth and welfare status, play a crucial role in aquaculture management. Stereo vision technology, which simulates parallax perception of the human eye, can obtain the three-dimensional phenotypic characteristics and movement trajectories of fish through different types of sensors. It can overcome the limitations in dealing with fish deformation, frequent occlusions and understanding three-dimension scenes compared to the traditional two-dimensional computer vision techniques. With the deep learning development and application in aquaculture, stereo vision has become a super computer vision technology that can provide more precise and interpretable information for intelligent aquaculture management, such as size estimation, counting and behavioral analysis of fish. Hence, it is very beneficial for researchers, managers, and entrepreneurs to possess a thorough comprehension about the fast-developing stereo vision technology for modern aquaculture. This study provides a critical review of relevant topics, including the four-layer application structure of stereo vision technology in aquaculture, various deep learning-based technologies used, and specific application scenarios. The review contributes to research development by identifying the current challenges and provide valuable suggestions for future research directions. This review can serve as a useful resource for developing future studies and applications of stereo vision technology in smart aquaculture, focusing on phenotype feature extraction and behavioral analysis of fish. [ABSTRACT FROM AUTHOR]

Fusing data from many sources helps to achieve improved analysis and results. In this work, we present a new algorithm to fuse data from multiple cameras with data from multiple lidars. This algorithm was developed to increase the sensitivity and specificity of autonomous vehicle perception systems, where the most accurate sensors measuring the vehicle's surroundings are cameras and lidar devices. Perception systems based on data from one type of sensor do not use complete information and have lower quality. The camera provides two-dimensional images; lidar produces three-dimensional point clouds. We developed a method for matching pixels on a pair of stereoscopic images using dynamic programming inspired by an algorithm to match sequences of amino acids used in bioinformatics. We improve the quality of the basic algorithm using additional data from edge detectors. Furthermore, we also improve the algorithm performance by reducing the size of matched pixels determined by available car speeds. We perform point cloud densification in the final step of our method, fusing lidar output data with stereo vision output. We implemented our algorithm in C++ with Python API, and we provided the open-source library named Stereo PCD. This library very efficiently fuses data from multiple cameras and multiple lidars. In the article, we present the results of our approach to benchmark databases in terms of quality and performance. We compare our algorithm with other popular methods. [ABSTRACT FROM AUTHOR]

There has been substantial research on multi-medium visual measurement in fields such as underwater three-dimensional reconstruction and underwater structure monitoring. Addressing the issue where traditional air-based visual-measurement models fail due to refraction when light passes through different media, numerous studies have established refraction-imaging models based on the actual geometry of light refraction to compensate for the effects of refraction on cross-media imaging. However, the calibration of refraction parameters inevitably contains errors, leading to deviations in these parameters. To analyze the impact of refraction-parameter deviations on measurements in underwater structure visual navigation, this paper develops a dual-media stereo-vision measurement simulation model and conducts comprehensive simulation experiments. The results indicate that to achieve high-precision underwater-measurement outcomes, the calibration method for refraction parameters, the distribution of the targets in the field of view, and the distance of the target from the camera must all be meticulously designed. These findings provide guidance for the construction of underwater stereo-vision measurement systems, the calibration of refraction parameters, underwater experiments, and practical applications. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this paper, an improved local based stereo vision disparity map (SVDM) algorithm is proposed. The proposed local based SVDM algorithm include four stages and they are matching cost computation, cost aggregation disparity optimization and disparity refinement. The matching cost computation started by combining pixel to pixel matching techniques, which are absolute difference (AD) and gradient matching (GM) in producing the initial disparity map. Next, the cost aggregation uses minimum spanning tree (MST) segmentation, which equipped with edge preserving properties and noise filtering. Then, disparity optimization uses local approach with winner-take-all (WTA) technique. At the final stage, disparity refinement uses bilateral filter (BF) with weighted median (WM), which can improve the disparity map through noise removing and edges preserving. Then, the research continues to optimize the proposed local based SVDM algorithm through parameters optimization in obtaining the final disparity map. Here, multiple parameters from the proposed SVDM algorithm are manipulated and they are constant values for GM and several constant parameters in BF. By selecting the optimum parameter values, the performance of the proposed SVDM algorithm increased, especially robustness towards the horizontal streaks. [ABSTRACT FROM AUTHOR]

The dual-spectrum triocular camera system composed of a binocular visible light camera and mid-infrared camera is used for high-precision thermal fault detection and thermal field reconstruction of industrial equipment. The realization of its function depends on the high-precision camera parameter calibration. The difficulty lies in how to realize the infrared camera calibration quickly and improve the parameter accuracy of multi-lens camera. In this paper, according to the characteristics of the dual-spectral triocular camera system, the theoretical model is constructed, and the circular asymmetric calibration plate under infrared supplementary light is selected as the calibration object through experiments. A method for combining infrared image adaptive histogram enhancement and binarization processing based on the Sauvola algorithm is proposed to effectively calibrate the infrared camera. A global parameter optimization method based on traditional passive vision binocular stereo calibration is proposed. The optimization of experimental parameters finds the reprojection error value is 0.16, which meets the demand of high-precision calibration and solves the problem of difficult-to-calibrate weak image texture of infrared camera and the calibration accuracy of the camera under different imaging capabilities. [ABSTRACT FROM AUTHOR]

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this paper, we present a multi-task model that predicts disparities and confidence levels in deep stereo matching simultaneously. We do this by combining its successful model for each separate task and obtaining a multi-task model that can be trained with a proposed loss function. We show the advantages of this model compared to training and predicting disparity and confidence sequentially. This method enables an improvement of 15% to 30% in the area under the curve (AUC) metric when trained in parallel rather than sequentially. In addition, the effect of weighting the components in the loss function on the stereo and confidence performance is investigated. By improving the confidence estimate, the practicality of stereo estimators for creating distance images is increased. [ABSTRACT FROM AUTHOR]

The present work aims to develop an object tracking controller for the Stewart platform using a computer vision-assisted machine learning-based approach. This research is divided into two modules. The first module focuses on the design of a motion controller for the Physik Instrumente (PI)-based Stewart platform. In contrast, the second module deals with the development of a machine-learning-based spatial object tracking algorithm by collecting information from the Zed 2 stereo vision system. Presently, simple feed-forward neural networks (NN) are used to predict the orientation of the top table of the platform. While training, the x, y, and z coordinates of the three-dimensional (3D) object, extracted from images, are used as the input to the NN. In contrast, the orientation information of the platform (that is, rotation about the x, y, and z-axes) is considered as the output from the network. The orientation information obtained from the network is fed to the inverse kinematics-based motion controller (module 1) to move the platform while tracking the object. After training, the optimised NN is used to track the continuously moving 3D object. The experimental results show that the developed NN-based controller has successfully tracked the moving spatial object with reasonably good accuracy. [ABSTRACT FROM AUTHOR]

In practical applications, many robots equipped with embedded devices have limited computing capabilities. These limitations often hinder the performance of existing dynamic SLAM algorithms, especially when faced with occlusions or processor constraints. Such challenges lead to subpar positioning accuracy and efficiency. This paper introduces a novel lightweight dynamic SLAM algorithm designed primarily to mitigate the interference caused by moving object occlusions. Our proposed approach combines a deep learning object detection algorithm with a Kalman filter. This combination offers prior information about dynamic objects for each SLAM algorithm frame. Leveraging geometric techniques like RANSAC and the epipolar constraint, our method filters out dynamic feature points, focuses on static feature points for pose determination, and enhances the SLAM algorithm's robustness in dynamic environments. We conducted experimental validations on the TUM public dataset, which demonstrated that our approach elevates positioning accuracy by approximately 54% and boosts the running speed by 75.47% in dynamic scenes. [ABSTRACT FROM AUTHOR]

In recent years, the technological landscape has undergone a profound metamorphosis catalyzed by the widespread integration of drones across diverse sectors. Essential to the drone manufacturing process is comprehensive testing, typically conducted in controlled laboratory settings to uphold safety and privacy standards. However, a formidable challenge emerges due to the inherent limitations of GPS signals within indoor environments, posing a threat to the accuracy of drone positioning. This limitation not only jeopardizes testing validity but also introduces instability and inaccuracies, compromising the assessment of drone performance. Given the pivotal role of precise GPS-derived data in drone autopilots, addressing this indoor-based GPS constraint is imperative to ensure the reliability and resilience of unmanned aerial vehicles (UAVs). This paper delves into the implementation of an Indoor Positioning System (IPS) leveraging computer vision. The proposed system endeavors to detect and localize UAVs within indoor environments through an enhanced vision-based triangulation approach. A comparative analysis with alternative positioning methodologies is undertaken to ascertain the efficacy of the proposed system. The results obtained showcase the efficiency and precision of the designed system in detecting and localizing various types of UAVs, underscoring its potential to advance the field of indoor drone navigation and testing. [ABSTRACT FROM AUTHOR]

High speed photography technique is potentially the most effective way to measure the motion parameter of warhead fragment benefiting from its advantages of high accuracy, high resolution and high efficiency. However, it faces challenge in dense objects tracking and 3D trajectories reconstruction due to the characteristics of small size and dense distribution of fragment swarm. To address these challenges, this work presents a warhead fragments motion trajectories tracking and spatio-temporal distribution reconstruction method based on high-speed stereo photography. Firstly, background difference algorithm is utilized to extract the center and area of each fragment in the image sequence. Subsequently, a multi-object tracking (MOT) algorithm using Kalman filtering and Hungarian optimal assignment is developed to realize real-time and robust trajectories tracking of fragment swarm. To reconstruct 3D motion trajectories, a global stereo trajectories matching strategy is presented, which takes advantages of epipolar constraint and continuity constraint to correctly retrieve stereo correspondence followed by 3D trajectories refinement using polynomial fitting. Finally, the simulation and experimental results demonstrate that the proposed method can accurately track the motion trajectories and reconstruct the spatio-temporal distribution of 1.0×10³ fragments in a field of view (FOV) of 3.2 m×2.5 m, and the accuracy of the velocity estimation can achieve 98.6%. [ABSTRACT FROM AUTHOR]

Assessing the condition of roads is crucial to the maintenance and rehabilitation process as a country's progress is closely linked to its transport infrastructure. Therefore, it is essential to have well-maintained roads and to be able to control and monitor them properly. Technological advancements have transformed the way pavement inspections are carried out. This study presents an innovative approach that combines stereo cameras and a GPS module for efficient and accurate data collection. This integration of low-cost technologies provides a detailed three-dimensional view of pavements, complemented by accurate geospatial information. The experimental results showed that the 3D images of pavement damage had a relative volume measurement error of 0.80 %. Unlike traditional systems such as LIDAR and ground-penetrating radar, which involve more expensive technologies, the proposed method offers a cost-effective solution. This methodology not only simplifies the inspection process but also improves the planning and execution of road maintenance and repair activities. Its low cost makes it a viable option for various projects and applications in road infrastructure.

Corn is a global crop that requires the breeding of superior varieties. A crucial aspect of the breeding process is the accurate extraction of phenotypic parameters from corn plants. The existing challenges in phenotypic parameter extraction include low precision, excessive manual involvement, prolonged processing time, and equipment complexity. This study addresses these challenges by opting for binocular cameras as the data acquisition equipment. The proposed stereo corn phenotype extraction algorithm (SCPE) leverages binocular images for phenotypic parameter extraction. The SCPE consists of two modules: the YOLOv7-SlimPose model and the phenotypic parameter extraction module. The YOLOv7-SlimPose model was developed by optimizing the neck component, refining the loss function, and pruning the model based on YOLOv7-Pose. This model can better detect bounding boxes and keypoints with fewer parameters. The phenotypic parameter extraction module can construct the skeleton of the corn plant and extract phenotypic parameters based on the coordinates of the keypoints detected. The results showed the effectiveness of the approach, with the YOLOv7-SlimPose model achieving a keypoint mean average precision (mAP) of 96.8% with 65.1 million parameters and a speed of 0.09 s/item. The phenotypic parameter extraction module processed one corn plant in approximately 0.2 s, resulting in a total time cost of 0.38 s for the entire SCPE algorithm to construct the skeleton and extract the phenotypic parameters. The SCPE algorithm is economical and effective for extracting phenotypic parameters from corn plants, and the skeleton of corn plants can be constructed to evaluate the growth of corn as a reference. This proposal can also serve as a valuable reference for similar functions in other crops such as sorghum, rice, and wheat. [ABSTRACT FROM AUTHOR]

Simple Summary: Pest control plays an important role in crop production. The cotton leafworm, Spodoptera litura, is well recognized as a pest that causes severe damage to a wide variety of crops. Because S. litura is nocturnal, it is challenging to control this species effectively. Recently, laser zapping has gained attention as a clean technology to control pest insects. It is important to precisely identify and predict the flight trajectories of free-flying moths under low-light conditions for better sighting during laser zapping. In this study, we developed an automatic detection pipeline based on point cloud time series data from stereoscopic images. Three-dimensional point cloud data were extracted from disparity images recorded under infrared and low-light conditions. We computed the size of the outline box and the directional angle of the 3D point cloud time series to remove noisy point clouds. We visually inspected the flight trajectories and found that the size and direction of the outline box were good indicators of the noisy data. Finally, we obtained 68 flight trajectories, and the average flight speed of free-flying S. litura was found to be 1.81 m/s. Pest control is crucial in crop production; however, the use of chemical pesticides, the primary method of pest control, poses environmental issues and leads to insecticide resistance in pests. To overcome these issues, laser zapping has been studied as a clean pest control technology against the nocturnal cotton leafworm, Spodoptera litura, which has high fecundity and causes severe damage to various crops. For better sighting during laser zapping, it is important to measure the coordinates and speed of moths under low-light conditions. To achieve this, we developed an automatic detection pipeline based on point cloud time series data from stereoscopic images. We obtained 3D point cloud data from disparity images recorded under infrared and low-light conditions. To identify S. litura, we removed noise from the data using multiple filters and a support vector machine. We then computed the size of the outline box and directional angle of the 3D point cloud time series to determine the noisy point clouds. We visually inspected the flight trajectories and found that the size of the outline box and the movement direction were good indicators of noisy data. After removing noisy data, we obtained 68 flight trajectories, and the average flight speed of free-flying S. litura was 1.81 m/s. [ABSTRACT FROM AUTHOR]

In this paper, the influence of parallax and image shape type on stereo vision perception is quantitatively explored by using RDS stimuli based on AR equipment through subjective scoring. Results show that an angular parallax amplitude between 0.6 ° and 1.2 ° can bring the best stereo sense. When amplitude ranged between 0 ° and 1.2°, the best visual comfort and the fastest image fusion speed could be achieved. [ABSTRACT FROM AUTHOR]

An artificial compound eye consists of multiple apertures that allow for a large field of view (FOV) while maintaining a small size. Each aperture captures a sub-image, and multiple sub-images are needed to reconstruct the full FOV. The reconstruction process is depth-related due to the parallax between adjacent apertures. This paper presents an all-in-focus 3D reconstruction method for a specific type of artificial compound eye called the electronic cluster eye (eCley). The proposed method uses edge matching to address the edge blur and large textureless areas existing in the sub-images. First, edges are extracted from each sub-image, and then a matching operator is applied to match the edges based on their shape context and intensity. This produces a sparse matching result that is then propagated to the whole image. Next, a depth consistency check and refinement method is performed to refine the depth of all sub-images. Finally, the sub-images and depth maps are merged to produce the final all-in-focus image and depth map. The experimental results and comparative analysis demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Machine vision research began with a single-camera system, but these systems had various limitations from having just one point-of-view of the environment and no depth information, therefore stereo cameras were invented. This paper proposes a hybrid method of a stereo matching algorithm with the goal of generating an accurate disparity map critical for applications such as 3D surface reconstruction and robot navigation to name a few. Convolutional neural network (CNN) is utilised to generate the matching cost, which is then input into cost aggregation to increase accuracy with the help of a bilateral filter (BF). Winner-take-all (WTA) is used to generate the preliminary disparity map. An edge-preserving filter (EPF) is applied to that output based on a transform that defines an isometry between curves on the 2D image manifold in 5D and the real line to eliminate these artefacts. The transform warps the input signal adaptively to allow linear 1D filtering. Due to the filter's resistance to high contrast and brightness, it is effective in refining and removing noise from the output image. Based on experimental research employing a Middlebury standard validation benchmark, this approach gives high accuracy with an average non-occluded error of 6.71% comparable to other published methods. [ABSTRACT FROM AUTHOR]

This paper proposes a new approach to reconstructing the three-dimensional shapes of target objects. The method is based on object detection using a convolutional neural network, followed by fusing the recognition results with the depth map from a stereo camera. The prospects of applying this new approach are discussed based on the obtained results. [ABSTRACT FROM AUTHOR]

Obstacle detection and avoidance are the major issues in autonomous navigation for partially known or unknown environments. With the proliferation of space debris, researchers are actively investigating debris removal to facilitate future space operations. This calls for the development of autonomous navigation techniques for space missions. Free-space object detection is a crucial task in intelligent systems, particularly for path planning. In this study, we propose a stereo vision-based intelligent system for space object detection, using two vertically aligned omnidirectional stereo cameras separated by 10 cm. Firstly, a single-shot multibox detector (SSD) based on deep learning is employed to identify the objects present in the image. Then, the triangulation method is used to determine the distance between the object and the system. The proposed system can provide object depth information up to a maximum range of 1.1 km in a space environment. • Problem Statement: Addresses the challenges of autonomous space navigation and space debris proliferation obstructing future space missions. • Proposed Solution: Proposes a novel long-range stereo vision for spaceborne systems, with applications in navigation and space debris mitigation. • Methodology: Uses SSD, rooted in deep learning, combined with triangulation for object identification and distance determination. • System Capabilities: Provides object depth up to 1.1 km in space, vital for navigation and debris removal. • Technical Requirements and Challenges : Emphasizes space-grade optics, real-time algorithm, and handling environmental factors like vibration and radiation. [ABSTRACT FROM AUTHOR]

Stereo vision is a key technology for 3D scene reconstruction from image pairs. Most approaches process perspective images from commodity cameras. These images, however, have a very limited field of view and only picture a small portion of the scene. In contrast, omnidirectional images, also known as fisheye images, exhibit a much larger field of view and allow a full 3D scene reconstruction with a small amount of cameras if placed carefully. However, omnidirectional images are strongly distorted which make the 3D reconstruction much more sophisticated. Nowadays, a lot of research is conducted on CNNs for omnidirectional stereo vision. Nevertheless, a significant gap between estimation accuracy and throughput can be observed in the literature. This work aims to bridge this gap by introducing a novel set of transformations, namely OmniGlasses. These are incorporated into the architecture of a fast network, i.e., AnyNet, originally designed for scene reconstruction on perspective images. Our network, Omni-AnyNet, produces accurate omnidirectional distance maps with a mean absolute error of around 13 cm at 48.4 fps and is therefore real-time capable. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Ordnance Equipment Engineering is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This research focuses on the performance evaluation of mobile stereonet for real-time navigation in mobile robots. The use of mobile stereonet for depth estimation and path planning has become increasingly important in the field of robotics. The research is conducting by implementing the mobile stereonet algorithm for real time navigation on a Raspberry Pi 4 and integrating it with a stereo camera to capture live video frames. The objective of this research is to evaluate the effectiveness of mobile stereonet in navigating mobile robots in different real time environments. The study will include the development of a mobile stereonet-based depth estimation and path planning system and its evaluation through simulations and experiments. The performance of the system will be evaluated based on various metrics, such as accuracy, efficiency, and robustness. The results of this study will provide insights into the potential applications of mobile stereonet in the field of robotics and contribute to the development of better navigation system for mobile robots using stereo vision technology. Implemented real time map generation using mobile stereonet is available on Github. [ABSTRACT FROM AUTHOR]

The eye tracking‐based autostereoscopic 3D display provides a large viewing angle, low crosstalk, and large depth perception 3D image experience without using any 3D eyeglasses. However, accurate and fast eye position detection and tracking are still challenging. This study presents an accurate and fast eye positioning algorithm by reducing the computational load of stereo matching and stereo rectification in stereo vision. The accuracy of distance measurement is up to 98.46%. The processing time needed to calculate the distance between the target and the cameras is less than 0.012s. [ABSTRACT FROM AUTHOR]

Stereo matching is an important method in computer vision for simulating human binocular vision to acquire spatial distance information. Implementing high-precision and real-time stereo-matching algorithms on hardware platforms with limited resources remains a significant challenge. Although the semi-global stereo-matching algorithm strikes a good balance between obtaining accuracy in the disparity map and computational complexity, it uses a fixed window for matching, resulting in lower matching accuracy in image regions with depth discontinuities and weak textures. To address the shortcomings of existing semi-global stereo-matching algorithms, an adaptive window semi-global stereo-matching algorithm is proposed, along with post-processing disparity optimization through left–right consistency check and median filtering. On test images provided by the Middlebury dataset, the average matching accuracy improved by 5.07% compared to traditional-matching algorithms. This algorithm is implemented on a Zynq UltraScale + chip, utilising 42,072 LUTs, 66,532 registers, and 101 BRAMs for the entire stereo-matching architecture. For images with a resolution of 1280 × 720 and 64 disparity levels, the final-processing speed can reach 54.24 fps. [ABSTRACT FROM AUTHOR]

This paper introduces a stereoscopic image and depth dataset created using a deep learning model. It addresses the challenge of obtaining accurate and annotated stereo image pairs with irregular boundaries for deep learning model training. Stereoscopic image and depth dataset provides a unique resource for training deep learning models to handle irregular boundary stereoscopic images, which are valuable for real-world scenarios with complex shapes or occlusions. The dataset is created using monocular depth estimation, a state-of-the-art depth estimation model, and it can be used in applications like rectifying images, estimating depth, detecting objects, and autonomous driving. Overall, this paper presents a novel dataset that demonstrates its effectiveness and potential for advancing stereo vision and developing deep learning models for computer vision applications. [ABSTRACT FROM AUTHOR]

Abstract Bird flocking is a paradigmatic case of self‐organised collective behaviours in biology. Stereo camera systems are employed to observe flocks of starlings, jackdaws, and chimney swifts, mainly on a spot‐fixed basis. A portable non‐fixed stereo vision‐based flocking observation system, namely FlockSeer, is developed by the authors for observing more species of bird flocks within field scenarios. The portable flocking observer, FlockSeer, responds to the challenges in extrinsic calibration, camera synchronisation and field movability compared to existing spot‐fixed observing systems. A measurement and sensor fusion approach is utilised for rapid calibration, and a light‐based synchronisation approach is used to simplify hardware configuration. FlockSeer has been implemented and tested across six cities in three provinces and has accomplished diverse flock‐tracking tasks, accumulating behavioural data of four species, including egrets, with up to 300 resolvable trajectories. The authors reconstructed the trajectories of a flock of egrets under disturbed conditions to verify the practicality and reliability. In addition, we analysed the accuracy of identifying nearest neighbours, and then examined the similarity between the trajectories and the Couzin model. Experimental results demonstrate that the developed flocking observing system is highly portable, more convenient and swift to deploy in wetland‐like or coast‐like fields. Its observation process is reliable and practical and can effectively support the study of understanding and modelling of bird flocking behaviours.

A considerable number of vehicular accidents occur in low-millage zones like school streets, neighborhoods, and parking lots, among others. Therefore, the proposed work aims to provide a novel ADAS system to warn about dangerous scenarios by analyzing the driver's attention and the corresponding distances between the vehicle and the detected object on the road. This approach is made possible by concurrent Head Pose Estimation (HPE) and Object/Pedestrian Detection. Both approaches have shown independently their viable application in the automotive industry to decrease the number of vehicle collisions. The proposed system takes advantage of stereo vision characteristics for HPE by enabling the computation of the Euler Angles with a low average error for classifying the driver's attention on the road using neural networks. For Object Detection, stereo vision is used to detect the distance between the vehicle and the approaching object; this is made with a state-of-the-art algorithm known as YOLO-R and a fast template matching technique known as SoRA that provides lower processing times. The result is an ADAS system designed to ensure adequate braking time, considering the driver's attention on the road and the distances to objects.

Stereo vision technology, as a significant branch of computer vision, has been widely applied in fields such as robot navigation, autonomous driving, and 3D reconstruction. Achieving embodied intelligence through edge hardware platforms remains challenging in balancing power consumption, real-time performance, and accuracy. Although the semi-global stereo matching algorithm has proven effective in balancing disparity map accuracy and computational complexity, its matching accuracy is often limited by weak textures, disparity discontinuities, lighting variations, and noise. To address the limitations of existing semi-global stereo matching algorithms and the power constraints of hardware platforms, this paper proposes a stereo matching algorithm based on FPGA that integrates the Sum of Absolute Differences (SAD) with adaptive Census transform. The goal is to enhance image and edge information and design a compact and efficient stereo vision hardware accelerator. During the hardware implementation phase, a pixel-level pipelined parallel matching cost computation structure is proposed. This structure significantly reduces data buffer requirements through multi-step parallel computation. Additionally, a two-stage four-layer parallel pipelined semi-global cost aggregation architecture is adopted, which effectively balances hardware resource utilization while maintaining accuracy. Evaluations of the Middlebury dataset show that compared to traditional SAD and Census algorithms, the proposed algorithm improves matching accuracy by 15.67% and 15.1%, respectively. On the Xilinx Zynq-7 platform, for images with resolutions of $1280\times 720$ and $640\times 480$ , the processing speeds reach 54.24fps and 81.34fps, respectively.

The recent advances in deep learning techniques enable 2D Multi-object tracking (MOT) to achieve remarkable performance over traditional methods. However, most 2D MOT algorithms primarily utilize only single-camera view. Therefore, they are prone to frequent tracking losses and track-ID switching under conditions due to limited viewpoints and occluded objects. To alleviate this problem, we propose a stereo-camera-based collaborated multi-object tracking (CollabMOT) method that performs online and dynamic association of multiple tracklets from baseline MOT algorithms in overlapping views of stereo cameras. CollabMOT utilizes appearance similarity to generate global tracking IDs that unify the same tracklets between viewpoints of stereo cameras. It then leverages the transitive information from these global tracking IDs to reconnect the disrupted tracklets in each camera view. CollabMOT improves the overall performance of baseline 2D MOT methods on a single camera view by mitigating the problem of ID switching. Evaluation of CollabMOT on Argoverse-HD and KITTI dataset shows improved performance over baseline MOT methods. As a result, the proposed method improves the performance of the recent state-of-the-art method on the 2D MOT task of the KITTI dataset from 79.5 to 80% on High Order Tracking Accuracy (HOTA) score for vehicles.

When light passes through air–glass and glass–water interfaces, refraction occurs, which affects the accuracy of stereo vision three-dimensional measurements of underwater targets. To eliminate the impact of refraction, we developed a refractive stereo vision measurement model based on light propagation paths, utilizing the normalized coordinate of the underwater target. This model is rigorous in theory, and easy to understand and apply. Additionally, we established an underwater simulation imaging model based on the principle that light travels the shortest time between two points. Simulation experiments conducted using this imaging model verified the performance of the underwater stereo vision measurement model. The results demonstrate that the accuracy achieved by the new measurement model is comparable to that of the stereo vision measurement model in the air and significantly higher than that of the existing refractive measurement model. This is because the light rays from the camera’s optical center to the refraction point at the air–glass interface do not always intersect. The experiments also indicate that the deviation in the refractive index of water lead to corresponding systematic errors in the measurement results. Therefore, in real underwater measurements, it is crucial to carefully calibrate the refractive index of water and maintain the validity of the calibration results.

This paper presents an innovative dataset designed explicitly for challenging agricultural environments, such as greenhouses, where precise location is crucial, but GNNS accuracy may be compromised by construction elements and the crop. The dataset was collected using a mobile platform equipped with a set of sensors typically used in mobile robots as it was moved through all the corridors of a typical Mediterranean greenhouse featuring tomato crops. This dataset presents a unique opportunity for constructing detailed 3D models of plants in such indoor-like spaces, with potential applications such as robotized spraying. For the first time, to the authors’ knowledge, a dataset suitable to test simultaneous localization and mapping (SLAM) methods is presented in a greenhouse environment, which poses unique challenges. The suitability of the dataset for this purpose is assessed by presenting SLAM results with state-of-the-art algorithms. The dataset is available online. [ABSTRACT FROM AUTHOR]

Recovering and recharging autonomous underwater vehicles (AUVs) on a regular basis allows for long‐term underwater activities. This research provides a vision‐based navigation strategy for AUVs to independently identify and reconstruct the docking station (DS). The proposed framework includes a light beacon detection approach, a filter‐based light beacon matching method, and a fusion pose estimation method for DS positioning. Four green LED light beacons are mounted symmetrically on the docking ring, enabling the stereo camera to observe them from close range. A method for detecting light beacons is proposed that assures detection accuracy by identifying false positives. On a single frame from a stereo camera, filter‐based matching follows the light beacons precisely. In addition, we construct a position estimate method that significantly improves accuracy and efficiency by consisting of analyzation and iteration. A series of virtual and real‐world experiments demonstrate that our methodology can provide AUVs with reliable docking navigation. [ABSTRACT FROM AUTHOR]