Search

With the goal of solving the problem of feature extractors lacking strong supervision training and insufficient time information concerning single‐sequence model learning, a hierarchical sequence memory network with a multi‐level iterative optimisation strategy is proposed for continuous sign language recognition. This method uses the spatial‐temporal fusion convolution network (STFC‐Net) to extract the spatial‐temporal information of RGB and Optical flow video frames to obtain the multi‐modal visual features of a sign language video. Then, in order to enhance the temporal relationships of visual feature maps, the hierarchical memory sequence network is used to capture local utterance features and global context dependencies across time dimensions to obtain sequence features. Finally, the decoder decodes the final sentence sequence. In order to enhance the feature extractor, the authors adopted a multi‐level iterative optimisation strategy to fine‐tune STFC‐Net and the utterance feature extractor. The experimental results on the RWTH‐Phoenix‐Weather multi‐signer 2014 dataset and the Chinese sign language dataset show the effectiveness and superiority of this method. [ABSTRACT FROM AUTHOR]

The interest of science community towards planet Mars has progressively increased in past decades. Rovers are already doing an excellent job in exploring the Martian surface environment and showed that the Red Planet is not suited for life. However, there is a possibility of enabling human exploration by having astronauts take shelter in habitats located within lava tubes, which can provide protection against radiation and dust storms. These tubes are extremely hard to explore with rovers, thus a flying vehicle will be more suitable for such tasks. INGENUITY (Martian helicopter) had already demonstrated that it is possible to fly on Mars by doing small and simple manoeuvres. Based on NASA's design, this article will introduce the development of guidance, navigation, and control operations of a Martian Inspection Drone (MID) that will be capable of finding the lava tube and scan it in real time using a RealSense camera. Linux based system was chosen in the development process, which allow ROS catkin workspace to connect ArduPilot and Gazebo. The navigation is accomplished using a combination of an inertial measurement unit, a camera, and a laser rangefinder. The flight computer runs an optical flow algorithm for control and navigation purpose. Also, a convolutional neural network is used to perform lava tube entrance detection and inspection. • Martian flying drone simulated with ArduPilot, Gazebo and ROS. • Martian environment simulation. • Martian lava tube detection using convolutional neural networks. • Martian lava tube inspection using a realsense camera. [ABSTRACT FROM AUTHOR]

To properly maintain and manage track surfaces, the amount of collapsed settlement of a ballast bed must be determined. However, high maintenance costs make it difficult for railway operators to fully maintain and manage track surfaces. Therefore, we propose a method of estimating the shape of the ballast bed using Dense Inverse Search (DIS) optical flow in images captured using a camcorder, which is inexpensive. This paper describes the outline of the method, an evaluation of its accuracy through simulation and model experiments, and the results of applying the method to actual images captured. [ABSTRACT FROM AUTHOR]

Overlapping occlusion has seriously limited the yield estimation of bagged grapes in recent years, due to the ever-increasing grape volume after bagging and the large surface area of grape leaves. The unstable speed of manual video shooting can also lead to the loss of bagging grape targets. In this study, a yield estimation was proposed for the bagged grape using self-correcting Non-Maximum Suppression (NMS)-ByteTrack. Firstly, the bagged grapes were detected in the video using object detection (YOLOv5s). The NMS operation was also post-positioned to the tracking stage to retain the fruit detection boxes that filtered under the occlusion. Specifically, the detection boxes of bagged grapes were detected by object detection (YOLOv5s), whereas, the prediction boxes of bagged grapes were predicted to calculate the intersection over union using the Kalman filter. If the intersection over the union was less than the given threshold, the detection box was filtered to reserve the detection boxes closer to the predicted position. Then, the final detection box was obtained using the NMS operation. Secondly, the camera motion compensation and improved Kalman filter algorithm were added to automatically correct the position of the fruit prediction boxes and track them using ByteTrack. Specifically, the camera motion compensation was used to first extract the background key points in the bagged grape pictures of the previous frame and the current frame except for the tracking target. The sparse optical flow was used to match the extracted background in the key points of the bagged grapes. Then the affine transformation matrix of background motion was calculated by the RANSAC algorithm, and the Kalman filter was utilized to predict the bagged grape prediction boxes of the current frame. Finally, the affine transformation matrix was obtained to convert the prediction boxes in the coordinate system from the previous to the current frame, in order to realize the position information correction for the prediction boxes of the current frame. In addition, the improved Kalman filter algorithm can be used to change the aspect ratio in the state vector, and then to express the position of the bagged grape tracking boxes in the Kalman filter directly as the width, in order to more accurately estimate the position of the bagged grape tracking boxes. As such, a line-counting strategy was proposed to automatically count the bagged grapes. The strategy was also to set the counting line in the middle of the video, and also retain the idle frames without the bagged grapes in the first few seconds of the video. Furthermore, the bagged grapes with the same ID were only counted once to avoid repeated collision line counting. The successful collision between the center point of the tracking box of bagged grapes and the counting line was achieved in the automatic counting of bagged grapes. The dataset was collected from the Paidengte Agricultural Science and Technology Demonstration Park, Bishan District, Chongqing of China. The mobile phone cameras of Redmi K40 and OPPO Reno6pro+ were selected to capture the pictures of the same bagged grape at 8:00, 12:00, and 18:00 from different angles, such as frontal, sideways, and overhead shots. The total shooting time was about 6 h, the shooting height was about 1.5 m from the ground, and the shooting route was line by line. A total of 500 images and six videos were obtained for the bagged grapes. Among them, 500 images were expanded to 2000 images after saturation, brightness enhancement, brightness reduction, and mirror operations. Then, 2000 images were randomly divided into the training and validation sets in the ratio of 8:2, where six valid videos were used as the test set. Relevant experiments were conducted using this dataset. The experimental results showed that the yield estimation achieved a significant improvement in the tracking performance of bagged grapes using self-correcting NMS-ByteTrack. The multi-object tracking accuracy and precision, as well as the identification F1-score, were 64.6%, 82.4% and 80.8%, respectively, which increased by 1.7, 1.0 and 4.1 percentage points, respectively, compared with the ByteTrack. The number of ID switch was reduced by 3 times. Then, the average counting accuracy reached 82.8% in terms of counting performance, compared with manual counting. In addition, a comparison was also made with the five tracking methods. A better tracking performance was achieved, compared with the rest. The applicability of this estimation was also verified in tracking and counting bagged grapes. Therefore, the yield estimation can be expected to effectively promote the tracking and counting of bagged grapes in real scenarios using self-correcting NMS-ByteTrack. More accurate yield estimation of bagged grapes was also achieved in this field. [ABSTRACT FROM AUTHOR]

In recent years, autonomous robots have been practically used outdoors for transportation, delivery, and other applications. To ensure safety and reliability, it is necessary to measure and evaluate the accuracy of a robot's movement from the outside. In this study, we develop a device to externally measure the coordinates and azimuth angle of a robot by combining image processing and distance measurement. In this study, an azimuth angle detection method that combines coarse-angle detection using AKAZE feature detection and small-angle detection using optical flow is proposed. The experimental results show that the azimuth angle can be detected with a standard deviation of σ=0.70° indoors and σ=0.99° outdoors. [ABSTRACT FROM AUTHOR]

This study presents the algorithm ACTION, defined as automatic center detection of tropical cyclone (TC) using image processing based on the operational radar network. Based on the high visibility of weather radar imagery, a TC's motion vector is calculated from the continuous image change using optical flow, producing the TC's rotation center. The algorithm performance was verified by analyzing the typhoons (TCs in the northwestern Pacific) that affected the Korean Peninsula from 2018–2021, demonstrating a high detection rate of 80.8% within an error distance of 40 km against the best track of the Korea Meteorological Administration (KMA). The detection rate was 100% for typhoons with temporally consistent morphological characteristics. ACTION automatically generates TC center information upon the TC's initial entry inside the observation radius even in the absence of uniform radar data. As ACTION is capable of performing real-time calculations that are directly applied to rapidly generated weather radar images, it is currently being utilized by the KMA. The high temporal resolution TC center information calculated through ACTION is expected to improve the efficiency of TC forecasting. [ABSTRACT FROM AUTHOR]

Change detection is an important task in remote sensing image processing and analysis. However, due to position errors and wind interference, bi-temporal low-altitude remote sensing images collected by SUAVs often suffer from different viewing angles. The existing methods need to use an independent registration network for registration before change detection, which greatly reduces the integrity and speed of the task. In this work, we propose an end-to-end network architecture RegCD-Net to address change detection problems in the bi-temporal SUAVs' low-altitude remote sensing images. We utilize global and local correlations to generate an optical flow pyramid and realize image registration through layer-by-layer optical flow fields. Then we use a nested connection to combine the rich semantic information in deep layers of the network and the precise location information in the shallow layers and perform deep supervision through the combined attention module to finally achieve change detection in bi-temporal images. We apply this network to the task of change detection in the garbage-scattered areas of nature reserves and establish a related dataset. Experimental results show that our RegCD-Net outperforms several state-of-the-art CD methods with more precise change edge representation, relatively few parameters, fast speed, and better integration without additional registration networks. [ABSTRACT FROM AUTHOR]

— In this study, an image processing-based deep learning approach for short-term forecast of solar radiation has been developed. For this purpose, firstly, cloud movements occurred during the day are tracked and future cloud movements are forecasted, accordingly. Subsequently, using the cloud motion estimation and extraterrestrial solar radiation data, 1-min averaged solar radiation values are estimated for 5-min time horizon. Shi-Tomasi method is employed to determine the feature points to be tracked on the sky images whereas, Lucas-Kanade optical flow method is employed to track the determined feature points on the sequential images. Average cloud velocity and directions are calculated by the help of linear regression method from tracked cloud movements. A hybrid approach including K-means and red/blue ratio is built to classify the pixels of the image whether they are clouds or sky. Finally, short-term solar radiations are estimated using the Long-Short Term Memory (LSTM) deep learning method. The performance of the proposed approach is compared with other methods in the literature. As a result it is concluded that, developed approach outperforms most methods in the literature with RMSE values of 47.576, 53.830, 68.103, and 92.386 for four different days and can be used as an alternative approach. [ABSTRACT FROM AUTHOR]

As image acquisition devices have outstanding potential for gathering vibration information, computer vision has received a lot of interest in structural health monitoring (SHM). In this work, a fully automated peak picking methodology based on computer vision in tandem with deep learning is proposed to realize vibration measurements and identify natural frequencies from the plot of the power spectral density transmissibility (PSDT). A deep-learning-enhanced image processing technology was used to extract the vibration signals with automatic active pixel selection, while a convolutional neural network was used to further process the vibration measurements so that the frequencies could be identified from PSDT-based functions. The proposed method was verified by three case studies, including the dynamic testing of two laboratory models and the field testing of the stay cable. The findings showed that the proposed deep-learning-enhanced approach has a high potential for use in SHM by automatically performing vibration measurement and frequency extraction. [ABSTRACT FROM AUTHOR]

This study sought an effective detection method not only for flame but also for the smoke generated in the event of a fire. To this end, the flame region was pre-processed using the color conversion and corner detection method, and the smoke region could be detected using the dark channel prior and optical flow. This eliminates unnecessary background regions and allows selection of fire-related regions. Where there was a pre-processed region of interest, inference was conducted using a deep-learning-based convolutional neural network (CNN) to accurately determine whether it was a flame or smoke. Through this approach, the detection accuracy is improved by 5.5% for flame and 6% for smoke compared to when a fire is detected through the object detection model without separate pre-processing. [ABSTRACT FROM AUTHOR]

Optical flow estimation is a fundamental task in computer vision and image processing. Due to the difficulty in obtaining the ground truth of flow field, unsupervised learning approaches attract more and more research interests in recent years. However, despite of their good generalization capability, unsupervised optical flow methods suffer in the scenarios with large displacement, small objects, and occlusions. In this work, we propose a novel optical flow network based on decoder with multi-scale kernels. Different from previous U-Net like or pyramidal methods, we design our network based on RAFT architecture that with a 4D correlation layer and recurrent decoder. More importantly, we incorporate three novel ideas with regard to the input, information processing and output of the update units improve the performance. Firstly, we utilize various motion-related information as input to the update units. Secondly, we propose a module of multi-scale update unit. Thirdly, for the final flow up-sampling procedure, we propose an image-guided up-sampling loss to guide the learning of up-sampling masks. Our model is trained by the occlusion-aware photometric loss, edge-aware smoothness loss, self-supervised loss, and image-guided up-sampling loss. Experimental results demonstrate that our model achieves the state-of-the-art performance on both Sintel and KITTI and outperforms other unsupervised optical flow methods remarkably. [ABSTRACT FROM AUTHOR]

Fractional calculus is an abstract idea exploring interpretations of differentiation having non-integer order. For a very long time, it was considered as a topic of mere theoretical interest. However, the introduction of several useful definitions of fractional derivatives has extended its domain to applications. Supported by computational power and algorithmic representations, fractional calculus has emerged as a multifarious domain. It has been found that the fractional derivatives are capable of incorporating memory into the system and thus suitable to improve the performance of locality-aware tasks such as image processing and computer vision in general. This article presents an extensive survey of fractional-order derivative-based techniques that are used in computer vision. It briefly introduces the basics and presents applications of the fractional calculus in six different domains viz. edge detection, optical flow, image segmentation, image de-noising, image recognition, and object detection. The fractional derivatives ensure noise resilience and can preserve both high and low-frequency components of an image. The relative similarity of neighboring pixels can get affected by an error, noise, or non–homogeneous illumination in an image. In that case, the fractional differentiation can model special similarities and help compensate for the issue suitably. The fractional derivatives can be evaluated for discontinuous functions, which help estimate discontinuous optical flow. The order of the differentiation also provides an additional degree of freedom in the optimization process. This study shows the successful implementations of fractional calculus in computer vision and contributes to bringing out challenges and future scopes. [ABSTRACT FROM AUTHOR]

In the field of computer vision, high-dynamic dance motion recognition is a difficult problem to solve. Its goal is to recognize human motion by analyzing video data using image processing and classification recognition technology. Video multifeature fusion has sparked a surge in research in a variety of fields. Several pixel points that can be distinguished and displayed in several adjacent images that can reflect their characteristics are referred to as multifeature fusion. It is responsible for a significant portion of the similarity results between the two video segments. Motion recognition relies heavily on video multifeature fusion, which has a direct impact on the robustness and accuracy of recognition results. The directional gradient histogram features, optical flow direction histogram features, and audio features extracted from dance video are used to characterize dance movements after all of the characteristics of dance movements have been considered. This paper focuses on the high-dynamic dance action recognition method based on video multifeature fusion, which aims to combine high-dynamic dance action recognition and video multifeature fusion. [ABSTRACT FROM AUTHOR]

• We would like to submit the manuscript entitled "High-order relational generative adversarial network for video super-resolution", which we wish to be considered as a research article for publication in "Pattern Recognition". Because our contributed research covers the area" Image processing and representation", this manuscript belongs in this journal. No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. • Both the instability of motion compensation and feature aggregation often undermine the SR results of videos. Only the global spatio-temporal dependencies of same scale features are considered as one-order relations, with difficulty in handling the complex motions and context variations. To deal with these issues, we argue that high-order relations should be modelled and applied in the generative strategies to exploit feature alignments and long-range information fusion. In fact, the high-order relations mean that the dependencies of features among pixel positions are mined not only from global feature representations but also through local patch interactions. Moreover, the stronger relations can be further revealed across different scales, by which the aggregation weight matrices with high-order statistics are adaptively determined. The videos are regarded as the complementary fusion of motion and context. Thus, simultaneously capturing the underlying relations of the motion and context is the key to video SR. In this paper, we propose a high-order relational generative adversarial network (HOR-GAN) for accurate video SR, which has strong capacity of alleviating the erroneous motion compensation and merging the useful information among the consecutive frames. • In summary, we highlight the main contributions of this paper as follows: We propose a HOR-GAN framework for highly accurate and realistic video SR. By exploiting high-order relations of feature patches under the construction of the pyramid graphs, the proposed HOR-GAN works well to produce a better feature alignment and fuse more spatio-temporal information. We adopt the dual discriminators to provide spatially coherent feedback to the generator, thus making the generator focus more on fine-grained features. The effectiveness of the proposed method has been justified through the extensive experiments. • We design a motion-aware relation module to accurately align neighboring frames with the reference ones. In this module, a patch-wise matching strategy is first used to build cross-scale correspondences of similar patches. The graph attention layers then adaptively aggregate the local patch features to further decrease the alignment error. We develop a context-aware relation module to make full use of high-order dependencies among all warped feature patches. We introduce the multi-scale graph convolution layers to mine contextual interaction relations for better fusing the spatio-temporal features, in which the position information is encoded to augment the detail recovery. Finally, each pixel is enhanced via a global self-attention. • We deeply appreciate your consideration of our manuscript, and we look forward to receiving comments from the reviewers. If you have any queries, please don't hesitate to contact me. Yours sincerely, Rui Chen Video super-resolution can reconstruct a sequence of high-resolution frames with temporally consistent contents from their corresponding low-resolution sequences. The key challenge for this task is how to effectively utilize both inter-frame temporal relations and intra-frame spatial relations. The existing methods for super-resolving the videos commonly estimate optical flows to align the features of multiple frames based on temporal correlations. However, motion estimation is often error-prone and hence largely hinders the recovery of plausible details. Moreover, high-order contextual dependencies in the feature space are rarely exploited for further enhancing the spatio-temporal information fusion. To this end, we propose a novel generative adversarial network to super-resolve low-resolution videos, which makes full use of patch embeddings and is effective in exploring high-order spatio-temporal relations of the feature patches. Specifically, a motion-aware relation module is designed to handle the alignment between neighboring frames and reference ones. Depending on a patch-matching strategy for adaptive selection of multiple most similar patches, the cross-scale graph is constructed to reliably aggregate these patches using a feature pyramid. Based on the structure of multi-scale graph, a context-aware relation module is developed to capture high-order dependencies among resulting warped patches for better leveraging long-range complementary contexts. To further enhance reconstruction ability, the temporal position information of video sequences is also encoded into this module. Dual discriminators with cycle consistent constraints are adopted to provide more informative feedback to the generator while maintaining the global coherence. Extensive experiments have demonstrated the effectiveness of the proposed method in terms of quantitative and qualitative evaluation metrics. [ABSTRACT FROM AUTHOR]

Cryo-electron microscopy using single particle analysis requires the computational averaging of thousands of projection images captured from identical macromolecules. However, macromolecules usually present some degree of flexibility showing different conformations. Computational approaches are then required to classify heterogeneous single particle images into homogeneous sets corresponding to different structural states. Nonetheless, sometimes the attainable resolution of reconstructions obtained from these smaller homogeneous sets is compromised because of reduced number of particles or lack of images at certain macromolecular orientations. In these situations, the current solution to improve map resolution is returning to the electron microscope and collect more data. In this work, we present a fast approach to partially overcome this limitation for heterogeneous data sets. Our method is based on deforming and then moving particles between different conformations using an optical flow approach. Particles are then merged into a unique conformation obtaining reconstructions with improved resolution, contrast and signal-to-noise ratio. We present experimental results that show clear improvements in the quality of obtained 3D maps, however, there are also limits to this approach, i.e., the method is restricted to small deformations and cannot determine local patterns of flexibility of small elements, such as secondary structures, which we discuss in the manuscript. [Display omitted] • Cryo-EM using SPA requires the computational averaging of thousands of projections captured from identical macromolecules. • Computational approaches are required to classify heterogeneous particles into homogeneous sets at different conformations. • We propose a fast and efficient approach that is aimed at "making the best" from relatively limited data sets. • Our method focuses on artificially increasing the number of particles in a class. • In this regard, we use a regularized optical flow approach to move particles between different classes/conformations. [ABSTRACT FROM AUTHOR]

High‐dynamic‐range image (HDRI) displays scenes as vivid as the real scenes. HDRI can be reconstructed by fusing a set of bracketed‐exposure low‐dynamic‐range images (LDRI). For the reconstruction, many works succeed in removing the ghost artefacts caused by moving objects. The critical issue is reconstructing the areas which are saturated due to bad exposure and occluded due to motion with no ghost artefacts. To overcome this issue, this paper proposes texture and exposure awareness based refill. The proposed work first locates the saturated and occluded areas existing in input image set, then refills background textures or patches containing rough exposure and colour information into located areas. Proposed work can be integrated with multiple existing ghost removal works to improve the reconstruction result. Experimental results show that proposed work removes the ghost artefacts caused by saturated and occluded areas in subjective evaluation. For the objective evaluation, the proposed work improves the HDR‐VDP‐2 evaluation result for multiple conventional works by 1.33% on average. [ABSTRACT FROM AUTHOR]

Visual odometry (VO) is one of the essential techniques in mobile robots field; an accurate VO system is of great significance for mobile robot simultaneous localization and mapping. As for traditional monocular VO systems, they work by presuming the monocular scale is 1 (scale = 1), or relying on ground truth (GT) to estimate scale. As a result, the traditional monocular VO systems estimate the pose state with big drift or cannot work on the image sequence without GT. Although some classical monocular VO systems have been proposed, they still have imperfect performance or even unable to work in some extreme scene conditions, such as scene is monotony without obvious texture information or camera large-scale displacement motion. As for learning-based VO system, it is realized by training deep neural networks in supervised or self-supervised manner to end-to-end estimate the pose state; however, the accuracy of pose estimation entirely depends on the ability of networks. Although the ability of networks can be improved by increasing the number of training data sets and optimizing the network structure, it is inevitable to encounter problems such as insufficient generalization ability and insufficient accuracy on rotational pose estimation. In this article, a monocular VO system named DL_Hybrid is proposed, which takes full advantage of DL networks used in image processing and geometric localization theory based on hybrid pose estimation methods. The DL_Hybrid VO system can estimate a six-DoF pose one-frame-by-one-frame and recover camera trajectory, and it can extract accurate key points from per-frame even in extreme scene condition, and it has good performance even in the extreme moving condition, such as camera rotation-only action or static action, also it can work well in the condition of camera large-scale displacement motion. The real scale is also accurately estimated without depending on GT, and the pose estimation method is designed based on hybrid 2d–2d and 3d–2d localization theory to make the DL_Hyrid VO system to estimate translational and rotational information with accuracy and robustness. Experimental results show that the proposed DL_Hybrid VO system has a better performance than traditional and learning-based VO systems. [ABSTRACT FROM AUTHOR]

Unmanned Ariel Vehicles (UAVs) require identifying water surfaces during flight maneuvers, mainly for safety in execution and its applications. We introduce two novel techniques to identify water surfaces from front-facing and downward-facing cameras mounted on a UAV. The first method — UNet-RAU, a unique architecture based on UNet and Reflection Attention Units, segments water pixels from front-facing camera views, utilizing the reflection property of water surfaces. On the On-Road and Off-Road datasets of Puddle-1000, UNet-RAU improved its performance by 2% over the state-of-the-art FCN-RAU. Additionally, the UNet-RAU generated an F1-score of 80.97% on our Drone-Water-Front dataset. The second method — Dense Optical Flow based Water Detection (DOF-WD), detects water surfaces in videos of downward-facing cameras. This method utilizes downwash-generated ripples and natural texture features on a water surface to identify water in low and high altitudes, respectively. We empirically validated the performance of the DOF-WD method using our Drone-Water-Down dataset. [ABSTRACT FROM AUTHOR]

Purpose: To develop an image processing methodology for noninvasive three‐dimensional (3D) quantification of microwave thermal ablation zones in vivo using x‐ray computed tomography (CT) imaging without injection of a contrast enhancing material. Methods: Six microwave (MW) thermal ablation procedures were performed in three pigs. The ablations were performed with a constant heating duration of 8 min and power level of 30 W. During the procedure images from sixty 1 mm thick slices were acquired every 30 s. At the end of all ablation procedures for each pig, a contrast‐enhanced scan was acquired for reference. Special algorithms for addressing challenges stemming from the 3D in vivo setup and processing the acquired images were prepared. The algorithms first rearranged the data to account for the oblique needle orientation and for breathing motion. Then, the gray level variance changes were analyzed, and optical flow analysis was applied to the treated volume in order to obtain the ablation contours and reconstruct the ablation zone in 3D. The analysis also included a special correction algorithm for eliminating artifacts caused by proximal major blood vessels and blood flow. Finally, 3D reference reconstructions from the contrast‐enhanced scan were obtained for quantitative comparison. Results: For four ablations located >3 mm from a large blood vessel, the mean dice similarity coefficient (DSC) and the mean absolute radial discrepancy between the contours obtained from the reference contrast‐enhanced images and the contours produced by the algorithm were 0.82 ± 0.03 and 1.92 ± 1.47 mm, respectively. In two cases of ablation adjacent to large blood vessels, the average DSC and discrepancy were: 0.67 ± 0.6 and 2.96 ± 2.15 mm, respectively. The addition of the special correction algorithm utilizing blood vessels mapping improved the mean DSC and the mean absolute discrepancy to 0.85 ± 0.02 and 1.19 ± 1.00 mm, respectively. Conclusions: The developed algorithms provide highly accurate detailed contours in vivo (average error < 2.5 mm) and cope well with the challenges listed above. Clinical implementation of the developed methodology could potentially provide real time noninvasive 3D accurate monitoring of MW thermal ablation in‐vivo, provided that the radiation dose can be reduced. [ABSTRACT FROM AUTHOR]

Mobile robots equipped with camera sensors are required to perceive humans and their actions for safe autonomous navigation. For simultaneous human detection and action recognition, the real-time performance of the robot vision is an important issue. In this paper, we propose a robot vision system in which original images captured by a camera sensor are described by the optical flow. These images are then used as inputs for the human and action classifications. For the image inputs, two classifiers based on convolutional neural networks are developed. Moreover, we describe a novel detector (a local search window) for clipping partial images around the target human from the original image. Since the camera sensor moves together with the robot, the camera movement has an influence on the calculation of optical flow in the image, which we address by further modifying the optical flow for changes caused by the camera movement. Through the experiments, we show that the robot vision system can detect humans and recognize the action in real time. Furthermore, we show that a moving robot can achieve human detection and action recognition by modifying the optical flow. [ABSTRACT FROM AUTHOR]

All terrain cranes often work in construction sites. Blind spots, limited information, and high mental workload are problems encountered by crane operators. A top-view camera mounted on the boom head offers a valuable perspective on the workspace that can help eliminate blind spots and provide the basis for assisting operation. In this study, a visual 2D map of a crane workspace is generated from images captured by a top-view camera. Various types of information can be overlaid on this visual 2D map to assist the operator, such as recording the operation and projecting the boom head's expected path through the workspace. Herein, the process of generating a visual map by stitching and locating the boom head trajectory in that visual map is described. Preliminary proof-of-concept tests show that a precise map and projected trajectories can be generated via image-processing techniques that discriminate foreground objects from the scene below the crane. The location error is analyzed and verified to confirm its applicability. These results show a way to help the operator make more precise operation easily and reduce the operator's mental workload. [ABSTRACT FROM AUTHOR]

This paper presents a novel technique for progressive online integration of uncalibrated image sequences with substantial geometric and/or photometric discrepancies into a single, geometrically and photometrically consistent image. Our approach can handle large sets of images, acquired from a nearly planar or infinitely distant scene at different resolutions in object domain and under variable local or global illumination conditions. It allows for efficient user guidance as its progressive nature provides a valid and consistent reconstruction at any moment during the online refinement process. Our approach avoids global optimization techniques, as commonly used in the field of image refinement, and progressively incorporates new imagery into a dynamically extendable and memory‐efficient Laplacian pyramid. Our image registration process includes a coarse homography and a local refinement stage using optical flow. Photometric consistency is achieved by retaining the photometric intensities given in a reference image, while it is being refined. Globally blurred imagery and local geometric inconsistencies due to, e.g. motion are detected and removed prior to image fusion. We demonstrate the quality and robustness of our approach using several image and video sequences, including handheld acquisition with mobile phones and zooming sequences with consumer cameras. [ABSTRACT FROM AUTHOR]

Current vision-based displacement measurement methods have limitations such as being in need of manual targets and parameter adjustment, and significant user involvement to reach the desired result. This study proposes a novel structural displacement measurement method using deep learning-based full field optical flow methods. The performance of the proposed method is verified via a laboratory experiment conducted on a grandstand structure with a comparative study, where the same data samples are analysed with a commonly used vision-based method, and a displacement sensor measurement is used as the ground truth. Statistical analysis of the comparative results show that the proposed method gives higher accuracy than the traditional optical flow algorithm and shows consistent results in compliance with displacement sensor measurements. Image collection, tracking, and non-uniform sampling are investigated in the experimental data and suggestions are made to obtain more accurate displacement measurements. A field-validation on a footbridge showed that the measurement error induced by the camera motion is mitigated by a camera motion subtraction procedure. The proposed method has good potential to be applied by structural engineers, who have little or no experience in computer vision and image processing, to do vision-based displacement measurements. [ABSTRACT FROM AUTHOR]

Video endoscopy (VE) and video fluorography (VF) are the classic inspection methods used for evaluating the function of swallowing, but are disadvantageous as they are invasive. Therefore, we have proposed a noninvasive method that uses pharyngeal ultrasound videos to estimate the internal flow characteristics of foods and evaluates the swallowing function level quantitatively. We are proposing a new method that combines optical flow with Maximally Stable Extremal Regions (MSERs) to extract the movement velocity and position of esophagus and bolus. The results showed that the movement velocity could be used as an indicator to quantify the internal flow characteristics of foods. The displacement of the esophagus indicates the esophageal opening; this could be used as an indicator to evaluate the swallowing. In this study, we confirmed that a classification of esophageal and bolus movements should be performed automatically by the developed method to decrease the evaluation time. [ABSTRACT FROM AUTHOR]

Accurate tracking of organ motion during treatment is needed to improve the efficacy of radiation therapy. This work investigates the feasibility of tracking an uncontoured target using the motion detected within a moving treatment aperture. Tracking was achieved with a weighted optical flow algorithm, and three different techniques for updating the reference image were evaluated. The accuracy and susceptibility of each approach to the accumulation of position errors were verified using a 3D-printed tumor (mounted on an actuator) and a virtual treatment aperture. Tumor motion up to 15.8 mm (peak-to-peak) taken from the breathing patterns of seven lung cancer patients was acquired using an amorphous silicon portal imager at ~ 7.5 frames/s. The first approach (INI) used the initial image detected, as a fixed reference, to determine the target motion for each new incoming image, and performed the best with the smallest errors. This method was also the most robust against the accumulation of position errors. Mean absolute errors of 0.16, 0.32, and 0.38 mm were obtained for the three methods, respectively. Although the errors are comparable to other tracking methods, the proposed method does not require prior knowledge of the tumor shape and does not need a tumor template or contour for tracking. Graphical abstract. [ABSTRACT FROM AUTHOR]

Mobile robots equipped with camera sensors are required to perceive surrounding humans and their actions for safe and autonomous navigation. In this work, moving humans are the target objects. For robot vision, real-time performance is an important requirement. Therefore, we propose a robot vision system in which the original images captured by a camera sensor are described by optical flow. These images are then used as inputs to a classifier. For classifying images into human and not-human classifications, and the actions, we use a convolutional neural network (CNN), rather than coding invariant features. Moreover, we present a local search window as a novel detector for clipping partial images around target objects in an original image. Through the experiments, we ultimately show that the robot vision system is able to detect moving humans and recognize action in real time. [ABSTRACT FROM AUTHOR]

Background Oriented Schlieren (BOS) is a non-intrusive optical method used to visualize density variations in fluids. A camera is used to measure the distortion of a background image displaying a random pattern due to the refraction of the light induced by the local gradients in the refractive index of the under study. In contrast to conventional schlieren, for which direct observation is possible, the images obtained with the BOS technique have to be processed. In this contribution, a computationally affordable approach based on linear least-squares minimization and adapted from optical flow techniques is presented in detail and tested in the context of BOS with the Matlab GUI application comBOS. A series of synthetic images are first analyzed to devise design rules for the definition of an optimal random Gaussian dot background pattern. Tests are performed for comBOS, the commercial BOS module included in DaVis 10, and the free PIV tool PIVlab. It is found that the diameter of the dot images on the camera sensor should be at least 3 pixels, but can be larger without affecting the accuracy of the measurement with comBOS and DaVis 10, while PIVlab severely degrades the quality of the results for dots larger than 5 pixels. The contrast and the density of dots must also be sufficient such that a minimum of 75 % of the surface of the background is shaded. If these conditions are met, the maximum systematic error in the measured distortions is 0.006 pixel for comBOS and 0.001 pixel with DaVis 10 when local gradients are small. Similar errors are found with PIVlab for dots smaller than 5 pixels in diameter, but they increase rapidly for larger dots. The random error is typically smaller than 0.004 pixel in optimal conditions, but it increases significantly for images affected by a random noise or local gradients of the distortion. Finally, it is concluded with a realistic test case that both comBOS and DaVis 10 are well suited to capture the main features of a complex compressible flow with Schlieren imaging. Thus, the main advantage of comBOS is its affordability and robustness for researchers and educators willing to implement BOS, for example in educational contexts. • A BOS Image Processing algorithm that is used in the application comBOS is presented. • Unlike common tools, linear least squares fitting is used instead of correlation. • Design rules are devised to build an optimal background for comBOS and DaVis 10. • Performances are found similar between comBOS and the BOS module of DaVis 10. [ABSTRACT FROM AUTHOR]

Amorphous thin films of Ge 2 Sb 2 Te 5 , sputter-deposited on polycarbonate (PC) substrates, are investigated for purpose of understanding the phase change process by transient optical reflectivity measurements. It was found that phase transition dynamics induced by nanosecond laser pulses were complex optical evolution processes. To understand the effect of heat flow conditions, the aluminium (Al) film with the thickness of 200 nm was introduced as a heat sink layer. The result indicates that the phase change process can be controlled and tunneled by heating conditions due to the addition of Al under-layer with larger thermal conductivity. Finally, a three-level reflectivity was achieved induced by well-selected multi-pulse with identical laser pulse fluence. [ABSTRACT FROM AUTHOR]

In a photo, motion blur can be used as an artistic style to convey motion and to direct attention. In panning or tracking shots, a moving object of interest is followed by the camera during a relatively long exposure. The goal is to get a blurred background while keeping the object sharp. Unfortunately, it can be difficult to impossible to precisely follow the object. Often, many attempts or specialized physical setups are needed. This paper presents a novel approach to create such images. For capturing, the user is only required to take a casually recorded hand‐held video that roughly follows the object. Our algorithm then produces a single image which simulates a stabilized long time exposure. This is achieved by first warping all frames such that the object of interest is aligned to a reference frame. Then, optical flow based frame interpolation is used to reduce ghosting artifacts from temporal undersampling. Finally, the frames are averaged to create the result. As our method avoids segmentation and requires little to no user interaction, even challenging sequences can be processed successfully. In addition, artistic control is available in a number of ways. The effect can also be applied to create videos with an exaggerated motion blur. Results are compared with previous methods and ground truth simulations. The effectiveness of our method is demonstrated by applying it to hundreds of datasets. The most interesting results are shown in the paper and in the supplemental material. [ABSTRACT FROM AUTHOR]

Additive alignment inside a polymer composite significantly affects the physical properties of the material. Many studies have attempted to predict, measure, and control additive alignment inside polymer composites with the aim of enhancing their properties. The objective of this study is to analyze additive alignment inside a mixture of ball-milled carbon fibers and liquid polydimethylsiloxane flowing in a channel. To this end, an image-processing method was developed to more quickly and accurately analyze the additive alignment in images obtained from flow visualization experiments. A process was developed to calculate the angle and length of an additive according to its location and the time at which the image was captured, by analyzing a sequence of images obtained from an optical microscope with a high-speed camera. The proposed process was applied to images composed of arbitrarily drawn lines and actual flow visualization images, and the results were verified by being compared with those of calculations based on the Hough transform or with directly measured values. [ABSTRACT FROM AUTHOR]

In this paper, we deal with the problem of estimating the local strain tensor from a sequence of micro-structural images realized during deformation tests of engineering materials. Since the strain tensor is defined via the Jacobian of the displacement field, we propose to compute the displacement field by a variational model which takes care of properties of the Jacobian of the displacement. In particular, we are interested in areas of high strain. The data term of our variational model relies on the brightness invariance property of the image sequence. As prior we choose the second order total generalized variation of the displacement field. This splits the Jacobian into a smooth and a non-smooth part. The latter reflects the material cracks. An additional constraint is incorporated to handle physical properties of the non-smooth part for tensile tests. We prove that the resulting convex model has a minimizer and show how a primal-dual method can be applied to find a minimizer. The corresponding algorithm has the advantage that the strain tensor is directly computed within the iteration process. It is further equipped with a coarse-to-fine strategy to cope with larger displacements. Numerical examples with simulated and experimental data demonstrate the very good performance of our algorithm. In comparison to state-of-the-art engineering software, our method can resolve local phenomena much better. [ABSTRACT FROM AUTHOR]

Abstract Dense motion estimations obtained from optical flow techniques play a significant role in many image processing and computer vision tasks. Remarkable progress has been made in both theory and its application in practice. In this paper, we provide a systematic review of recent optical flow techniques with a focus on the variational method and approaches based on Convolutional Neural Networks (CNNs). These two categories have led to state-of-the-art performance. We discuss recent modifications and extensions of the original model, and highlight remaining challenges. For the first time, we provide an overview of recent CNN-based optical flow methods and discuss their potential and current limitations. Highlights • Introducing optical flow: the basic concepts, the characteristics of the variational and CNN-based techniques, and the evaluation measures. • Discussing developments of the variational method, analyzing the challenges and illustrating the corresponding treating strategies of it. • Describing the conception of the CNN-based technique, and give a detailed discussion of the issues of this technique. [ABSTRACT FROM AUTHOR]

Due to the factors like rapidly fast motion, cluttered backgrounds, arbitrary object appearance variation and shape deformation, an effective target representation plays a key role in robust visual tracking. Existing methods often employ bounding boxes for target representations, which are easily polluted by noisy clutter backgrounds that may cause drifting problem when the target undergoes large-scale non-rigid or articulated motions. To address this issue, in this paper, motivated by the spatio-temporal nonlocality of target appearance reoccurrence in a video, we explore the nonlocal information to accurately represent and segment the target, yielding an object likelihood map to regularize a correlation filter (CF) for visual tracking. Specifically, given a set of tracked target bounding boxes, we first generate a set of superpixels to represent the foreground and background, and then update the appearance of each superpixel with its long-term spatio-temporally nonlocal counterparts. Then, with the updated appearances, we formulate a spatio-temporally graphical model comprised of the superpixel label consistency potentials. Afterwards, we generate segmentation by optimizing the graphical model via iteratively updating the appearance model and estimating the labels. Finally, with the segmentation mask, we obtain an object likelihood map that is employed to adaptively regularize the CF learning by suppressing the clutter background noises while making full use of the long-term stable target appearance information. Extensive evaluations on the OTB50, SegTrack, Youtube-Objects datasets demonstrate the effectiveness of the proposed method, which performs favorably against some state-of-art methods. [ABSTRACT FROM AUTHOR]

Micro-expression analysis is an interesting and challenging task in computer vision. It has inspired a series of possible applications in many areas such as police–criminal interrogation and important business negotiation. One of the most crucial step in a micro-expression recognition system is the extraction of well discriminating features. In this paper, we propose a new feature based on the fusion of motion boundary histograms (FMBH). This feature is generated by combining both the horizontal and the vertical components of the differential of optical flow as inspired from the motion boundary histograms (MBH). The proposed feature is then validated and evaluated through the leave-one-subject-out (LOSO) protocol for micro-expression recognition. Moreover, the proposed method is compared to state-of-the-art methods on four well-known databases CASME, CASME II, SMIC and CAS(ME) 2 . Comparative experimental results demonstrate that the proposed FMBH feature descriptor yields promising performance. [ABSTRACT FROM AUTHOR]

We developed a technique, using fractal complexity analysis of optical flow in two-dimensional (2-D) videos, to characterize information content in observed motion. Several lines of evidence demonstrate that visually available properties of motion can characterize the state of a system. This paper will describe the method used and will present a test case regarding the accuracy of the method. An analytical comparison of simple human movement (arranging items on a table) and American Sign Language (ASL) will be given as an example application. The normalized spectral density in the range of 0.1 to 15 Hz indicated significantly higher fractal complexity in the optical flow of ASL video data, indicating that information content in 2-D video data can be characterized using complexity analysis of optical flow. The technique used for quantification of information content in visual motion data is likely to be applicable for distinguishing biological versus nonbiological motion in 2-D video data, making inferences about the states of biological objects from the dynamics of optical flow, and in assessing likelihood of information content in a video stream. [ABSTRACT FROM AUTHOR]

An automatic content-based video retrieval (CBVR) system utilises visual data of videos to search for user's desired query from a large collection of videos. The need for CBVR can be attributed to the massive growth of video data, which made the manual-based retrieval of videos a challenging task. CBVR (for human action videos) relies on action recognition to search for videos containing specific actions. This paper proposes an optical flow based local motion texture analysis scheme for human action representation. A single video was used as a query to retrieve similar videos from the database. Local binary pattern for three orthogonal planes over temporal cubes of an optical flow provides the spatial and temporal description of motion. A minimum distance classifier can retrieve videos based on a distance measure between the query and stored videos. The proposed retrieval scheme is able to achieve sixty six percent precision using an existing dataset, which compares well with the state-of-art approaches. [ABSTRACT FROM AUTHOR]

We focus on the challenging cross-view action recognition problem. The key to this problem is to find the correspondence between two different views, which is realized in two stages. First, we construct a dualcodebook for the two views, which contains one codebook for each view. Each codeword in one codebook has a corresponding codeword in the other codebook, whereas traditional methods implement independent codebooks for the views. We propose an effective coclustering algorithm based on seminonnegative matrix factorization to derive the dual-codebook. Additionally, to represent actions in one view, unlike most other methods using the codebook of that view only, we also exploit the codebook-specific information from the other view. Thus, we construct mapped-codebooks via codebook transformation, complementing the codebook-to-codebook correspondence within the dual-codebook. In the second stage, observing that the temporal relationship between action segments within an action is view invariant, we further propose a hierarchical transfer framework based on a temporal structure that can effectively capture such action-segment temporal relationship at multiple timescales, which is more discriminative than the usual video-level transfer strategy. Extensive experimental results on the INRIA xmas motion acquisition sequences and West Virginia University datasets demonstrate superiority of the proposed method compared with state-of-the-art approaches. ©2018 SPIE and IS&T [DOI: 10 .1117/1JEI.27.4.043044] [ABSTRACT FROM AUTHOR]

The detection of foreign substances in transparent-bottled liquid is a key procedure in production. An improved algorithm based on the combination of guided filter and visual background extractor (ViBe) is proposed for the detection of foreign substances in transparent-bottled turbid liquid on the market. In the stage of image preprocessing, the image enhancement is carried out by the guided filter and then the liquid region is detected by the Hough transform. Finally, the area of the connected region is determined for the image processed by the ViBe algorithm to remove the tiny noise in detection images. The simulation results show that the proposed algorithm can not only detect foreign substances in transparent liquid but also detect foreign substances in the turbid liquid. © 2018 SPIE and IS&T [DOI: 10.1117/1.JEI.27.4.043015] [ABSTRACT FROM AUTHOR]

Due to the rapid growth of technology and the dropping cost of cameras, multiview imaging applications have attracted many researchers in recent years. Free viewpoint and 3D Televisions are among these interesting applications. One of the problems that should be solved to realize such applications is rendering. In this paper, we propose an optical flow-assisted adaptive patch-based view synthesis algorithm. This patch-based scheme reduces the size and number of holes during reconstruction. The size of patch is determined in response to edge information for better reconstruction, especially near the boundaries. In the first stage of the algorithm, disparity is obtained using optical flow estimation. Then, a reconstructed version of the left and right views is generated using our adaptive patch-based algorithm. The mismatches between each view and its reconstructed version are obtained in the mismatch detection steps. This stage results in two masks as outputs, which help with the refinement of disparities and the selection of the best patches for final synthesis. Finally, the remaining holes are filled using our simple hole-filling scheme and the refined disparities. The objective and subjective performances of the proposed algorithm are compared with recent methods. The results show that the proposed algorithm achieves an improvement of 2.14 dB on average. [ABSTRACT FROM AUTHOR]

Because of its own advantages, micro air vehicles can accomplish reconnaissance / surveillance tasks in low altitude and indoor flight environments, with inertial / satellite integrated navigation system. As a useful supplement to its navigation system, optical flow can assist INS to update position and attitude information in some occasions without GPS assistent. Considering the low calculation precision of optical flow in unconstant illumination condition, optical flow navigation method based on ROF denoising is studied, convex optimization theory and dual principle is discussed, and image decomposition method in the light texture changes is researched on. The obtained texture part is not affected by illumination variations, and used to calculate the movement of field information of the image. Then, the movement information is further calculated for position and attitude of micro air vehichles. The experimental results show that the accuracy of optical flow calculation is improved by decomposing and denoising. [ABSTRACT FROM AUTHOR]

Examination of the common carotid artery (CCA) based on an ultrasound video sequence is an effective method for detecting cardiovascular diseases. Here, we propose a video processing method for the automated geometric analysis of CCA transverse sections. By explicitly compensating the parasitic phenomena of global movement and feature drift, our method enables a reliable and accurate estimation of the movement of the arterial wall based on ultrasound sequences of arbitrary length and in situations where state-of-the-art methods fail or are very inaccurate. The method uses a modified Viola-Jones detector and the Hough transform to localize the artery in the image. Then it identifies dominant scatterers, also known as interest points (IPs), whose positions are tracked by means of the pyramidal Lucas-Kanade method. Robustness to global movement and feature drift is achieved by a detection of global movement and subsequent IP re-initialization, as well as an adaptive removal and addition of IPs. The performance of the proposed method is evaluated using simulated and real ultrasound video sequences. Using the Harris detector for IP detection, we obtained an overall root-mean-square error, averaged over all the simulated sequences, of 2.16 ± 1.18 px. The computational complexity of our method is compatible with real-time operation; the runtime is about 30-70 ms/frame for sequences with a spatial resolution of up to 490 × 490 px. We expect that in future clinical practice, our method will be instrumental for non-invasive early-stage diagnosis of atherosclerosis and other cardiovascular diseases. [ABSTRACT FROM AUTHOR]

Abstract: We propose a novel real-time video stitching method using camera path estimation and homography refinement. The method can stably stitch multiple frames acquired from moving cameras in real time. In the proposed method, one initial between-camera (BC) homography and each camera path (CP) homography are used to estimate the BC homography at every frame. The BC homography is refined by using block matching to adjust the errors of estimated CPs (homography refinement). For fast processing, we extract features using the difference of intensities and use the optical flow to estimate camera motion (CM) homographies, which are multiplied with the previous CMs to calculate CPs (camera path estimations). In experiments, we demonstrated the performance of the CP estimation and homography refinement approach by comparing it with other methods. The experimental results show that the proposed method can stably stitch two image sequences at a rate exceeding 13 fps (frames per second). [ABSTRACT FROM AUTHOR]

Falls are the leading cause of injury and death in elderly individuals. Unfortunately, fall detectors are typically based on wearable devices, and the elderly often forget to wear them. In addition, fall detectors based on artificial vision are not yet available on the market. In this paper, we present a new low-cost fall detector for smart homes based on artificial vision algorithms. Our detector combines several algorithms (background subtraction, Kalman filtering and optical flow) as input to a machine learning algorithm with high detection accuracy. Tests conducted on over 50 different fall videos have shown a detection ratio of greater than 96%. [ABSTRACT FROM AUTHOR]

Recently, the stereo imaging-based image enhancement approach has attracted increasing attention in the field of video analysis. This paper presents a dual camera-based stereo image defogging algorithm. Optical flow is first estimated from the stereo foggy image pair, and the initial disparity map is generated from the estimated optical flow. Next, an initial transmission map is generated using the initial disparity map. Atmospheric light is then estimated using the color line theory. The defogged result is finally reconstructed using the estimated transmission map and atmospheric light. The proposed method can refine the transmission map iteratively. Experimental results show that the proposed method can successfully remove fog without color distortion. The proposed method can be used as a pre-processing step for an outdoor video analysis system and a high-end smartphone with a dual camera system. [ABSTRACT FROM AUTHOR]

By combining a polymer-clad optic fiber and a vitreous-clad optic fiber, we proposed and fabricated a novel optic fiber surface plasmon resonance (SPR) sensor to conduct two-channel sensing at the same detection area. The traditional optic fiber SPR sensor has many disadvantages; for example, removing the cladding requires corrosion, operating it is dangerous, adjusting the dynamic response range is hard, and producing different resonance wavelengths in the sensing area to realize a multi-channel measurement is difficult. Therefore, in this paper, we skillfully used bare fiber grinding technology and reverse symmetry welding technology to remove the cladding in a multi-mode fiber and expose the evanescent field. On the basis of investigating the effect of the grinding angle on the dynamic range change of the SPR resonance valley wavelength and sensitivity, we combined polymer-clad fiber and vitreous-clad fiber by a smart design structure to realize at a single point a two-channel measurement fiber SPR sensor. In this paper, we obtained a beautiful spectral curve from a multi-mode fiber two-channel SPR sensor. In the detection range of the refractive rate between 1.333 RIU and 1.385 RIU, the resonance valley wavelength of channel I shifted from 622 nm to 724 nm with a mean average sensitivity of 1961 nm/RIU and the resonance valley wavelength of channel II shifted from 741 nm to 976 nm with a mean average sensitivity of 4519 nm/RIU. [ABSTRACT FROM AUTHOR]

The paper presents a study of existing Block Matching Optical Flow algorithms, which estimate the motion from an image sequence. The seven algorithms selected for our performance and comparative study (ES, TSS, SETSS, NTSS, FSS, DS and ARPS) are finding a minimum cost match between the current image reference block and one of the considered blocks in the next image. The benchmark used for the performance testing was the Middlebury Dataset, with the conclusions leading to a parameters and algorithm selection criteria. [ABSTRACT FROM AUTHOR]

The paper proposes a vision based guidance strategy for an unmanned aerial vehicle (UAV) navigating through urban canyons. The UAV has a single forward facing camera and relies on optical flow computed from the sequence of images captured by the camera. The steering command is inversely proportional to the optical flow difference between the two sides of the image. This leads to high turning and safe motion while directly facing an obstacle or a wall. Analysis is carried out for infinite and finite length obstacles to highlight the obstacle avoidance properties of the proposed strategy as compared to a widely used existing strategy. Realistic simulation studies with straight, L, and T shape canyons and combination of them show the viability of the proposed strategy while navigating through urban canyons. [ABSTRACT FROM AUTHOR]

Open-air optical flow estimation is crucial for autonomous driving and traffic monitoring. However, it is still challenging to obtain high-precision optical flow under severe imaging environments. To alleviate this problem, we propose a self-adaptive optical flow estimation framework for harsh outdoor scenes, which consists of a frame-adaptive learnable module (FAL-module), a streak-invariant feature extractor (SFE), and a feature matching and flow propagation network. In the framework, three components are jointly learned in an end-to-end manner. The input frames can be adaptively enhanced without sacrificing optical flow-related information by FAL-module. And the SFE decreases the influence of streak-like noise generated by rainy and snowy. Compared with the state-of-the-art methods, the experimental results show that our approach can significantly improve the optical flow estimation quality in multiple harsh outdoor scenes without performance penalties in normal scenes. [ABSTRACT FROM AUTHOR]