Search

In video action recognition, effective spatiotemporal modeling is crucial. However, traditional two-stream methods face challenges in integrating spatial information from RGB images and temporary information from optical flow without long-range temporal modelling. To address these limitations, we propose the Deep Fusion Module (DFM), which focuses on the deep fusion of spatial and temporal information and consists of two components. First, we propose an Attention Fusion Module (AFM) to effectively fuse the shallow features obtained from a two-stream network, thereby facilitating the integration of spatial and temporal information. Next, we incorporate a SpatioTemporal Module (STM), comprising a ConvGRU and a 1×1 convolution, to model long-range temporal dependency and fuse spatial-temporal features. Experiments on the UCF101 dataset show that our method achieves 96.5% accuracy, outperforming baseline two-stream models by 0.3%. [ABSTRACT FROM AUTHOR]

Background: Regional Wall Motion Abnormality (RWMA) serves as an early indicator of myocardial infarction (MI), the global leader in mortality. Accurate and early detection of RWMA is vital for the successful treatment of MI. Current automated echocardiography analyses typically concentrate on peak values from left ventricular (LV) displacement curves, based on LV contour annotations or key frames during the heart's systolic or diastolic phases within a single echocardiographic cycle. This approach may overlook the rich motion field features available in multi-cycle cardiac data, which could enhance RWMA detection. Methods: In this research, we put forward an innovative approach to detect RWMA by harnessing motion information across multiple echocardiographic cycles and multi-views. Our methodology synergizes U-Net-based segmentation with optical flow algorithms for detailed cardiac structure delineation, and Temporal Convolutional Networks (ConvNet) to extract nuanced motion features. We utilize a variety of machine learning and deep learning classifiers on both A2C and A4C views echocardiograms to enhance detection accuracy. A three-phase algorithm—originating from the HMC-QU dataset—incorporates U-Net for segmentation, followed by optical flow for cardiac wall motion field features. Temporal ConvNet, inspired by the Temporal Segment Network (TSN), is then applied to interpret these motion field features, independent of traditional cardiac parameter curves or specific key phase frame inputs. Results: Employing five-fold cross-validation, our SVM classifier demonstrated high performance, with a sensitivity of 93.13%, specificity of 83.61%, precision of 88.52%, and an F1 score of 90.39%. When compared with other studies using the HMC-QU datasets, these Fig s stand out, underlining our method's effectiveness. The classifier also attained an overall accuracy of 89.25% and Area Under the Curve (AUC) of 95%, reinforcing its potential for reliable RWMA detection in echocardiographic analysis. Conclusions: This research not only demonstrates a novel technique but also contributes a more comprehensive and precise tool for early myocardial infarction diagnosis. [ABSTRACT FROM AUTHOR]

Complex steerable pyramid (CSP) performs well when applied to magnify subtle motions of structures for observing the dynamic characteristics of facilities. However, the impact of the types and parameters of CSP filters upon the performance of phase-based optical flow (PBOF) in measuring motion parameters has not been systematically studied. The purpose of this study is to comprehensively evaluate the impact of different CSP filter types (Octave, HalfOctave, SmoothHalfOctave, and QuarterOctave) and parameters on the performance of PBOF in measuring motion parameters. Firstly, by measuring simulated translational motion, the influence of the CSP's down-sampling rates on the displacement measurement accuracy of PBOF is analyzed to determine appropriate settings. Subsequently, the effective displacement measurement interval and accuracy of PBOF using the CSP are studied through simulated and experimental translational motion measurements. Further, the vibration parameter's accuracy is analyzed through simulated periodic vibration measurements. Finally, the characteristics of PBOF using the four kinds of CSP and practical considerations are discussed. Simulation and experimental results demonstrate that when using middle-level filters within the effective level range of HalfOctave, PBOF achieves the best overall displacement measurement performance. Additionally, this method can easily integrate with signal processing techniques in analyzing structural dynamic characteristics under field conditions. [ABSTRACT FROM AUTHOR]

The increasing interest in developing robots capable of navigating autonomously has led to the necessity of developing robust methods that enable these robots to operate in challenging and dynamic environments. Visual odometry (VO) has emerged in this context as a key technique, offering the possibility of estimating the position of a robot using sequences of onboard cameras. In this paper, a VO algorithm is proposed that achieves sub-pixel precision by combining optical flow and direct methods. This approach uses only a downward-facing, monocular camera, eliminating the need for additional sensors. The experimental results demonstrate the robustness of the developed method across various surfaces, achieving minimal drift errors in calculation. [ABSTRACT FROM AUTHOR]

In this study, we analyze mucociliary transport (MCT) by measuring the magnitude and identifying regions of ciliary beats using high-frame-rate microscopic videos. Our methodology, integrating dense optical flow (DOF), connected component labeling (CCL), Butterworth filter, and Fast Fourier Transform (FFT), captures ciliary movement and magnitude. We focus on region extraction, quantification of ciliary activity, and classification of power and recovery strokes in ciliary beat frequency (CBF), which are crucial for evaluating MCT efficiency. Our approach was able to extract the ciliary region semi-automatically, obtain the CBF, and visualize the ciliary movement in each frame. Despite dataset challenges and limited ground truth, our approach shows a promising result for ciliary dynamics research and medical diagnostics. We hope for future open-source datasets with ground-truth ciliary beat patterns to enable developing and evaluating automated ciliary analysis techniques, leading to improved assessment. [ABSTRACT FROM AUTHOR]

Real-time hybrid simulation has gained popularity over the last 20 years as a viable and cost-effective method of testing dynamic systems that cannot be tested using traditional methods. The emergence of multi-axial Real-time Hybrid Simulation (maRTHS) has led to an increase in the allowable fidelity of the numerical and experimental substructures. The testing community can now replicate multiple-degree-of-freedom (MDOF) responses of both substructures and thus can perform more representative tests. However, with this increased fidelity of the substructures comes an increased complexity of controlling these components. Specifically, multi-axial hydraulic actuator assemblages require nonlinear coordinate transformations to derive plant displacements as the force transducers on the actuators are not capable of performing this task directly. Recently, benchmark problems have been provided to the RTHS community in the form of virtual simulations. Virtual simulation refers to a fully virtual testing methodology where numerical and physical components are represented virtually. This approach enables the RTHS community to evaluate various control algorithms without the need to recreate physical components. This project aims to demonstrate the capability of computer vision-based displacement tracking in a realistic virtual simulation of the experimental substructure in avoiding excess nonlinear coordinate transforms. The tracking algorithm utilizing the Lucas-Kanade optical flow method is tested in the virtual simulation environment which is set up using real-time 3D creation engine, Unreal Engine 4 (UE4), and computer graphics software, Blender. This environment interfaces with MATLAB/Simulink, more specifically "Simulation Tool for v-maRTHS benchmark" developed for multi-axial tests. The result of this study establishes a novel framework for applying computer vision-based tracking algorithms and sensing in v-maRTHS simulations using simulated cameras within virtual simulation environments. A computer vision displacement tracking algorithm is developed and optimized to work in tandem with a MIMO PI controller to reduce tracking time delays within 31.25 milliseconds while tracking the nodal displacement and rotation of the frame within a normalized RMSE of 1.24 and 1.10 respectively. [ABSTRACT FROM AUTHOR]

In recent years, the increasing frequency of climate change and extreme weather events has significantly elevated the risk of levee breaches, potentially triggering large-scale floods that threaten surrounding environments and public safety. Rapid and accurate measurement of river surface velocities is crucial for developing effective emergency response plans. Video image velocimetry has emerged as a powerful new approach due to its non-invasive nature, ease of operation, and low cost. This paper introduces the Dynamic Feature Point Pyramid Lucas–Kanade (DFP-P-LK) optical flow algorithm, which employs a feature point dynamic update fusion strategy. The algorithm ensures accurate feature point extraction and reliable tracking through feature point fusion detection and dynamic update mechanisms, enhancing the robustness of optical flow estimation. Based on the DFP-P-LK, we propose a river surface velocity measurement model for rapid levee breach emergency response. This model converts acquired optical flow motion to actual flow velocities using an optical flow-velocity conversion model, providing critical data support for levee breach emergency response. Experimental results show that the method achieves an average measurement error below 15% within the velocity range of 0.43 m/s to 2.06 m/s, demonstrating high practical value and reliability. [ABSTRACT FROM AUTHOR]

In this paper, we proposed Mix-VIO, a monocular and binocular visual-inertial odometry, to address the issue where conventional visual front-end tracking often fails under dynamic lighting and image blur conditions. Mix-VIO adopts a hybrid tracking approach, combining traditional handcrafted tracking techniques with Deep Neural Network (DNN)-based feature extraction and matching pipelines. The system employs deep learning methods for rapid feature point detection, while integrating traditional optical flow methods and deep learning-based sparse feature matching methods to enhance front-end tracking performance under rapid camera motion and environmental illumination changes. In the back-end, we utilize sliding window and bundle adjustment (BA) techniques for local map optimization and pose estimation. We conduct extensive experimental validations of the hybrid feature extraction and matching methods, demonstrating the system's capability to maintain optimal tracking results under illumination changes and image blur. [ABSTRACT FROM AUTHOR]

Data and reports indicate an increasing frequency and intensity of natural disasters worldwide. Buildings play a crucial role in disaster responses and damage assessments, aiding in planning rescue efforts and evaluating losses. Despite advances in applying deep learning to building extraction, challenges remain in handling complex natural disaster scenes and reducing reliance on labeled datasets. Recent advances in satellite video are opening a new avenue for efficient and accurate building extraction research. By thoroughly mining the characteristics of disaster video data, this work provides a new semantic segmentation model for accurate and efficient building extraction based on a limited number of training data, which consists of two parts: the prediction module and the automatic correction module. The prediction module, based on a base encoder–decoder structure, initially extracts buildings using a limited amount of training data that are obtained instantly. Then, the automatic correction module takes the output of the prediction module as input, constructs a criterion for identifying pixels with erroneous semantic information, and uses optical flow values to extract the accurate corresponding semantic information on the corrected frame. The experimental results demonstrate that the proposed method outperforms other methods in accuracy and computational complexity in complicated natural disaster scenes. [ABSTRACT FROM AUTHOR]

The objective of this article is to examine potential techniques for suppressing solar heat flow on the optical antenna of a laser communication sensor. Firstly, the characteristics of the geosynchronous Earth orbit's (GEO) space radiation environment are analysed, and a combined passive and active thermal control solution is proposed. Secondly, the temperature distribution of the lasercom sensor under extreme operating conditions is simulated utilising IDEAS-TMG (6.8 NX Series) software, which employs Monte Carlo and radiative heat transfer numerical calculation methods. Finally, a strategy for avoiding direct sunlight around midnight is proposed. The simulation results demonstrated that the thermal control solution and solar avoidance strategy proposed in this paper achieved long-term fine-stable control of the temperature field of the optical antenna, which met the thermal permissible communication hours per daily orbit cycle in excess of 14 h per day. [ABSTRACT FROM AUTHOR]

High spatial resolution and high accuracy estimation of 3D velocity fields are important for tomographic particle image velocimetry (Tomo-PIV), especially when measuring complex flow fields with delicate 3D structures. However, the widely used cross-correlation-based methods have limited spatial resolution, while the recently developed optical flow-based methods have low robustness and are sensitive to particle volume reconstruction errors. Therefore, 3D velocity estimation methods that simultaneously exhibit high resolution and robustness must be developed. In this study, we propose a novel velocity estimation method for Tomo-PIV measurement using the guided filter-based 3D hybrid variational optical flow (GF-HVOF) method to achieve high spatial resolution and highly accurate measurement of 3D flow field structure. First, we propose a novel L1-norm regularization term based on the Helmholtz decomposition theorem to preserve the divergence and vorticity of the fluid flow. Second, we propose a guided-filter-based constraint term using the result of the cross-correlation-based method as the guided flow field to improve the robustness of the optical flow method. Third, we propose a hybrid constraint term based on particle tracking velocimetry (PTV) method and a spatially weighted data term to reduce the effect of ghost particles and discrete errors generated during the reconstruction of particle volumes. The newly proposed hybrid method combines the advantages of optical-flow-based and cross-correlation-based methods and corrects the flow field using the PTV method. Velocity fields are estimated over synthetic and experimental particle volumes. The results show that the newly proposed GF-HVOF method achieves better performance and greater measurement accuracy than existing 3D fluid motion estimation methods. [ABSTRACT FROM AUTHOR]

Background: Vision-based pulmonary diagnostics present a unique approach for tracking and measuring natural breathing behaviors through remote imaging. While many existing methods correlate chest and diaphragm movements to respiratory behavior, we look at how the direct visualization of thermal CO2 exhale flow patterns can be tracked to directly measure expiratory flow. Methods: In this work, we present a novel method for isolating and extracting turbulent exhale flow signals from thermal image sequences through flow-field prediction and optical flow measurement. The objective of this work is to introduce a respiratory diagnostic tool that can be used to capture and quantify natural breathing, to identify and measure respiratory metrics such as breathing rate, flow, and volume. One of the primary contributions of this work is a method for capturing and measuring natural exhale behaviors that describe individualized pulmonary traits. By monitoring subtle individualized respiratory traits, we can perform secondary analysis to identify unique personalized signatures and abnormalities to gain insight into pulmonary function. In our study, we perform data acquisition within a clinical setting to train an inference model (FieldNet) that predicts flow-fields to quantify observed exhale behaviors over time. Results: Expiratory flow measurements capturing individualized flow signatures from our initial cohort demonstrate how the proposed flow field model can be used to isolate and analyze turbulent exhale behaviors and measure anomalous behavior. Conclusions: Our results illustrate that detailed spatial flow analysis can contribute to unique signatures for identifying patient specific natural breathing behaviors and abnormality detection. This provides the first-step towards a non-contact respiratory technology that directly captures effort-independent behaviors based on the direct measurement of imaged CO2 exhaled airflow patterns. [ABSTRACT FROM AUTHOR]

Static and moving objects often occur in real-life videos. Most video object segmentation methods only focus on extracting and exploiting motion cues to perceive moving objects. Once faced with the frames of static objects, the moving object predictors may predict failed results caused by uncertain motion information, such as low-quality optical flow maps. Besides, different sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only consider either the RGB or RGB and optical flow. In this paper, we propose a novel adaptive multi-source predictor for zero-shot video object segmentation (ZVOS). In the static object predictor, the RGB source is converted to depth and static saliency sources, simultaneously. In the moving object predictor, we propose the multi-source fusion structure. First, the spatial importance of each source is highlighted with the help of the interoceptive spatial attention module (ISAM). Second, the motion-enhanced module (MEM) is designed to generate pure foreground motion attention for improving the representation of static and moving features in the decoder. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By using the ISAM, MEM and FPM, the multi-source features are effectively fused. In addition, we put forward an adaptive predictor fusion network (APF) to evaluate the quality of the optical flow map and fuse the predictions from the static object predictor and the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Experiments show that the proposed model outperforms the state-of-the-art methods on three challenging ZVOS benchmarks. And, the static object predictor precisely predicts a high-quality depth map and static saliency map at the same time. [ABSTRACT FROM AUTHOR]

The occlusion problem remains a crucial challenge in optical flow estimation (OFE). Despite the recent significant progress brought about by deep learning, most existing deep learning OFE methods still struggle to handle occlusions; in particular, those based on two frames cannot correctly handle occlusions because occluded regions have no visual correspondences. However, there is still hope in multi-frame settings, which can potentially mitigate the occlusion issue in OFE. Unfortunately, multi-frame OFE (MOFE) remains underexplored, and the limited studies on it are mainly specially designed for pyramid backbones or else obtain the aligned previous frame's features, such as correlation volume and optical flow, through time-consuming backward flow calculation or non-differentiable forward warping transformation. This study proposes an efficient MOFE framework named SplatFlow to address these shortcomings. SplatFlow introduces the differentiable splatting transformation to align the previous frame's motion feature and designs a Final-to-All embedding method to input the aligned motion feature into the current frame's estimation, thus remodeling the existing two-frame backbones. The proposed SplatFlow is efficient yet more accurate, as it can handle occlusions properly. Extensive experimental evaluations show that SplatFlow substantially outperforms all published methods on the KITTI2015 and Sintel benchmarks. Especially on the Sintel benchmark, SplatFlow achieves errors of 1.12 (clean pass) and 2.07 (final pass), with surprisingly significant 19.4% and 16.2% error reductions, respectively, from the previous best results submitted. The code for SplatFlow is available at https://github.com/wwsource/SplatFlow. [ABSTRACT FROM AUTHOR]

The earthquake that occurred in Palu-Sulawesi Indonesia in 2018 has caused many problems to infrastructure buildings. One of the impacts of the earthquake was the reduction the level of hardness includes the level of stiffness saturated sandy and condition makes the foundation structure experience greater lateral deformation, which can lead to the collapse of the building above it. This phenomenon is called liquefaction. This article describes the results of laboratory simulations using a one-way shaking table. It aims to obtain the lateral resistance of a group of short pile foundations. The lateral resistance is investigated from the amount of lateral deformation of the pile cap. Laboratory modeling used field and laboratory comparisons at a scale of 1:10. Pile foundations are used in 2×2 pile groups. To obtain the lateral deformation of the pile, Optic Flow is used which is placed on top of the pile cap as high as 30cm. Meanwhile, to obtain the increase in pore water pressure, a PWP sensor was used which was inserted at a certain soil depth of 30cm from the ground surface. The test results show that the lateral deformation of the pile cap due to liquefaction can be observed well. The phenomenon of liquefaction can be observed the excess the pressure of pore water into the soil is caused by loading of seismic. Furthermore, observation results were compared through analysis using the Plaxis 3D program, which showed a good agreement. [ABSTRACT FROM AUTHOR]

When analyzing microscopic time-lapse observations, frame alignment is an essential task to visually understand the morphological and translation dynamics of cells and tissue. While in traditional single-sample microscopy, the region of interest (RoI) is fixed, multi-sample microscopy often uses a single microscope that scans multiple samples over a long period of time by laterally relocating the sample stage. Hence, the relocation of the optics induces a statistical RoI offset and can introduce jitter as well as drift, which results in a misaligned RoI for each sample's time-lapse observation (stage drift). We introduce a robust approach to automatically align all frames within a time-lapse observation and compensate for frame drift. In this study, we present a sub-pixel precise alignment approach based on recurrent all-pairs field transforms (RAFT); a deep network architecture for optical flow. We show that the RAFT model pre-trained on the Sintel dataset performed with near perfect precision for registration tasks on a set of ten contextually unrelated time-lapse observations containing 250 frames each. Our approach is robust for elastically undistorted and translation displaced (x,y) microscopic time-lapse observations and was tested on multiple samples with varying cell density, obtained using different devices. The approach only performed well for registration and not for tracking of the individual image components like cells and contaminants. We provide an open-source command-line application that corrects for stage drift and jitter. [ABSTRACT FROM AUTHOR]

High-speed turbulence induces significant aero-optical effects that severely disrupt the functionality of imaging systems of hypersonic vehicles. In this study, the aero-optical correction of various jet cooling modes is investigated using a Terminal High Altitude Area Defense (THAAD)-like seeker model and the imaging impact of high-speed flow field and flow control on the optical window is analyzed by the Delayed Detached Eddy Simulation (DDES) method. The findings reveal that a jet mode parallel to the window exhibits better cooling effectiveness compared to a perpendicular jet mode along the body axis; however, it introduces additional wavefront distortion, leading to degraded imaging quality. Although micro-vortex generators (MVGs) can reduce density fluctuations near the window from a refractive index perspective, they do not effectively mitigate wavefront distortion or improve window cooling efficiency. Finally, incorporating suction control, a comprehensive flow control solution, significantly improves the flow field structure near the window, resulting in a more uniform temperature distribution and reduced wavefront distortion. Applying this flow control method results in a 14.7% reduction in wavefront distortion at 3 Ma and an approximately 20% maximum value reduction at 5 Ma. This study proposes a novel and comprehensive flow control method to effectively mitigate the aero-optical effect in hypersonic flows, providing a new avenue for subsequent researchers in this field. [ABSTRACT FROM AUTHOR]

This research proposes a sensor for tracking the motion of a human head via optical tactile sensing. It implements the use of a fibrescope a non-metal alternative to a webcam. Previous works have included robotics grippers to mimic the sensory features of human skin, that used monochrome cameras and depth cameras. Tactile sensing has shown advantages in feedback-based interactions between robots and their environment. The methodology in this paper is utilised to track motion of objects in physical contact with these sensors to replace external camera based motion capture systems. Our immediate application is related to detection of human head motion during radiotherapy procedures. The motion was analysed in two degrees of freedom, respective to the tactile sensor (translational in z-axis, and rotational around y-axis), to produce repeatable and accurate results. The movements were stimulated by a robot arm, which also provided ground truth values from its end- effector. The fibrescope was implemented to ensure the device's compatibility with electromagnetic waves. The cameras and the ground truth values were time synchronised using robotics operating systems tools. Image processing methods were compared between grayscale and binary image sequences, followed by motion tracking estimation using deterministic approaches. These included Lukas-Kanade Optical Flow and Simple Blob Detection, by OpenCV. The results showed that the grayscale image processing along with the Lukas- Kanade algorithm for motion tracking can produce better tracking abilities, although further exploration to improve the accuracy is still required. [ABSTRACT FROM AUTHOR]

Event-based cameras, also known as biologically inspired visual sensors, are capable of capturing real-time scene changes efficiently. Unlike traditional frame-based cameras, event cameras solely report triggered pixel-level brightness changes which are referred to as events. Event-based cameras show many advantages such as high temporal resolution, low latency, and high dynamic range, making them very attractive in robotics and computer vision, especially in challenging scenarios that are too demanding for traditional cameras. In this paper, we provide a comprehensive overview of the emerging field of event-based vision, focusing on the operation principle, sampling mechanisms, and algorithms that take advantage of their superior features. We also delve into the various tasks for which event cameras are utilized, such as object tracking, optical flow estimation, 3D reconstruction, SLAM, image reconstruction, and recognition. Additionally, we highlight the challenges and future opportunities for event cameras, seeking a more efficient way for machines to perceive and interact with the world. [ABSTRACT FROM AUTHOR]

Edge computing is essential to handle increasing data volumes and processing capacities. It provides real-time and secure data processing near data sources, like smart devices, alleviating cloud computing energy use, and saving network bandwidth. Specialized accelerators, like GPUs and FPGAs, are vital for low-latency edge computing but the requirements to customized code for different hardware and vendors suppose important compatibility issues. This paper evaluates the potential of SYCL in addressing code portability issues encountered in edge computing. We employed the Polybench suite to compare various SYCL implementations, specifically DPC++ and AdaptiveCpp, with the native solution, CUDA. The disparity between SYCL implementations was negligible, at just 5%. Furthermore, we evaluated SYCL in the context of specific edge computing applications such as video processing using three different optical flow algorithms. The results revealed a slight performance gap of 3% when transitioning from CUDA to SYCL. Upon evaluating energy consumption, the observed difference ranged from ± 10 % , depending on the application utilized. These gaps are the price one may need to pay when achieving the ability to successfully run the same code on two distinct edge boards. These findings underscore SYCL's capacity to increase productivity in terms of development costs and facilitate IoT deployment without being locked into a particular platform or manufacturer. [ABSTRACT FROM AUTHOR]

Parallax processing and structure preservation have long been important and challenging tasks in image stitching. In this paper, an image stitching method based on sliding camera to eliminate perspective deformation and asymmetric optical flow to solve parallax is proposed. By maintaining the viewpoint of two input images in the mosaic non-overlapping area and creating a virtual camera by interpolation in the overlapping area, the viewpoint is gradually transformed from one to another so as to complete the smooth transition of the two image viewpoints and reduce perspective deformation. Two coarsely aligned warped images are generated with the help of a global projection plane. After that, the optical flow propagation and gradient descent method are used to quickly calculate the bidirectional asymmetric optical flow between the two warped images, and the optical-flow-based method is used to further align the two warped images to reduce parallax. In the image blending, the softmax function and registration error are used to adjust the width of the blending area, further eliminating ghosting and reducing parallax. Finally, by comparing our method with APAP, AANAP, SPHP, SPW, TFT, and REW, it has been proven that our method can not only effectively solve perspective deformation, but also gives more natural transitions between images. At the same time, our method can robustly reduce local misalignment in various scenarios, with higher structural similarity index. A scoring method combining subjective and objective evaluations of perspective deformation, local alignment and runtime is defined and used to rate all methods, where our method ranks first. [ABSTRACT FROM AUTHOR]

The consecutive submesoscale sea surface processes observed by an unmanned aerial vehicle (UAV) were used to decompose into spatial waves and current features. For the image decomposition, the Fast and Adaptive Multidimensional Empirical Mode Decomposition (FA-MEMD) method was employed to disintegrate multicomponent signals identified in sea surface optical images into modulated signals characterized by their amplitudes and frequencies. These signals, referred to as Bidimensional Intrinsic Mode Functions (BIMFs), represent the inherent two-dimensional oscillatory patterns within sea surface optical data. The BIMFs, separated into seven modes and a residual component, were subsequently reconstructed based on the physical frequencies. A two-dimensional Fast Fourier Transform (2D FFT) for each high-frequency mode was used for surface wave analysis to illustrate the wave characteristics. Wavenumbers (Kx, Ky) ranging between 0.01–0.1 radm−1 and wave directions predominantly in the northeastward direction were identified from the spectral peak ranges. The Optical Flow (OF) algorithm was applied to the remaining consecutive low-frequency modes as the current signal under 0.1 Hz for surface current analysis and to estimate a current field with a 1 m spatial resolution. The accuracy of currents in the overall region was validated with in situ drifter measurements, showing an R-squared (R2) value of 0.80 and an average root-mean-square error (RMSE) of 0.03 ms−1. This study proposes a novel framework for analyzing individual sea surface dynamical processes acquired from high-resolution UAV imagery using a multidimensional signal decomposition method specialized in nonlinear and nonstationary data analysis. [ABSTRACT FROM AUTHOR]

Background: The placenta is a transient organ critical for fetal development. Disruptions of normal placental functions can impact health throughout an individual's entire life. Although being recognized by the NIH Human Placenta Project as an important organ, the placenta remains understudied, partly because of a lack of non-invasive tools for longitudinally evaluation for key aspects of placental functionalities. Objective: Our goal is to create a non-invasive preclinical imaging pipeline that can longitudinally probe murine placental health in vivo. We use advanced imaging processing schemes to establish functional biomarkers for non-invasive longitudinal evaluation of placental development. Methodology: We implement dynamic contrast enhancement magnetic resonance imaging (DCE-MRI) and analysis pipeline to quantify uterine contraction and placental perfusion dynamics. We use optic flow and time-frequency analysis to quantify and characterize contraction-related placental motion. Our novel imaging and analysis pipeline uses subcutaneous administration of gadolinium for steepest slope-based perfusion evaluation, enabling non-invasive longitudinal monitoring. Results: We demonstrate that the placenta exhibits spatially asymmetric contractile motion that develops from E14.5 to E17.5. Additionally, we see that placental perfusion, perfusion delivery rate, and substrate delivery all increase from E14.5 to E17.5, with the High Perfusion Chamber (HPC) leading the placental changes that occur from E14.5 to E17.5 Discussion: We advance the placental perfusion chamber paradigm with a novel, physiologically based threshold model for chamber localization and demonstrate spatially varying placental chambers using multiple functional metrics that assess mouse placental development and remodeling throughout gestation. Conclusion: Our pipeline enables the non-invasive, longitudinal assessment of multiple placenta functions from a single imaging session. Our pipeline serves as a key toolbox for advancing research in mouse models of placental disease and disorder. [ABSTRACT FROM AUTHOR]

Segregation of the solidification structure of an immiscible alloy is related to the convection induced by the density differences of the components under gravity during solidification. Here, the solidification of the Al–Bi alloy in both opposing and parallel directions to the gravity force was investigated in situ using radiography and optical flow techniques. A quantitative analysis was performed on the flow, solute, and constitutional undercooling fields at the interface. During upward solidification, a flat solute-enriched layer was observed, with the melt flowing toward the center from both sides of the interface. During downward solidification, the solute-enriched layer was deformed, and solute plumes were observed, with the melt flowing toward the sides from the center of the interface. A high solute concentration gradient enhances the local flow and solute transport. The elongation of the plumes was attributed not only to the convective diffusion of the solute but also to the dispersion of droplet clusters. The forces acting on the droplets were calculated, and it was found that the resultant force on the droplets increased the area of the solute-enriched layer. Convection within the bulk melt and the motion of the droplets have synergistic effects on the flow pattern and solute distribution. Distinct microstructures were observed in opposite directions of solidification owing to the differences in constitutional undercooling and flow instability. This work offers insights into the quantitative measurement and understanding of the multi-physics fields during the solidification process. [ABSTRACT FROM AUTHOR]

Current trends in network traffic management rely on the efficient control of individual flows. Software-defined networking popularized this notion. Per-flow management is perfectly viable in standard IP networks, in which packet processing is in the electric domain. However, optical networks provide more restrictions and constraints making per-flow traffic management difficult. One of the most important challenges is to reduce the concurrent number of flows present in the flow tables to make the switching process quicker. In this paper, we propose a mechanism to manage flow timeout values that uses idle timeout and hard timeout parameters. To calculate the appropriate values of the parameters, the mechanism analyzes the packet inter-arrival times. The algorithm also takes into account the current occupancy of the flow table. [ABSTRACT FROM AUTHOR]

Aiming at the problem that dynamic targets in indoor environments lead to low accuracy and large errors in the localization and position estimation of visual SLAM systems and the inability to build maps containing semantic information, a semantic visual SLAM algorithm based on the semantic segmentation network DeepLabV3+ and LK optical flow is proposed based on the ORB-SLAM2 system. First, the dynamic target feature points are detected and rejected based on the lightweight semantic segmentation network DeepLabV3+ and LK optical flow method. Second, the static environment occluded by the dynamic target is repaired using the time-weighted multi-frame fusion background repair technique. Lastly, the filtered static feature points are used for feature matching and position calculation. Meanwhile, the semantic labeling information of static objects obtained based on the lightweight semantic segmentation network DeepLabV3+ is fused with the static environment information after background repair to generate dense point cloud maps containing semantic information, and the semantic dense point cloud maps are transformed into semantic octree maps using the octree spatial segmentation data structure. The localization accuracy of the visual SLAM system and the construction of the semantic maps are verified using the widely used TUM RGB-D dataset and real scene data, respectively. The experimental results show that the proposed semantic visual SLAM algorithm can effectively reduce the influence of dynamic targets on the system, and compared with other advanced algorithms, such as DynaSLAM, it has the highest performance in indoor dynamic environments in terms of localization accuracy and time consumption. In addition, semantic maps can be constructed so that the robot can better understand and adapt to the indoor dynamic environment. [ABSTRACT FROM AUTHOR]

Macro-expression spotting is an important prior step in many dynamic facial expression analysis applications. It automatically detects the onset and offset image frames of a macro-expression in the video. The state-of-the-art methods of macro-expression spotting characterize the movement of facial muscle through explicit analysis of the optical flow map and have achieved promising results. However, optical flow map estimation and expression spotting in these methods are performed in two separate and successive stages. In this paper, we propose a new dual-branch network to achieve unified optimization for expression spotting and optical flow estimation tasks. The proposed dual-branch network implicitly learns optical flow during training and enriches the feature representation with motion information. During inference, we use only the encoder of the optical flow estimation network for motion feature extraction and integrate it with expression spotting into a one-stage framework. The proposed method eliminates the need to construct optical flow maps explicitly during inference and significantly reduces the computational cost. We also apply a consistency constraint on the global- and local-level semantic features of the clip to guide the model to focus on the category-consistent regions of the video. We evaluate the proposed methods extensively on two popular facial expression spotting datasets, CAS(ME) 2 and SAMM Long Videos. The experimental results show that compared with the state-of-the-art methods, the proposed method improves the F1-scores for MaE spotting by 5.81 % and 1.57 % on the CAS(ME) 2 and SAMM Long Videos datasets respectively. [ABSTRACT FROM AUTHOR]

Background: Being an image-based optical technique for full-field deformation measurements, the ultimate purpose of digital image correlation (DIC) is to realize accurate, precise and pixel-wise displacement/strain measurements in a full-automatic manner without users' inputs. Objective: In this work, we propose a task-optimized neural network, called RAFT-DIC, to achieve user-independent, accurate and pixel-wise displacement field measurements. Methods: RAFT-DIC is based on the state-of-the-art optical flow architecture: Recurrent All-Pairs Field Transforms (RAFT). We make two targeted improvements that fundamentally enhanced its measurement accuracy and generalization performance. Firstly, we remove all the down-sampling operations in the encode module to improve the perception of spatial information, and reduce the number of pyramid levels of the correlation layer to increase the small displacement accuracy. By building the correlation layer to compute the similarity of pixel pairs, and iteratively updating the displacement field through a recurrent unit, RAFT-DIC introduces the prior information of DIC measurement to guide the displacement estimation with high accuracy. Secondly, we develop a novel dataset generation method to synthesize customized speckle patterns and diverse displacement fields, which facilitate the construction of a robust and adaptable dataset to improve the network generalization. Results: Both simulated and real experimental results demonstrate that the accuracy of the proposed method is approximately an order of magnitude higher than pervious deep learning-based DIC (DL-DIC). Conclusions: The proposed RAFT-DIC shows higher accuracy as well as stronger practicality and cross-dataset generalization performance over existing DL-DIC methods, and is expected to be a new standard architecture for DL-DIC. [ABSTRACT FROM AUTHOR]

Aiming at the problem that the range profile of the current array lidar has a low resolution and contains few target details and little edge information, a super-resolution reconstruction method based on projection onto convex sets (POCS) combining the Lucas–Kanade (LK) optical flow method with a Gaussian pyramid was proposed. Firstly, the reference high-resolution range profile was obtained by the nearest neighbor interpolation of the single low-resolution range profile. Secondly, the LK optical flow method was introduced to achieve the motion estimation of low-resolution image sequences, and the Gaussian pyramid was used to perform multi-scale correction on the estimated vector, effectively improving the accuracy of motion estimation. On the basis of data consistency constraints, gradient constraints were introduced based on the distance value difference between the target edge and the background to enhance the reconstruction ability of the target edge. Finally, the residual between the estimated distance and the actual distance was calculated, and the high-resolution reference range profile was iteratively corrected by using the point spread function according to the residual. Bilinear interpolation, bicubic interpolation, POCS, POCS with adaptive correction threshold, and the proposed method were used to reconstruct the range profile of the datasets and the real datasets. The effectiveness of the proposed method was verified by the range profile reconstruction effect and objective evaluation index. The experimental results show that the index of the proposed method is improved compared to the interpolation method and the POCS method. In the redwood-3dscan dataset experiments, compared to the traditional POCS, the average gradient (AG) of the proposed method is increased by at least 8.04%, and the edge strength (ES) is increased by at least 4.84%. In the real data experiments, compared to the traditional POCS, the AG of the proposed method is increased by at least 5.85%, and the ES is increased by at least 7.01%, which proves that the proposed method can effectively improve the resolution of the reconstructed range map and the quality of the detail edges. [ABSTRACT FROM AUTHOR]

FlowDenoising is a new approach to noise reduction in biological volumes obtained with three-dimensional electron microscopy (3DEM). Its abilities to enhance the structural features stem from the fact that an anisotropic Gaussian filtering is steered according to the local structures. To this end, the Optical Flow (OF) among consecutive slices is estimated, which is the most computationally expensive step in this approach. In this article, a hybrid CPU/GPU implementation of FlowDenoising is introduced and evaluated. It exploits parallel computing by distributing the workload among multiple cores and takes advantage of the massive processing in GPUs to accelerate the OF estimation. The hybrid implementation provides remarkable speed-up factors and an important reduction of the processing time, which is particularly relevant for the denoising of huge volumes typically found in 3DEM. [ABSTRACT FROM AUTHOR]

The growing use of Unmanned Aerial Vehicles (UAVs) raises the need to improve their autonomous navigation capabilities. Visual odometry allows for dispensing positioning systems, such as GPS, especially on indoor flights. This paper reports an effort toward UAV autonomous navigation by proposing a translational velocity observer based on inertial and visual measurements for a quadrotor. The proposed observer complementarily fuses available measurements from different domains and is synthesized following the Immersion and Invariance observer design technique. A formal Lyapunov-based observer error convergence to zero is provided. The proposed observer algorithm is evaluated using numerical simulations in the Parrot Mambo Minidrone App from Simulink-Matlab. [ABSTRACT FROM AUTHOR]

Effective and accurate action recognition is essential to the intelligent breeding of the Jinnan cattle. However, there are still several challenges in the current Jinnan cattle action recognition. Traditional methods are based on manual characteristics and low recognition accuracy. This study is aimed at the efficient and accurate development of Jinnan cattle action recognition methods to overcome existing problems and support intelligent breeding. The acquired data from the previous methods contain a lot of noise, which will cause individual cattle to have excessive behaviors due to unsuitability. Concerning the high labor costs, low efficiency, and low model accuracy of the above approaches, this study developed a bottleneck attention-enhanced two-stream (BATS) Jinnan cattle action recognition method. It primarily comprises a Spatial Stream Subnetwork, a Temporal Stream Subnetwork, and a Bottleneck Attention Module. It can capture the spatial-channel dependencies in RGB and optical flow two branches respectively, so as to extract richer and more robust features. Finally, the decision of the two branches can be fused to gain improved cattle action recognition performance. Compared with the traditional methods, the model proposed in this study has achieved state-of-the-art recognition performance, and the accuracy of motion recognition was 96.53%, which was 4.60% higher than other models. This method significantly improves the efficiency and accuracy of behavior recognition and provides an important research foundation and direction for the development of higher-level behavior analysis models in the future development of smart animal husbandry. [ABSTRACT FROM AUTHOR]

Compressive sensing provides a promising sampling paradigm for video acquisition for resource‐limited sensor applications. However, the reconstruction of original video signals from sub‐sampled measurements is still a great challenge. To exploit the temporal redundancies within videos during the recovery, previous works tend to perform alignment on initial reconstructions, which are too coarse to provide accurate motion estimations. To solve this problem, the authors propose a novel reconstruction network, named TSRN, for compressive video sensing. Specifically, the authors utilise a number of stacked temporal shift reconstruction blocks (TSRBs) to enhance the initial reconstruction progressively. Each TSRB could learn the temporal structures by exchanging information with last and next time step, and no additional computations is imposed on the network compared to regular 2D convolutions due to the high efficiency of temporal shift operations. After the enhancement, a bidirectional alignment module to build accurate temporal dependencies directly with the help of optical flows is employed. Different from previous methods that only extract supplementary information from the key frames, the proposed alignment module can receive temporal information from the whole video sequence via bidirectional propagations, thus yielding better performance. Experimental results verify the superiority of the proposed method over other state‐of‐the‐art approaches quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

In the next generation of smart cities, Unmanned Aerial Vehicles (UAV) also known as drones are playing a vital role in many advanced applications such as power transmission line inspection, transportation, aerospace and surveillance etc. Due to the excessively high and wide transmission tower heights, the conventional methods of power line inspection are generally ineffective. This manuscript’s primary focus is the development of an autonomous UAV/ quadcopter that can hover over transmission towers and capture photographs and videos by flying along pre-planned routes. Quadcopters have a distinct feature that distinguishes them with the existing aerial vehicles and have a vital role in wide range of applications such as live monitoring of traffic and crowded areas, remote locations, delivery and inspection. This manuscript also explains about the advanced sensors & components such as Global Navigation Satellite System (GNSS), optical flow sensor and Here Link etc. required for fabrication of an autonomous quadcopter for power transmission line applications. The fabricated quadcopter includes a light weight S-500 frame equipped with intelligent controller such as Pixhawk cube orange (2.1) and NVIDIA nano board for receiving and analyzing the data from the onboard sensors and camera based on pre-determined criteria. The proposed approach increases effectiveness and accuracy, has a promising future for intelligent insulator detection and inspection which is a valuable addition to power networks. The suggested deep learning technique has a detection speed of 51.8 frames/sec and a detection accuracy of up to 90.31 percent. The suggested DL algorithm has a promising future in terms of intelligent insulator inspection in power grids. [ABSTRACT FROM AUTHOR]

Rationale. Object tracking has significance in many applications ranging from control of unmanned vehicles to autonomous monitoring of specific situations and events, especially when providing safety for patients with certain adverse conditions such as epileptic seizures. Conventional tracking methods face many challenges, such as the need for dedicated attached devices or tags, influence by high image noise, complex object movements, and intensive computational requirements. We have developed earlier computationally efficient algorithms for global optical flow reconstruction of group velocities that provide means for convulsive seizure detection and have potential applications in fall and apnea detection. Here, we address the challenge of using the same calculated group velocities for object tracking in parallel. Methods. We propose a novel optical flow-based method for object tracking. It utilizes real-time image sequences from the camera and directly reconstructs global motion-group parameters of the content. These parameters can steer a rectangular region of interest surrounding the moving object to follow the target. The method successfully applies to multi-spectral data, further improving its effectiveness. Besides serving as a modular extension to clinical alerting applications, the novel technique, compared with other available approaches, may provide real-time computational advantages as well as improved stability to noisy inputs. Results. Experimental results on simulated tests and complex real-world data demonstrate the method's capabilities. The proposed optical flow reconstruction can provide accurate, robust, and faster results compared to current state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Existing homography and optical flow methods are erroneous in challenging scenes, such as fog, rain, night, and snow because the basic assumptions such as brightness and gradient constancy are broken. To address this issue, we present an unsupervised learning approach that fuses gyroscope into homography and optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Second, we design a self-guided fusion module (SGF) to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. Meanwhile, we propose a homography decoder module (HD) to combine gyro field and intermediate results of SGF to produce the homography. To the best of our knowledge, this is the first deep learning framework that fuses gyroscope data and image content for both deep homography and optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-the-art methods in both regular and challenging scenes. The code and dataset are available at https://github.com/lhaippp/GyroFlowPlus. [ABSTRACT FROM AUTHOR]

Copyright of Biomédica: Revista del Instituto Nacional de Salud is the property of Instituto Nacional de Salud of Colombia and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The remarkable damage-tolerance of enamel has been attributed to its hierarchical microstructure and the organized bands of decussated rods. A thorough characterization of the microscale rod evolution within the enamel is needed to elucidate this complex structure. While prior efforts in this area have made use of single particle tracking to track a single rod evolution to various degrees of success, such a process can be both computationally and labor intensive, limited to the evolution path of a single rod, and is therefore prone to error from potentially tracking outliers. Particle image velocimetry (PIV) is a well-established algorithm to derive field information from image sequences for processes that are time-dependent, such as fluid flows and structural deformation. In this work, we demonstrate the use of PIV in extracting the full-field microstructural distribution of rods within the enamel. Enamel samples from a wild African lion were analyzed using high-energy synchrotron X-ray micro-tomography. Results from the PIV analysis provide sufficient full-field information to reconstruct the growth of individual rods that can potentially enable rapid analysis of complex microstructures from high resolution synchrotron datasets. Such information can serve as a template for designing damage-tolerant bioinspired structures for advanced manufacturing. Thorough characterization and analysis of biological microstructures (viz. dental enamel) allows us to understand the basis of their excellent mechanical properties. Prior efforts have successfully replicated these microstructures via single particle tracking, but the process is computationally and labor intensive. In this work, optical flow imaging algorithms were used to extract full-field microstructural distribution of enamel rods from synchrotron X-ray computed tomography datasets, and a field method was used to reconstruct the growth of individual rods. Such high throughput information allows for the rapid production/prototyping and advanced manufacturing of damage-tolerant bioinspired structures for specific engineering applications. Furthermore, the algorithms used herein are freely available and open source to broaden the availability of the proposed workflow to the general scientific community. [Display omitted] [ABSTRACT FROM AUTHOR]

This paper presents a method for stereo visual odometry and mapping that integrates VINS-Fusion-based visual odometry estimation with deep learning techniques for camera pose tracking and stereo image matching. Traditional approaches in the VINS-Fusion relied on classical methods for feature extraction and matching, which often resulted in inaccuracies in triangulation-based 3D position estimation. These inaccuracies could be mitigated by incorporating IMU-based position estimation, which yielded more accurate odometry estimates compared to using stereo camera only in three-dimensional space. Consequently, the original VINS-stereo algorithm necessitated a tightly-coupled integration of IMU sensor measurements with estimated visual odometry.To address these challenges, our work proposes replacing the traditional feature extraction method used in VINS-Fusion, the Shi-Tomasi (Good Features to Track) technique, with feature extraction via the SuperPoint deep network. This approach has demonstrated promising experimental results. Additionally, we have applied deep learning models to the matching of feature points that project the same three-dimensional point to pixel coordinates in different images. Instead of using the KLT optical flow algorithm previously employed by VINS-Fusion, our proposed method utilizes SuperGlue, a deep graph neural network for graph matching, to improve image tracking and stereo image matching performance. The performance of the proposed algorithm is evaluated using the publicly available EuRoC dataset, providing a comparison with existing algorithms. [ABSTRACT FROM AUTHOR]

Traditional crowd counting (optical flow or feature matching) techniques have been upgraded to deep learning (DL) models due to their lack of automatic feature extraction and low-precision outcomes. Most of these models were tested on surveillance scene crowd datasets captured by stationary shooting equipment. It is very challenging to perform people counting from the videos shot with a head-mounted moving camera; this is mainly due to mixing the temporal information of the moving crowd with the induced camera motion. This study proposed a transfer learning-based PeopleNet model to tackle this significant problem. For this, we have made some significant changes to the standard VGG16 model, by disabling top convolutional blocks and replacing its standard fully connected layers with some new fully connected and dense layers. The strong transfer learning capability of the VGG16 network yields in-depth insights of the PeopleNet into the good quality of density maps resulting in highly accurate crowd estimation. The performance of the proposed model has been tested over a self-generated image database prepared from moving camera video clips, as there is no public and benchmark dataset for this work. The proposed framework has given promising results on various crowd categories such as dense, sparse, average, etc. To ensure versatility, we have done self and cross-evaluation on various crowd counting models and datasets, which proves the importance of the PeopleNet model in adverse defense of society. [ABSTRACT FROM AUTHOR]

Dangerous human behavior in the driving sense may cause traffic accidents and even cause economic losses and casualties. Accurate identification of dangerous human behavior can prevent potential risks. To solve the problem of difficulty retaining the temporal characteristics of the existing data, this paper proposes a human behavior recognition model based on utilized optical flow and hybrid deep learning model-based 3D CNN-LSTM in stacked autoencoder and uses the abnormal behavior of humans in real traffic scenes to verify the proposed model. This model was tested using HMDB51 datasets and JAAD dataset and compared with the recent related works. For a quantitative test, the HMDB51 dataset was used to train and test models for human behavior. Experimental results show that the proposed model achieved good accuracy of about 86.86%, which outperforms recent works. For qualitative analysis, we depend on the initial annotations of walking movements in the JAAD dataset to streamline the annotating process to identify transitions, where we take into consideration flow direction, if it is cross-vehicle motion (to be dangerous) or if it is parallel to vehicle motion (to be of no danger). The results show that the model can effectively identify dangerous behaviors of humans and then test on the moving vehicle scene. [ABSTRACT FROM AUTHOR]

Background: Threat and individual differences in threat-processing bias perception of stimuli in the environment. Yet, their effect on perception of one's own (body-based) self-motion in space is unknown. Here, we tested the effects of threat on self-motion perception using a multisensory motion simulator with concurrent threatening or neutral auditory stimuli. Results: Strikingly, threat had opposite effects on vestibular and visual self-motion perception, leading to overestimation of vestibular, but underestimation of visual self-motions. Trait anxiety tended to be associated with an enhanced effect of threat on estimates of self-motion for both modalities. Conclusions: Enhanced vestibular perception under threat might stem from shared neural substrates with emotional processing, whereas diminished visual self-motion perception may indicate that a threatening stimulus diverts attention away from optic flow integration. Thus, threat induces modality-specific biases in everyday experiences of self-motion. [ABSTRACT FROM AUTHOR]

The sky may seem big enough for two flying vehicles to collide, but the facts show that mid-air collisions still occur occasionally and are a significant concern. Pilots learn manual tactics to avoid collisions, such as see-and-avoid, but these rules have limitations. Automated solutions have reduced collisions, but these technologies are not mandatory in all countries or airspaces, and they are expensive. These problems have prompted researchers to continue the search for low-cost solutions. One attractive solution is to use computer vision to detect obstacles in the air due to its reduced cost and weight. A well-trained deep learning solution is appealing because object detection is fast in most cases, but it relies entirely on the training data set. The algorithm chosen for this study is optical flow. The optical flow vectors can help us to separate the motion caused by camera motion from the motion caused by incoming objects without relying on training data. This paper describes the development of an optical flow-based airborne obstacle detection algorithm to avoid mid-air collisions. The approach uses the visual information from a monocular camera and detects the obstacles using morphological filters, optical flow, focus of expansion, and a data clustering algorithm. The proposal was evaluated using realistic vision data obtained with a self-developed simulator. The simulator provides different environments, trajectories, and altitudes of flying objects. The results showed that the optical flow-based algorithm detected all incoming obstacles along their trajectories in the experiments. The results showed an F-score greater than 75% and a good balance between precision and recall. [ABSTRACT FROM AUTHOR]

Accurate estimation of global winds is crucial for various scientific and practical applications, such as global chemical transport modeling and numerical weather prediction. One valuable source of wind measurements is atmospheric motion vectors (AMVs), which play a vital role in the global observing system and numerical weather prediction models. However, errors in AMV retrievals need to be addressed before their assimilation into data assimilation systems, as they can affect the accuracy of outputs. An assessment of the bias and uncertainty in passive-sensor AMVs can be done by comparing them with information from independent sources such as active-sensor winds. In this paper, we examine the benefit and performance of a colocation scheme using independent and sparse lidar wind observations as a dependent variable in a supervised machine learning model. We demonstrate the feasibility and performance of this approach in an observing system simulation experiment (OSSE) framework, with reference geophysical state data obtained from high-resolution Weather Research and Forecasting (WRF) model simulations of three different weather events. Lidar wind data are typically available in only one direction, and our study demonstrates that this single component of wind in high-precision active-sensor data can be leveraged (via a machine learning algorithm to model the conditional mean) to reduce the bias in the passive-sensor winds. Further, this active-sensor wind information can be leveraged through an algorithm that models the conditional quantiles to produce stable estimates of the prediction intervals, which are helpful in the design and application of error analysis, such as quality filters. [ABSTRACT FROM AUTHOR]

This study aims to generate visually useful imagery by preventing cropping while maintaining resolution and minimizing the degradation of stability and distortion to enhance the stability of a video for Augmented Reality applications. The focus is placed on conducting research that balances maintaining execution speed with performance improvements. By processing Inertial Measurement Unit (IMU) sensor data using the Versatile Quaternion-based Filter algorithm and optical flow, our research first applies motion compensation to frames of input video. To address cropping, PCA-flow-based video stabilization is then performed. Furthermore, to mitigate distortion occurring during the full-frame video creation process, neural rendering is applied, resulting in the output of stabilized frames. The anticipated effect of using an IMU sensor is the production of full-frame videos that maintain visual quality while increasing the stability of a video. Our technique contributes to correcting video shakes and has the advantage of generating visually useful imagery at low cost. Thus, we propose a novel hybrid full-frame video stabilization algorithm that produces full-frame videos after motion compensation with an IMU sensor. Evaluating our method against three metrics, the Stability score, Distortion value, and Cropping ratio, results indicated that stabilization was more effectively achieved with robustness to flow inaccuracy when effectively using an IMU sensor. In particular, among the evaluation outcomes, within the "Turn" category, our method exhibited an 18% enhancement in the Stability score and a 3% improvement in the Distortion value compared to the average results of previously proposed full-frame video stabilization-based methods, including PCA flow, neural rendering, and DIFRINT. [ABSTRACT FROM AUTHOR]

Traditional video frame interpolation methods based on deep convolutional neural networks face challenges in handling large motions. Their performance is limited by the fact that convolutional operations cannot directly integrate the rich temporal and spatial information of inter-frame pixels, and these methods rely heavily on additional inputs such as optical flow to model motion. To address this issue, we develop a novel framework for video frame interpolation that uses Transformer to efficiently model the long-range similarity of inter-frame pixels. Furthermore, to effectively aggregate spatio-temporal features, we design a novel attention mechanism divided into temporal attention and spatial attention. Specifically, spatial attention is used to aggregate intra-frame information, integrating both attention and convolution paradigms through the simple mapping approach. Temporal attention is used to model the similarity of pixels on the timeline. This design achieves parallel processing of these two types of information without extra computational cost, aggregating information in the space–time dimension. In addition, we introduce a context extraction network and multi-scale prediction frame synthesis network to further optimize the performance of the Transformer. Our method and state-of-the-art methods are extensively quantitatively and qualitatively experimented on various benchmark datasets. On the Vimeo90K and UCF101 datasets, our model achieves improvements of 0.09 dB and 0.01 dB in the PSNR metrics over UPR-Net-large, respectively. On the Vimeo90K dataset, our model outperforms FLAVR by 0.07 dB, with only 40.56% of its parameters. The qualitative results show that for complex and large-motion scenes, our method generates sharper and more realistic edges and details. [ABSTRACT FROM AUTHOR]

The goal of this study is to address the challenges associated with identifying and planning a mobile land robot's path to optimize its speed in a stationary environment. Our focus was on devising routes that navigate around obstacles in various spatial arrangements. To achieve this, we employed MATLAB R2023b for trajectory simulation and optimization. On-board data processing was conducted, while obstacle detection relied on the omnidirectional video processing system integrated into the robot. Odometry was facilitated by engine encoders and optical flow sensors. Additionally, an external video system was utilized to verify the experimental data pertaining to the robot's movement. Last but not least, the algorithms and hardware equipment used enabled the robot to go along the path at greater speeds. Limiting the amount of time and energy required to travel allowed us to avoid obstacles. [ABSTRACT FROM AUTHOR]

It is a challenging and meaningful task to carry out UAV-based livestock monitoring in high-altitude (more than 4500 m on average) and cold regions (annual average – 4 °C) on the Qinghai Tibet Plateau. The purpose of artificial intelligence (AI) is to execute automated tasks and to solve practical problems in actual applications by combining the software technology with the hardware carrier to create integrated advanced devices. Only in this way, the maximum value of AI could be realized. In this paper, a real-time tracking system with dynamic target tracking ability is proposed. It is developed based on the tracking-by-detection architecture using YOLOv7 and Deep SORT algorithms for target detection and tracking, respectively. In response to the problems encountered in the tracking process of complex and dense scenes, our work (1) Uses optical flow to compensate the Kalman filter, to solve the problem of mismatch between the target bounding box predicted by the Kalman filter (KF) and the input when the target detection in the current frame is complex, thereby improving the prediction accuracy; (2) Using a low confidence trajectory filtering method to reduce false positive trajectories generated by Deep SORT, thereby mitigating the impact of unreliable detection on target tracking. (3) A visual servo controller has been designed for the Unmanned Aerial Vehicle (UAV) to reduce the impact of rapid movement on tracking and ensure that the target is always within the field of view of the UAV camera, thereby achieving automatic tracking tasks. Finally, the system was tested using Tibetan yaks on the Qinghai Tibet Plateau as tracking targets, and the results showed that the system has real-time multi tracking ability and ideal visual servo effect in complex and dense scenes. [ABSTRACT FROM AUTHOR]

Motion object detection (MOD) with freely moving cameras is a challenging task in computer vision. To extract moving objects, most studies have focused on the difference in motion features between foreground and background, which works well for dynamic scenes with relatively regular movements and variations. However, abrupt illumination changes and occlusions often occur in real-world scenes, and the camera may also pan, tilt, rotate, and jitter, etc., resulting in local irregular variations and global discontinuities in motion features. Such complex and changing scenes bring great difficulty in detecting moving objects. To solve this problem, this paper proposes a new MOD method that effectively leverages local and global visual information for foreground/background segmentation. Specifically, on the global side, to support a wider range of camera motion, the relative inter-frame transformations are optimized to absolute transformations referenced to intermediate frames in a global form after enriching the inter-frame matching pairs. The global transformation is fine-tuned using the spatial transformer network (STN). On the local side, to address the problem of dynamic background scenes, foreground object detection is optimized by utilizing the pixel differences between the current frame and the local background model, as well as the consistency of local spatial variations. Then, the spatial information is combined using optical flow segmentation methods, enhancing the precision of the object information. The experimental results show that our method achieves a detection accuracy improvement of over 1.5% compared with the state-of-the-art methods on the datasets of CDNET2014, FBMS-59, and CBD. It demonstrates significant effectiveness in challenging scenarios such as shadows, abrupt changes in illumination, camera jitter, occlusion, and moving backgrounds. [ABSTRACT FROM AUTHOR]

With the development of simultaneous positioning and mapping technology in the field of automatic driving, the current simultaneous localization and mapping scheme is no longer limited to a single sensor and is developing in the direction of multi-sensor fusion to enhance the robustness and accuracy. In this study, a localization and mapping scheme named LVI-fusion based on multi-sensor fusion of camera, lidar and IMU is proposed. Different sensors have different data acquisition frequencies. To solve the problem of time inconsistency in heterogeneous sensor data tight coupling, the time alignment module is used to align the time stamp between the lidar, camera and IMU. The image segmentation algorithm is used to segment the dynamic target of the image and extract the static key points. At the same time, the optical flow tracking based on the static key points are carried out and a robust feature point depth recovery model is proposed to realize the robust estimation of feature point depth. Finally, lidar constraint factor, IMU pre-integral constraint factor and visual constraint factor together construct the error equation that is processed with a sliding window-based optimization module. Experimental results show that the proposed algorithm has competitive accuracy and robustness. [ABSTRACT FROM AUTHOR]