Search

A popular approach to estimate scene flow is to utilize point cloud data from various Lidar scans. However, there is little attention to learning 3D motion from camera images. Learning scene flow from a monocular camera remains a challenging task due to its ill-posedness as well as lack of annotated data. Self-supervised methods demonstrate learning scene flow estimation from unlabeled data, yet their accuracy lags behind (semi-)supervised methods. In this paper, we introduce a self-supervised monocular scene flow method that substantially improves the accuracy over the previous approaches. Based on RAFT, a state-of-the-art optical flow model, we design a new decoder to iteratively update 3D motion fields and disparity maps simultaneously. Furthermore, we propose an enhanced upsampling layer and a disparity initialization technique, which overall further improves accuracy up to 7.2%. Our method achieves state-of-the-art accuracy among all self-supervised monocular scene flow methods, improving accuracy by 34.2%. Our fine-tuned model outperforms the best previous semi-supervised method with 228 times faster runtime. Code will be publicly available to ensure reproducibility. [ABSTRACT FROM AUTHOR]

As the fields of aviation and aerospace optics continue to evolve, there is an increasing demand for enhanced detection capabilities in equipment. Nonetheless, in applications where both optical and mechanical constraints are stringent, the continuous expansion of optical aperture and focal length is impractical. Given the existing technological landscape, employing super-resolution algorithms to enhance the imaging capability of optical systems is both practical and highly relevant. This study capitalizes on using a 2D scanning galvanometer in optical systems to acquire micro-displacement information. Initially, an imaging model for optical systems equipped with a 2D scanning galvanometer was established, and the displacement vectors for both forward and sweep image motions were defined. On this foundation, we incorporated micro-displacement information that can induce high-frequency aliasing. Subsequently, the motion paths of the galvanometer were planned and modeled. To align image sequences with micro-displacement correlations, the Lucas–Kanade (L-K) optical flow method was employed with multi-layer pyramid iteration. Then, super-resolution reconstruction was performed using kernel regression techniques. Ultimately, we tested the algorithm on an aeronautical optoelectronic pod to evaluate its impact on optical resolution and imaging quality. Compared with the original images, the 16-frame image demonstrated a 39% improvement in optical resolution under laboratory conditions. Moreover, the algorithm exhibited satisfactory performance under both nighttime and daytime conditions, as well as during aerial tests. [ABSTRACT FROM AUTHOR]

For translationally moving objects with fixed cameras, such as robots and cars, blurring can often be more pronounced in objects that are closer to the camera. A depth-normalized, least-squares objective function is proposed for the simultaneous recovery of shape and motion parameters from optical flow, together with an efficient iterative optimization algorithm. Simulation and experiments demonstrate that for scenes with sufficient depth variation, our algorithm provides robust, statistically consistent estimates of shape and motion. c 2023 Society for Imaging Science and Technology [ABSTRACT FROM AUTHOR]

Studies on many dense correspondence tasks in the field of computer vision attempt to find spatially smooth results. A typical way to solve these problems is by smoothing the matching costs using edge-preserving filters. However, local filters generate locally optimal results, in that they only take the costs over a small support window into account, and non-local filters based on a minimum spanning tree (MST) tend to overuse the piece-wise constant assumption. In this paper, we propose a linear time non-local cost aggregation method based on two complementary spatial tree structures. The geodesic distances in both the spatial and intensity spaces along the tree structures are used to evaluate the similarity of pixels, and the final aggregated cost is the sum of the outputs from these two trees. The filtering output of a pixel on each tree can be obtained by recursively aggregating the costs along eight sub-trees with linear time complexity. The only difference between the filtering procedures on these two spatial tree structures is the order of the filtering. Experimental results in optical flow estimation and stereo matching on the Middlebury and KITTI datasets demonstrate the effectiveness and efficiency of our method. It turns out that our method outperforms typical non-local filters based on the MST in cost aggregation. Moreover, a comparison of handcrafted features and deep features learned by convolutional neural networks (CNNs) in calculating the matching cost is also provided. The code will be available soon. [ABSTRACT FROM AUTHOR]

Micro-expressions are a type of real emotional expression, which are unconscious and difficult to hide. Identifying these expressions has great potential applications in areas such as civil aviation security, criminal interrogation, and clinical medicine. However, because of their characteristics such as short duration, low intensity, and sparse action units, this makes micro-expression spotting difficult. To address this problem and inspired by object detection methods, we propose a VoVNet-based micro-expression spotting model, driven by multi-scale features. Firstly, VoVNet is used to achieve the extraction and reuse of different scale perceptual field features to improve the feature extraction capability. Secondly, multi-scale features are extracted and fused using the Feature Pyramid Network module, incorporating optical flow features, and by realizing the interactive fusion of fine-grained feature information and semantic feature information. Finally, the model is trained and optimized on CAS(ME)2 and SAMM Long Video. The experimental results show that the F1 score of the proposed model is improved by 0.1963 and 0.2441 on the two datasets compared with the baseline method, which outperforms the most popular spotting methods. [ABSTRACT FROM AUTHOR]

Video inpainting aims to complete the missing regions with content that is consistent both spatially and temporally. How to effectively utilize the spatio-temporal information in videos is critical for video inpainting. Recent advances in video inpainting methods combine both optical flow and transformers to capture spatio-temporal information. However, these methods fail to fully explore the potential of optical flow within the transformer. Furthermore, the designed transformer block cannot effectively integrate spatio-temporal information across frames. To address the above problems, we propose a novel video inpainting model, named Flow-Guided Spatial Temporal Transformer (FSTT), which effectively establishes correspondences between missing regions and valid regions in both spatial and temporal dimensions under the guidance of completed optical flow. Specifically, a Flow-Guided Fusion Feed-Forward module is developed to enhance features with the assistance of optical flow, mitigating the inaccuracies caused by hole pixels when performing MHSA. Additionally, a decomposed spatio-temporal MHSA module is proposed to effectively capture spatio-temporal dependencies in videos. To improve the efficiency of the model, a Global–Local Temporal MHSA module is further designed based on the window partition strategy. Extensive quantitative and qualitative experiments on the DAVIS and YouTube-VOS datasets demonstrate the superiority of our proposed method. [ABSTRACT FROM AUTHOR]

Introduction: Sensory inference and top-down predictive processing, reflected in human neural activity, play a critical role in higher-order cognitive processes, such as language comprehension. However, the neurobiological bases of predictive processing in higher-order cognitive processes are not well-understood. Methods: This study used electroencephalography (EEG) to track participants' cortical dynamics in response to Austrian Sign Language and reversed sign language videos, measuring neural coherence to optical flow in the visual signal. We then used machine learning to assess entropy-based relevance of specific frequencies and regions of interest to brain state classification accuracy. Results: EEG features highly relevant for classification were distributed across language processing-related regions in Deaf signers (frontal cortex and left hemisphere), while in non-signers such features were concentrated in visual and spatial processing regions. Discussion: The results highlight functional significance of predictive processing time windows for sign language comprehension and biological motion processing, and the role of long-term experience (learning) in minimizing prediction error. [ABSTRACT FROM AUTHOR]

In this paper, we propose a robust and integrated visual odometry framework exploiting the optical flow and feature point method that achieves faster pose estimate and considerable accuracy and robustness during the odometry process. Our method utilizes optical flow tracking to accelerate the feature point matching process. In the odometry, two visual odometry methods are used: global feature point method and local feature point method. When there is good optical flow tracking and enough key points optical flow tracking matching is successful, the local feature point method utilizes prior information from the optical flow to estimate relative pose transformation information. In cases where there is poor optical flow tracking and only a small number of key points successfully match, the feature point method with a filtering mechanism is used for posing estimation. By coupling and correlating the two aforementioned methods, this visual odometry greatly accelerates the computation time for relative pose estimation. It reduces the computation time of relative pose estimation to 40% of that of the ORB_SLAM3 front-end odometry, while ensuring that it is not too different from the ORB_SLAM3 front-end odometry in terms of accuracy and robustness. The effectiveness of this method was validated and analyzed using the EUROC dataset within the ORB_SLAM3 open-source framework. The experimental results serve as supporting evidence for the efficacy of the proposed approach. [ABSTRACT FROM AUTHOR]

Autonomous mobile robots (AMRs) require SLAM technology for positioning and mapping. Their accuracy and real-time performance are the keys to ensuring that the robot can safely and accurately complete the driving task. The visual SLAM systems based on feature points have high accuracy and robustness but poor real-time performance. A lightweight Visual Odometry (VO) based on Lucas–Kanade (LK) optical flow tracking is proposed. Firstly, a robust key point matching relationship between adjacent images is established by using a uniform motion model and a pyramid-based sparse optical flow tracking algorithm. Then, the grid-based motion statistics algorithm and the random sampling consensus algorithm are used to eliminate the mismatched points in turn. Finally, the proposed algorithm and the ORB-SLAM3 front-end are compared in a dataset to verify the effectiveness of the proposed algorithm. The results show that the proposed algorithm effectively improves the real-time performance of the system while ensuring its accuracy and robustness. [ABSTRACT FROM AUTHOR]

Traditional simultaneous localization and mapping (SLAM) performs well in a static environment; however, with the abrupt increase of dynamic points in dynamic environments, the algorithm is influenced by a lot of meaningless information, leading to low precision and poor robustness in pose estimation. To tackle this problem, a new visual SLAM algorithm of dynamic scenes named YG-SLAM is proposed, which creates an independent dynamic-object-detection thread and adds a dynamic-feature-point elimination step in the tracking thread. The YOLOv5 algorithm is introduced in the dynamic-object-detection thread for target recognition and deployed on the GPU to speed up image frame identification. The optic-flow approach employs an optic flow to monitor feature points and helps to remove the dynamic points in different dynamic objects based on the varying speeds of pixel movement. While combined with the antecedent information of object detection, the system can eliminate dynamic feature points under various conditions. Validation is conducted in both TUM and KITTI datasets, and the results illustrate that YG-SLAM can achieve a higher accuracy in dynamic indoor environments, with the maximum accuracy augmented from 0.277 m to 0.014 m. Meanwhile, YG-SLAM requires less processing time than other dynamic-scene SLAM algorithms, indicating its positioning priority in dynamic situations. [ABSTRACT FROM AUTHOR]

Precipitation nowcasting is mainly achieved by the radar echo extrapolation method. Due to the timing characteristics of radar echo extrapolation, convolutional recurrent neural networks (ConvRNNs) have been used to solve the task. Most ConvRNNs have been proven to perform far better than traditional optical flow methods, but they still have fatal problems. These models lack differentiation in the prediction of echoes of different intensities, which leads to the omission of responses from regions with high intensities. Moreover, because it is difficult for these models to capture long-term feature dependencies among multiple echo maps, the extrapolation effect declines sharply over time. This paper proposes an embedded multi-layer attention module (MLAM) to address the shortcomings of ConvRNNs. Specifically, an MLAM mainly enhances attention to critical regions in echo images and the processing of long-term spatiotemporal features through the interaction between input and memory features in the current moment. Comprehensive experiments were conducted on the radar dataset HKO-7 provided by the Hong Kong Observatory and the radar dataset HMB provided by the Hunan Meteorological Bureau. Experiments show that ConvRNNs embedded with MLAMs achieve more advanced results than standard ConvRNNs. [ABSTRACT FROM AUTHOR]

Due to its advantages of low latency, low power consumption, and high flexibility, FPGA-based acceleration technology has been more and more widely studied and applied in the field of computer vision in recent years. An FPGA-based feature extraction and tracking accelerator for real-time visual odometry (VO) and visual simultaneous localization and mapping (V-SLAM) is proposed, which can realize the complete acceleration processing capability of the image front-end. For the first time, we implement a hardware solution that combines features from accelerated segment test (FAST) feature points with Gunnar Farneback (GF) dense optical flow to achieve better feature tracking performance and provide more flexible technical route selection. In order to solve the scale invariance and rotation invariance lacking problems of FAST features, an efficient pyramid module with a five-layer thumbnail structure was designed and implemented. The accelerator was implemented on a modern Xilinx Zynq FPGA. The evaluation results showed that the accelerator could achieve stable tracking of features of violently shaking images and were consistent with the results from MATLAB code running on PCs. Compared to PC CPUs, which require seconds of processing time, the processing latency was greatly reduced to the order of milliseconds, making GF dense optical flow an efficient and practical technical solution on the edge side. [ABSTRACT FROM AUTHOR]

In a space-based infrared system, the enhancement of targets plays a crucial role in improving target detection capabilities. However, the existing methods of target enhancement face challenges when dealing with multi-target scenarios and a low signal-to-noise ratio (SNR). Furthermore, false enhancement poses a serious problem that affects the overall performance. To address these issues, a new enhancement method for a dim moving multi-target with strong robustness for false enhancement is proposed in this paper. Firstly, multi-target localization is applied by spatial–temporal filtering and connected component detection. Then, the motion vectors of each target are obtained using optical flow detection. Finally, the consecutive images are convoluted in 3D based on the convolution kernel guided by the motion vectors of the target. This process allows for the accumulation of the target energy. The experimental results demonstrate that this algorithm can adaptively enhance a multi-target and notably improve the SNR under low SNR conditions. Moreover, it exhibits outstanding performance compared to other algorithms and possesses strong robustness against false enhancement. [ABSTRACT FROM AUTHOR]

Visually guided action in humans occurs in part through the use of control laws, which are dynamical equations in which optical information modulates an actor's interaction with their environment. For example, humans locomote through the center of a corridor by equalizing the speed of optic flow across their left and right fields of view. This optic flow equalization control law relies on a crucial assumption: that the shape of the body relative to the eyes is laterally symmetrical. Humans engaging in tool use are often producing person-plus-object systems that are not laterally symmetrical, such as when they hold a tool, bag, or briefcase in one hand, or when they drive a vehicle. This experiment tests a new generalized control law for centered steering that accounts for asymmetries produced by external tool use. Participants held an asymmetrical bar and centered themselves within a virtual moving hallway while the speed of the virtual walls were systematically changed. The results demonstrate that humans engaging with an asymmetrical tool can (1) perceive the asymmetry of a person-plus-object system, (2) use that information to modulate the use of optic flow equalization control laws for centered steering, and (3) functionally incorporate the asymmetrical tool into their perception-action system to successfully navigate their environment. [ABSTRACT FROM AUTHOR]

Advances in artificial intelligence have led to a surge in digital forensics, resulting in numerous image manipulation and processing tools. Hackers and cybercriminals utilize these techniques to create counterfeit images and videos by placing perturbations on facial traits. We propose a novel defensive framework that employs temporal and spatially aware features to efficiently identify deepfakes. This paper utilizes the facial landmarks in the video to train a self-attenuated VGG16 neural model to obtain the spatial attributes. Further, we generate optical flow feature vectors that extract temporal characteristics from the spatial vector. Another necessity of deepfake detection systems is the need for cross-dataset generalization. We built a custom dataset comprising samples from FaceForensics, Celeb-DF, and Youtube videos. Experimental analysis shows that the system achieves a detection accuracy of 98.4%. We evaluate the robustness of our proposed framework under various adversarial settings, employing the Adversarial Robustness Toolbox, Foolbox, and CleverHans tools. The experimental evaluation shows that the proposed method can classify real and fake videos with an accuracy of 74.27% under diverse holistic conditions. An extensive empirical investigation to evaluate the cross-dataset generalization capacity of the proposed framework is also performed. [ABSTRACT FROM AUTHOR]

Doppler radars measure Doppler velocity within the [−VN, VN] range, where VN is the Nyquist velocity. Doppler velocities outside this range are "folded" within this interval. All Doppler "unfolding" techniques use the folded velocities themselves. In this work, we investigate the potential of using velocities derived from optical flow techniques applied to the radar reflectivity field for that purpose. The analysis of wind speed errors using six months of multi-Doppler wind retrievals showed that 99.9% of all points are characterized by errors smaller than 26 m s−1 below 5-km height, corresponding to a failure rate of less than 0.1% if optical flow winds were used to unfold Doppler velocities for VN = 26 m s−1. These errors largely increase above 5-km height, indicating that vertical continuity tests should be included to reduce failure rates at higher elevations. Following these results, we have developed the Two-step Optical Flow Unfolding (TOFU) technique, with the specific objective to accurately unfold Doppler velocities with VN = 26 m s−1. The TOFU performance was assessed using challenging case studies, comparisons with an advanced Doppler unfolding technique using higher Nyquist velocities, and 6 months of high VN (47.2 m s−1) data artificially folded to 26 m s−1. TOFU failure rates were found to be very low. Three main situations contributed to these errors: high low-level wind shear, elevated cloud layers associated with high winds, and radar data artifacts. Our recommendation is to use these unfolded winds as the first step of advanced Doppler unfolding techniques. Significance Statement: The potential of using optical flow winds operationally to accurately unfold Doppler velocities is demonstrated in this work. The operational significance is that the Nyquist velocity can confidently be reduced to 26 m s−1, allowing for extended first trip radar maximum range and reduced contamination from dual pulse repetition frequency artifacts. [ABSTRACT FROM AUTHOR]

Clean energy is a major trend. The importance of photovoltaic power generation is also growing. Photovoltaic power generation is mainly affected by the weather. It is full of uncertainties. Previous work has relied chiefly on historical photovoltaics data for time series forecasts. However, unforeseen weather conditions can sometimes skew. Consequently, a spatial‐temporal‐meteorological‐long short‐term memory prediction model (STM‐LSTM) is proposed to compensate for the shortage of photovoltaic prediction models for uncertainties. This model can simultaneously process satellite image data, historical meteorological data, and historical power generation data. In this way, historical patterns and meteorological change information are extracted to improve the accuracy of photovoltaic prediction. STM‐LSTM processes raw satellite data to obtain cloud image data. It can extract cloud motion information using the dense optical flow method. First, the cloud images are processed to extract cloud position information. By adaptive attentive learning of images in different bands, a better representation for subsequent tasks can be obtained. Second, it is important to process historical meteorological data to learn meteorological change patterns. Last but not least, the historical photovoltaic power generation sequences are combined to obtain the final photovoltaic prediction results. After a series of experimental validation, the performance of the proposed STM‐LSTM model has a good improvement compared with the baseline model. [ABSTRACT FROM AUTHOR]

We devise a novel lightweight image matching architecture (LIMA), which is designed and optimized for particle image velocimetry (PIV). LIMA is a convolutional neural network (CNN) that performs symmetric image matching and employs an iterative residual refinement strategy, which allows us to optimize the total number of refinement steps to balance accuracy and computational efficiency. The network is trained on kinematic datasets with a loss function that penalizes larger gradients. We consider a six-level (LIMA-6) and a four-level (LIMA-4) version of the network and demonstrate that they are considerably leaner and faster than a state-of-the-art network designed for optical flow. LIMA-6 reconstructs the velocity field from synthetic and experimental PIV images with an accuracy comparable or superior both to existing CNNs as well as to state-of-the-art cross-correlation methods (i.e., a commercial implementation of WIDIM). Although less accurate, LIMA-4 allows a significant reduction of the computational costs with respect to any other method considered. All CNNs prove more robust than WIDIM with respect to particle loss and allow effective error reduction by increasing the particle seeding density. Thanks to reduced computational cost and memory requirement, we envision the deployment of LIMA on low-cost devices to provide affordable, real-time inference of the flow field during PIV measurements. [ABSTRACT FROM AUTHOR]

The most prominent features of fractal dimension analysis are high sensitivity to damage and instant determination of damage location. However, an intrinsic deficiency is that it is difficult to identify damage in a structure while suppressing noise simultaneously. In addition, the vibration measurement with low-spatial-resolution is also the main factor restricting the accurate damage location. In this paper, a novel damage detection method based on fractal dimension is proposed, in order to provide the high-spatial-resolution vibration information, the phase-based optical flow is introduced, which is a vision-based measurement method that can calculate the structural vibration with high-spatial-resolution. Then, to detect damage in a structure while suppressing noise, an advanced signal analysis method is proposed, with this method, the fractal dimension (FRAC) is mapped into Difference of Gaussian (DOG) multi-scale space, producing the FRAC-DOG to reveal damage-induced singularities in mode shapes. Finally, a data fusion technique is applied to the proposed method to provide a reasonable damage detection result that maximises the elimination of uncertainty. The numerical and experimental results demonstrate that the proposed method can detect damage with high-precision in noisy environments by comparing it with the existing damage detection methods. [ABSTRACT FROM AUTHOR]

The study explores the feasibility of optical flow‐based neural network from real‐world thermal aerial imagery. While traditional optical flow techniques have shown adequate performance, sparse techniques do not work well during cold‐soaked low‐contrast conditions, and dense algorithms are more accurate in low‐contrast conditions but suffer from the aperture problem in some scenes. On the other hand, optical flow from convolutional neural networks has demonstrated good performance with strong generalization from several synthetic public data set benchmarks. Ground truth was generated from real‐world thermal data estimated with traditional dense optical flow techniques. The state‐of‐the‐art Recurrent All‐Pairs Field Transform for the Optical Flow model was trained with both color synthetic data and the captured real‐world thermal data across various thermal contrast conditions. The results showed strong performance of the deep‐learning network against established sparse and dense optical flow techniques in various environments and weather conditions, at the cost of higher computational demand. [ABSTRACT FROM AUTHOR]

Micro-expressions typically reflect suppressed feelings and they can provide an accurate indication of the real feelings and motivations of a person. Accordingly, they have been used for clinical diagnosis, business negotiations, interrogations, and security research. However, it is difficult to detect micro-expressions because of their instantaneity and imperceptibility. Therefore, micro-expression recognition is challenging. So far, various micro-expression recognition algorithms have been proposed to improve micro-expression recognition performance, which LBP, optical flow method and deep learning methods have made good progress as mainstream algorithms. In this survey, we aim to provide a review of micro-expression recognition based on LBP, optical flow, and deep learning. We first introduce the current commonly used micro-expression datasets. Then the three mainstream classical methods, LBP, optical flow, and deep learning, are described and summarized respectively. The existing methods in the last 5 years are discussed in regard to datasets, pre-processing, evaluation metrics and accuracy. Finally, we explain the shortcomings and challenges of micro-expression recognition and propose the future directions. This can help researchers to more quickly understand the current status of research, problems and future development directions, and provide a reference point for further research. [ABSTRACT FROM AUTHOR]

Moving object detection methods, MOD, must solve complex situations found in video scenarios related to bootstrapping, illumination changes, bad weather, PTZ, intermittent objects, color camouflage, camera jittering, low camera frame rate, noisy videos, shadows, thermal videos, night videos, etc. Some of the most promising MOD methods are based on convolutional neural networks, which are among the best-ranked algorithms in the CDnet14 dataset. Therefore, this paper presents a novel CNN to detect moving objects called Two-Frame CNN, 2FraCNN. Unlike best-ranked algorithms in CDnet14, 2FraCNN is a non-transfer learning model and employs temporal information to estimate the motion of moving objects. The architecture of 2FraCNN is inspired by how the optical flow helps to estimate motion, and its core is the FlowNet architecture. 2FraCNN processes temporal information through the concatenation of two consecutive frames and an Encoder-Decoder architecture. 2FraCNN includes a novel training scheme to deal with unbalanced pixel classes background/foreground. 2FraCNN was evaluated using three different schemes: the CDnet14 benchmark for a state-of-the-art comparison; against human performance metric intervals for a realistic evaluation; and for practical purposes with the performance instrument PVADN that considers the quantitative criteria of performance, speed, auto-adaptability, documentation, and novelty. Findings show that 2FraCNN has a performance comparable to the top ten algorithms in CDnet14 and is one of the best twelve in the PVADN evaluation. Also, 2FraCNN demonstrated that can solve many video challenges categories with human-like performance, such as dynamic backgrounds, jittering, shadow, bad weather, and thermal cameras, among others. Based on these findings, it can be concluded that 2FraCNN is a robust algorithm solving different video conditions with competent performance regarding state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

This study investigates the influence of high-order twisting phases on polarization states and optical angular momentum of a vector light field with locally linear polarization and a hybrid state of polarization (SoP). The twisted vector optical field (TVOF) is experimentally generated based on the orthogonal polarization bases with high-order twisting phases. The initial SoP of a TVOF modulated by the high-order twisting phase possesses various symmetric distributions. The propagation properties of a high-order TVOF with locally linear polarization and hybrid SoP are explored, including the intensity compression, expansion, and conversion between the linear and circular polarization components. In particular, orbital angular momentum (OAM) appears in a high-order TVOF during propagation where no OAM exists in the initial field. The variation of OAM distribution in cross-section becomes more frequent with the increase of the twisting phase order. In addition, a non-symmetric OAM distribution appears in a non-isotropic TVOF, leading to the rotation of the beam around the propagation axis during propagation. The optical energy flow distribution of a high-order TVOF provides a more profound understanding of the propagation dynamics of high-order TVOF. These results provide a new approach for optical field manipulation in a high-order TVOF. [ABSTRACT FROM AUTHOR]

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Gait is a kind of biological behavioral characteristic which can be recognized from a distance and has gained an increased interest nowadays. Many existing silhouette-based methods ignore the instantaneous motion of gait, which is an important factor in distinguishing people with similar shapes. To further emphasize the instantaneous motion factor in human gait, the Gait Optical Flow Image (GOFI) is proposed to add the instantaneous motion direction and intensity to original gait silhouettes. The GOFI also helps to leverage both the temporal and spatial condition noises. Then, the gait features are extracted by the Gait Optical Flow Network (GOFN), which contains a Set Transition (ST) architecture to aggregate the image-level features to the set-level features and an Inherent Feature Pyramid (IFP) to exploit the multi-scaled partial features. The combined loss function is used to evaluate the similarity between different gaits. Experiments are conducted on two widely used gait datasets, the CASIA-B and the CASIA-C. The experiments show that the GOFN performs better on both datasets, which shows the effectiveness of the GOFN. [ABSTRACT FROM AUTHOR]

This paper proposes a new optical flow (OF) method for particle image velocimetry applications. The proposed method is based on the use of an a priori sparse knowledge of the flow. A particular insight is given to the optimization derivation based on an image-independent method. Two alternatives are introduced. The first one uses particle-tracking velocimetry estimates as subpixel information to describe the finest velocity scales. The expected true displacements related to the motion of the individual particles are used as anchors for the optimization procedure when the density of the particles is large enough. Alternatively, the second method solves the well-known median problem based on new image-independent functions in areas of low particle density. Studies have been carried out on synthetic images to characterize the error and analyze the impact of image parameters (particle density, particle size, or noise) on the methods. The new methods are compared with a reference method against synthetic data: two Lamb-Oseen vortex rings and a 3D Turbulent Homogeneous and Isotropic flow. The results show that the performances of the new method exceed those of the reference method in almost all tested cases, except for images with particles of relatively small size. It is notably shown that the new method is less dependent on the particle density and the noise embedded in the images than other OF estimators. [ABSTRACT FROM AUTHOR]

Hand gesture recognition (HGR) are the most significant tasks for communicating with the real-world environment. Recently, gesture recognition has been extensively utilized in diverse domains, including but not limited to virtual reality, augmented reality, health diagnosis, and robot interaction. On the other hand, accurate techniques typically utilize various modalities generated from RGB input sequences, such as optical flow which acquires the motion data in the images and videos. However, this approach impacts real-time performance due to its demand of substantial computational resources. This study aims to introduce a robust and effective approach to hand gesture recognition. We utilize two publicly available benchmark datasets. Initially, we performed preprocessing steps, including denoising, foreground extraction, and hand detection via associated component techniques. Next, hand segmentation is done to detect landmarks. Further, we utilized three multi-fused features, including geometric features, 3D point modeling and reconstruction, and angular point features. Finally, grey wolf optimization served useful features of artificial neural networks for hand gesture recognition. The experimental results have shown that the proposed HGR achieved significant recognition of 89.92% and 89.76% over IPN hand and Jester datasets, respectively. [ABSTRACT FROM AUTHOR]

Cinematic Shot Attribute Analysis aims to analyze the intrinsic attributes of movie shots, such as movement and scale. In previous methods, specialized architectures were designed for each specific task and relied on the use of optical flow maps. In this paper, we consider shot attribute analysis as a unified task of motion–static weight allocation, and propose a motion–static dual-path architecture for recognizing various shot attributes. In this architecture, we design a new action cue generation module for adapting the end-to-end training process instead of a pre-trained optical flow network; and, to address the issue of limited samples in movie shot datasets, we design a fixed-size adjustment strategy to enable the network to directly utilize pre-trained vision transformer models while adapting to shot data inputs at arbitrary sample rates. In addition, we quantitatively analyze the sensitivity of different shot attributes to motion and static features for the first time. Subsequent experimental results on two datasets, MovieShots and AVE, demonstrate that our proposed method outperforms all previous approaches without increasing computational cost. [ABSTRACT FROM AUTHOR]

Optical identification is often done with spatial or temporal visual pattern recognition and localization. Temporal pattern recognition, depending on the technology, involves a trade-off between communication frequency, range, and accurate tracking. We propose a solution with light-emitting beacons that improves this trade-off by exploiting fast event-based cameras and, for tracking, sparse neuromorphic optical flow computed with spiking neurons. The system is embedded in a simulated drone and evaluated in an asset monitoring use case. It is robust to relative movements and enables simultaneous communication with, and tracking of, multiple moving beacons. Finally, in a hardware lab prototype, we demonstrate for the first time beacon tracking performed simultaneously with state-of-the-art frequency communication in the kHz range.

Automated assessment of broiler chicken welfare poses particular problems due to the large numbers of birds involved and the variety of different welfare measures that have been proposed. Active (sustained, defect-free) walking is both a universally agreed measure of bird health and a behavior that can be recognized by existing technology. This makes active walking an ideal starting point for automated assessment of chicken welfare at both individual and flock level.

PurposeExisting automated seizure detection algorithms report sensitivities between 43% and 77% and specificities between 56% and 90%. The algorithms suffer from false alarms when applied to neonatal EEG because of the high degree of nurse handling and rhythmic patting used to soothe neonates. Computer vision technology that quantifies movement in real time could distinguish artifactual motion and improve automated neonatal seizure detection algorithms.MethodsThe authors used video EEG recordings from 43 neonates undergoing monitoring for seizures as part of the NEOLEV2 clinical trial. The Persyst neonatal automated seizure detection algorithm ran in real time during study EEG acquisitions. Computer vision algorithms were applied to extract detailed accounts of artifactual movement of the neonate or people near the neonate though dense optical flow estimation.ResultsUsing the methods mentioned above, 197 periods of patting activity were identified and quantified, of which 45 generated false-positive automated seizure detection events. A binary patting detection algorithm was trained with a subset of 470 event videos. This supervised detection algorithm was applied to a testing subset of 187 event videos with 8 false-positive events, which resulted in a 24% reduction in false-positive automated seizure detections and a 50% reduction in false-positive events caused by neonatal care patting, while maintaining 11 of 12 true-positive seizure detection events.ConclusionsThis work presents a novel approach to improving automated seizure detection algorithms used during neonatal video EEG monitoring. This artifact detection mechanism can improve the ability of a seizure detector algorithm to distinguish between artifact and true seizure activity.