Search

Artificial images and recordings are broad on the web via different media channels such as blogs, YouTube videos, etc. These manipulated and synthesized images tend to steal the identity of individuals and majorly contribute to establishing societal disruptions such as theft, political errors, social engineering, disinformation attacks and reputation fraud. These fake visual objects gradually came to be known as deep fakes. Different deep learning techniques are used to generate deepfake images which go unnoticed by human eyes. It is essential to develop a defense mechanism that can stop the common people from being manipulated and harnessed. The objective of this work is to develop an ensemble deep learning-based system that can differentiate between fake and real images. With the use of the recommended optical flow technique, a novel approach is proposed that extracts the apparent motion of image pixels which gives more accurate results compared to other state-of-the-art. FaceForensics + + dataset is used to test the extraction algorithms and ensemble model which fetched an accuracy of 86.02% for the DeepFake subset and 85.7% for the FaceSwap subset of the dataset. To the best knowledge, no one has completely used the ensemble model- OptiFake on the optical flow derived frames, highlighting a research gap in the field of deepfake detection. [ABSTRACT FROM AUTHOR]

Facial micro-expression (ME), consisting of uncontrollable muscle movements in faces, is an important clue for revealing real people's feelings. Due to the short duration and low intensity, the salient feature representation learning is the main challenge for robust facial ME recognition. To acquire the diverse and spatial relation representation, this paper proposes a simple and yet distinctive micro-expression recognition model based on multiscale convolutional fusion and multi-channel capsule network (MCFMCN). Firstly, the apex frame in a ME clip, located by computing the pixel difference between frames, is filtered by the optical flow transformation. Secondly, a multiscale fusion module is introduced to capture diverse ME related details. Then, to further explore the subtle spatial relations between parts in the ME faces, the multi-channel capsule network is designed to improve the feature representation performance of the traditional single channel capsule network. Finally, the entire ME recognition model is trained and verified on three benchmarks (CASMEII, SAMM, and SMIC) using the associated standard evaluation protocols: unweighted average recall rate (UAR) and unweighted F1 score (UF1). ME recognition experiments indicate that our method based on MCFMCN can improve the UAR (from 75.79% to 83.58%) and UF1(from79.37% to 87.06%) in comparison with the traditional capsule network. Extensive experimental results show the performance of proposed ME recognition is superior to that of works based on pervious single channel capsule network or other state-of-the-art CNN models, which validates the finding that combination of multi-scale analysis and multi-channel capsule network is feasible and effective to improve the ME recognition performance. [ABSTRACT FROM AUTHOR]

This paper proposes a method for an automatic detection of 3D-display-friendly scenes from video sequences. Manual selection of such scenes by a human user would be extremely time consuming and would require additional evaluation of the result on 3D display. The input videos can be intentionally captured or taken from other sources, such as films. First, the input video is analyzed and the camera trajectory is estimated. The optimal frame sequence that follows defined rules, based on optical attributes of the display, is then extracted. This ensures the best visual quality and viewing comfort. The following identification of a correct focusing distance is an important step to produce a sharp and artifact-free result on a 3D display. Two novel and equally efficient focus metrics for 3D displays are proposed and evaluated. Further scene enhancements are proposed to correct the unsuitably captured video. Multiple image analysis approaches used in the proposal are compared in terms of both quality and time performance. The proposal is experimentally evaluated on a state-of-the-art 3D display by Looking Glass Factory and is suitable even for other multi-view devices. The problem of optimal scene detection, which includes the input frames extraction, resampling, and focusing, was not addressed in any previous research. Separate stages of the proposal were compared with existing methods, but the results show that the proposed scheme is optimal and cannot be replaced by other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Since digital media has become increasingly popular, video processing has expanded in recent years. Video processing systems require high levels of processing, which is one of the challenges in this field. Various approaches, such as hardware upgrades, algorithmic optimizations, and removing unnecessary information, have been suggested to solve this problem. This study proposes a video saliency map based method that identifies the critical parts of the video and improves the system's overall performance. Using an image registration algorithm, the proposed method first removes the camera's motion. Subsequently, each video frame's color, edge, and gradient information are used to obtain a spatial saliency map. Combining spatial saliency with motion information derived from optical flow and color-based segmentation can produce a saliency map containing both motion and spatial data. A nonlinear function is suggested to properly combine the temporal and spatial saliency maps, which was optimized using a multi-objective genetic algorithm. The proposed saliency map method was added as a preprocessing step in several Human Action Recognition (HAR) systems based on deep learning, and its performance was evaluated. Furthermore, the proposed method was compared with similar methods based on saliency maps, and the superiority of the proposed method was confirmed. The results show that the proposed method can improve HAR efficiency by up to 6.5% relative to HAR methods with no preprocessing step and 3.9% compared to the HAR method containing a temporal saliency map. [ABSTRACT FROM AUTHOR]

The objective of this pilot study was to assess the reliability of superb microvascular ultrasound (SMI) for the measurement of the cerebrospinal fluid (CSF) flow within VPS systems as an indirect sign for shunt dysfunction. Asymptomatic hydrocephalus patients, with a VPS system implanted between 2017 and 2021, were prospectively enrolled in the study. Using SMI, the CSF flow within the proximal and distal catheters were analysed. Before and after pumping the shunt reservoir, intraabdominal free fluid, optical nerve sheath diameter (ONSD), and papilla diameter (PD) were evaluated and correlated with the amount of valve activation. Nineteen patients were included. A flow was detectable in 100% (N = 19) patients in the proximal and in 89.5% (N = 17) in the distal catheter. The distal catheter tip was detectable in 27.7% (N = 5) patients. Free intraabdominal fluid was initially detected in 21.4% (N = 4) patients and in 57.9% (N = 11) at the end of the examination (P = 0.049). ONSD was significantly lower after pump activation (4.4 ± 0.9 mm versus 4.1 ± 0.8 mm, P = 0.049). Both peak velocity and flow volume per second were higher in proximal compared to distal catheters (32.2 ± 45.2 versus 5.6 ± 3.7 cm/sec, P = 0.015; 16.6 ± 9.5 ml/sec versus 5.1 ± 4.0 ml/sec, P = 0.001, respectively). No correlation was found between the number of pump activations and the changes in ONSD (P = 0.975) or PD (P = 0.820). SMI appears to be a very promising non-invasive diagnostic tool to assess CSF flow within the VPS systems and therefore affirm their function. Furthermore, appearance of free intraperitoneal fluid followed by repeated compression of a shunt reservoir indicates an intact functioning shunt system. [ABSTRACT FROM AUTHOR]

Emulsification processes are often found in the process industry and their evaluation is crucial for product quality and safety. Numerous methods exist to analyze critical quality attributes (CQA) such as the droplet sizes and droplet size distribution (DSD) of an emulsification process. During the emulsification process, the optical process accessibility may be limited due to high disperse phase content of liquid-liquid systems. To overcome this challenge, a modular, optical measurement flow cell is presented to widen the application window of optical methods in emulsification processes. In this contribution, the channel geometry is subject of optimization to modify the flow characteristics and produce high optical quality. In terms of rapid prototyping, an iterative optimization procedure via SLA-3D printing was used to increase operability. The results demonstrated that the flow cell resulting from the optimization procedure provides a broad observation window for droplet detection. [ABSTRACT FROM AUTHOR]

Human action recognition from videos requires a sequence of frames to be analyzed involving high processing power and time. Moreover, subsequent frames in a video contain redundancy that does not contribute to extracting distinguishable features but rather consumes processing resources. To recognize action with lower processing overhead, researchers are focusing on key frames and have proposed different techniques based on inter-frame clustering, fix-frame interval, and inter-frame differences. Clustering and fixed interval approaches consider the predefined number of frames or fixed intervals, therefore, not equally effective for both faster and slower actions which differ in terms of frequency of change. Furthermore, all existing approaches consider pixels? Intensities only but neglect another equally important aspect of motion, i.e., direction. The three-channel optical flows are one of the motion representations that depict both the magnitude and the direction of movement between pairs of frames in the form of colors. Here we proposed a novel "KFF-algorithm" that processes a sequence of three-channel optical flows to extract key flow frames for action recognition. Being dynamic in terms of frame interval, it efficiently extracts key frames for both slower and faster actions by analyzing changes in direction, magnitude, and coverage of motion in subsequent frames. Furthermore, in comparison with other approaches, KFF-algorithm covers relatively extended motion patterns with least number of frames or frames with significant change only. Also, for the majority of classes, KFF-algorithm has achieved substantial per-class accuracy when evaluated with the 3D-ConvNet model over the UCF-101 benchmark dataset for human action recognition. [ABSTRACT FROM AUTHOR]

Convolutional neural network-based image style transfer models often suffer from temporal inconsistency when applied to video. Although several video style transfer models have been proposed to improve temporal consistency, they often trade off processing speed, perceptual style quality, and temporal consistency. In this work, we propose a novel approach for fast continuous patch-based arbitrary video style transfer that achieves high-quality transfer results while maintaining temporal coherence. Our approach begins with stylizing the first frame as a standalone single image using patch propagation within the content activation. Subsequent frames are computed based on the key insight that optical flow field evaluated from neighboring content activations provides meaningful information to preserve temporal coherence efficiently. To address the problems introduced from optical flow stage, we additionally incorporate a correction procedure as a post-process to ensure a high-quality stylized video. Finally, we demonstrate our method can transfer arbitrary styles on a set of examples and illustrate that our approach exhibits superior performance both qualitatively and quantitatively. [ABSTRACT FROM AUTHOR]

Scene-flow (SF) estimation is considered to be one of the most fundamental problems in scene understanding and autonomous control. The majority of the existing methods adopted for SF estimation suffer lack of robustness in some environments and cannot be easily applied for high-speed applications such as autonomous driving. Although some of the available methods are precise, they include high computational costs or require a GPU. The most serious challenge faced in SF estimation is its inability to strike a balance between speed, precision, robustness, and the computational costs. This paper, therefore, aims at proposing a novel sparse scene-flow (stereo-RSSF) method which is highly distinguished in terms of its faster speed, robustness, and precision using the following: stereo calibrated frames, sparse optical flow such as the LKT algorithm, a new inlier detection module based on spatial correlation analysis, epipolar geometry, and modified circular matching techniques. The comparisons made between stereo-RSSF and several advanced methods indicate that this sparse method has significantly higher accuracy than all the other state-of-the-art methods in the points it estimates. In this paper, the effects of each module and hyper-parameters of stereo-RSSF on the performance and running time are analyzed. Stereo-RSSF has also been evaluated on the KITTI test dataset, and the results have been independently verified by the reference group. The code for our implementation of stereo-RSSF is available at: https://github.com/salehierfan/Stereo-RSSF. [ABSTRACT FROM AUTHOR]

The rapid advancement of deep learning and computer vision technologies has given rise to a concerning class of deceptive media, commonly known as deepfakes. This paper addresses emerging trends in deepfakes, including the creation of hyper-realistic facial manipulations, the incorporation of synthesized human voices, and the addition of fabricated subtitles to video content. To effectively combat these multifaceted deepfake threats, we introduce an ensemble-based deepfake detection framework called the "D-Fence" layer. The D-Fence layer consists of two uni-modal classifiers designed to identify tampered facial and vocal elements, as well as two cross-modal classifiers for interactions between Video-Audio and Audio-Text domains to detect deepfakes across multiple modalities. To evaluate the effectiveness of our framework, we introduce two novel adversarial attacks: the "Bogus-in-the-middle" attack, which strategically inserts counterfeit video frames within authentic sequences, and the "Downsampling attack", designed to create deceptive audio. A comparative study of the D-Fence layer against various state-of-the-art multi-modal deepfake detection systems is conducted, demonstrating that our ensemble architecture outperforms existing classifiers. Under diverse adversarial conditions, our D-Fence layer achieves an impressive detection accuracy of 92%, showcasing its ability to detect deepfakes efficiently and reliably. [ABSTRACT FROM AUTHOR]

Geotechnical particle image velocimetry (GeoPIV), as a type of digital image correlation (DIC), represents the state-of-the-art methodology for non-contact full-field deformation measurement in geotechnical engineering. Yet, when applying GeoPIV on sand specimens with interests in grain level, the discontinuities detection at grain boundaries remains as a challenge for 2D GeoPIV applications. In order to facilitate the full-field measurement for microscopic study, a method is proposed in this study to realize 2D pixel-level motion calculation using supervised optical flow algorithm with deep networks. Using digital images acquired from direct shear testing, the performance of this approach is demonstrated and compared with the prevailing GeoPIV method. Two series of experiments using small and large displacement modes were conducted, respectively, to demonstrate the method's ability of revealing greater insights on soil behavior at grain level. To verify its accuracy, performance benchmarking of the approach was also conducted. Besides, a method was proposed to evaluate the errors in experimental images to ensure the accuracy and precision. It was demonstrated that the proposed method can achieve accurate pixel-level motion field calculation using images of common size and that the deformation discontinuities among particles can be clearly presented. [ABSTRACT FROM AUTHOR]

High spatial resolution and high accuracy estimation of 3D velocity fields are important for tomographic particle image velocimetry (Tomo-PIV), especially when measuring complex flow fields with delicate 3D structures. However, the widely used cross-correlation-based methods have limited spatial resolution, while the recently developed optical flow-based methods have low robustness and are sensitive to particle volume reconstruction errors. Therefore, 3D velocity estimation methods that simultaneously exhibit high resolution and robustness must be developed. In this study, we propose a novel velocity estimation method for Tomo-PIV measurement using the guided filter-based 3D hybrid variational optical flow (GF-HVOF) method to achieve high spatial resolution and highly accurate measurement of 3D flow field structure. First, we propose a novel L1-norm regularization term based on the Helmholtz decomposition theorem to preserve the divergence and vorticity of the fluid flow. Second, we propose a guided-filter-based constraint term using the result of the cross-correlation-based method as the guided flow field to improve the robustness of the optical flow method. Third, we propose a hybrid constraint term based on particle tracking velocimetry (PTV) method and a spatially weighted data term to reduce the effect of ghost particles and discrete errors generated during the reconstruction of particle volumes. The newly proposed hybrid method combines the advantages of optical-flow-based and cross-correlation-based methods and corrects the flow field using the PTV method. Velocity fields are estimated over synthetic and experimental particle volumes. The results show that the newly proposed GF-HVOF method achieves better performance and greater measurement accuracy than existing 3D fluid motion estimation methods. [ABSTRACT FROM AUTHOR]

Optic flow provides information on movement direction and speed during locomotion. Changing the relationship between optic flow and walking speed via training has been shown to influence subsequent distance and hill steepness estimations. Previous research has shown that experience with slow optic flow at a given walking speed was associated with increased effort and distance overestimation in comparison to experiencing with fast optic flow at the same walking speed. Here, we investigated whether exposure to different optic flow speeds relative to gait influences perceptions of leaping and jumping ability. Participants estimated their maximum leaping and jumping ability after exposure to either fast or moderate optic flow at the same walking speed. Those calibrated to fast optic flow estimated farther leaping and jumping abilities than those calibrated to moderate optic flow. Findings suggest that recalibration between optic flow and walking speed may specify an action boundary when calibrated or scaled to actions such as leaping, and possibly, the manipulation of optic flow speed has resulted in a change in the associated anticipated effort for walking a prescribed distance, which in turn influence one's perceived action capabilities for jumping and leaping. [ABSTRACT FROM AUTHOR]

Static and moving objects often occur in real-life videos. Most video object segmentation methods only focus on extracting and exploiting motion cues to perceive moving objects. Once faced with the frames of static objects, the moving object predictors may predict failed results caused by uncertain motion information, such as low-quality optical flow maps. Besides, different sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only consider either the RGB or RGB and optical flow. In this paper, we propose a novel adaptive multi-source predictor for zero-shot video object segmentation (ZVOS). In the static object predictor, the RGB source is converted to depth and static saliency sources, simultaneously. In the moving object predictor, we propose the multi-source fusion structure. First, the spatial importance of each source is highlighted with the help of the interoceptive spatial attention module (ISAM). Second, the motion-enhanced module (MEM) is designed to generate pure foreground motion attention for improving the representation of static and moving features in the decoder. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By using the ISAM, MEM and FPM, the multi-source features are effectively fused. In addition, we put forward an adaptive predictor fusion network (APF) to evaluate the quality of the optical flow map and fuse the predictions from the static object predictor and the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Experiments show that the proposed model outperforms the state-of-the-art methods on three challenging ZVOS benchmarks. And, the static object predictor precisely predicts a high-quality depth map and static saliency map at the same time. [ABSTRACT FROM AUTHOR]

The occlusion problem remains a crucial challenge in optical flow estimation (OFE). Despite the recent significant progress brought about by deep learning, most existing deep learning OFE methods still struggle to handle occlusions; in particular, those based on two frames cannot correctly handle occlusions because occluded regions have no visual correspondences. However, there is still hope in multi-frame settings, which can potentially mitigate the occlusion issue in OFE. Unfortunately, multi-frame OFE (MOFE) remains underexplored, and the limited studies on it are mainly specially designed for pyramid backbones or else obtain the aligned previous frame's features, such as correlation volume and optical flow, through time-consuming backward flow calculation or non-differentiable forward warping transformation. This study proposes an efficient MOFE framework named SplatFlow to address these shortcomings. SplatFlow introduces the differentiable splatting transformation to align the previous frame's motion feature and designs a Final-to-All embedding method to input the aligned motion feature into the current frame's estimation, thus remodeling the existing two-frame backbones. The proposed SplatFlow is efficient yet more accurate, as it can handle occlusions properly. Extensive experimental evaluations show that SplatFlow substantially outperforms all published methods on the KITTI2015 and Sintel benchmarks. Especially on the Sintel benchmark, SplatFlow achieves errors of 1.12 (clean pass) and 2.07 (final pass), with surprisingly significant 19.4% and 16.2% error reductions, respectively, from the previous best results submitted. The code for SplatFlow is available at https://github.com/wwsource/SplatFlow. [ABSTRACT FROM AUTHOR]

Purpose: Wearable ultrasound devices can be used to continuously monitor muscle activity. One possible application is to provide real-time feedback during physiotherapy, to show a patient whether an exercise is performed correctly. Algorithms which automatically analyze the data can be of importance to overcome the need for manual assessment and annotations and speed up evaluations especially when considering real-time video sequences. They even could be used to present feedback in an understandable manner to patients in a home-use scenario. The following work investigates three deep learning based segmentation approaches for abdominal muscles in ultrasound videos during a segmental stabilizing exercise. The segmentations are used to automatically classify the contraction state of the muscles. Methods: The first approach employs a simple 2D network, while the remaining two integrate the time information from the videos either via additional tracking or directly into the network architecture. The contraction state is determined by comparing measures such as muscle thickness and center of mass between rest and exercise. A retrospective analysis is conducted but also a real-time scenario is simulated, where classification is performed during exercise. Results: Using the proposed segmentation algorithms, 71% of the muscle states are classified correctly in the retrospective analysis in comparison to 90% accuracy with manual reference segmentation. For the real-time approach the majority of given feedback during exercise is correct when the retrospective analysis had come to the correct result, too. Conclusion: Both retrospective and real-time analysis prove to be feasible. While no substantial differences between the algorithms were observed regarding classification, the networks incorporating the time information showed temporally more consistent segmentations. Limitations of the approaches as well as reasons for failing cases in segmentation, classification and real-time assessment are discussed and requirements regarding image quality and hardware design are derived. [ABSTRACT FROM AUTHOR]

This study systematically investigates the flow birefringence of cellulose nanocrystal (CNC) suspensions. The aim is to clarify the importance of the stress component along the camera's optical axis in the stress-optic law (SOL), which describes the relationship between birefringence, the retardation of transmitted polarized light, and the stress field. More than 100 datasets pertaining to the retardation of CNC suspensions (concentrations of 0.1, 0.3, 0.5, and 1.0 wt%) in a laminar flow field within a rectangular channel (aspect ratios of 0.1, 1, and 3) are systematically obtained. The measured retardation data are compared with the predictions given by the conventional SOL excluding the stress component along the camera's optical axis and by the SOL including these components as second-order terms (2nd-order SOL). The results show that the 2nd-order SOL gives a significantly better agreement with the measurements. Based on the 2nd-order SOL, the retardation at the center of the channel, where the effect of the stress component along the camera's optical axis is most pronounced, is predicted to be proportional to the square of the flow rate, which agrees with the experimental data. The results confirm the importance of considering the stress component along the camera's optical axis in the flow birefringence of CNC suspensions at high flow rates, even for quasi-two-dimensional channel flow. [ABSTRACT FROM AUTHOR]

Micro-expressions are instantaneous flashes of facial expressions that reveal a person's true feelings and emotions. Micro-expression recognition (MER) is challenging due to its low motion intensity, short duration, and the limited number of publicly available samples. Although the present MER methods have achieved great progress, they face the problems of a large number of training parameters and insufficient feature extraction ability. In this paper, we propose a lightweight network MFE-Net with Res-blocks to extract multi-scale features for MER. To extract more valuable features, we incorporate Squeeze-and-Excitation attention and multi-headed self-attention mechanisms in our MFE-Net. The proposed network is used for learning features from three optical flow features (i.e. optical strain, horizontal and vertical optical flow images) which are calculated from the onset and apex frames. We employ the LOSO cross-validation strategy to conduct experiments on CASME II and the composite dataset selected by MEGC2019, respectively. The extensive experimental results demonstrate the viability and effectiveness of our method. [ABSTRACT FROM AUTHOR]

Underwater target motion estimation is a challenge for ocean military and scientific research. In this work, we propose a method based on the combination of polarization imaging and optical flow for turbid underwater target detection. Polarization imaging can reduce the influence of backscattered light and obtain high-quality images underwater. The optical flow shows the motion and structural information of the target. We use polarized optical flow to obtain the optical flow field and estimate the target motion. The experimental results of different targets under varying water turbidity levels illustrate that our method is realizable and robust. The precision is verified by comparing the results with the precise displacement data and calculating two error measures. The proposed method based on polarized optical flow can obtain accurate displacement information and a good recognition effect. Moving target segmentation based on the Otsu method further proves the superiority of the polarized optical flow under turbid water. This study is valuable for target detection and motion estimation in scattering environments. [ABSTRACT FROM AUTHOR]

The present study discusses the effect of symmetric and asymmetric grafting on the surface of CNCs (cellulose nanocrystals) on their dispersion properties using dialkyl azetidinium salts. Three dialkylamine of different size and chain length were successfully grafted to the sulfate groups on the surface of CNCs by conjugation of azetidinium salts. The coupling process resulted in the formation of 2-hydroxypropyl-N-dialkylamine conjugated to the CNC sulfate groups abbreviated as C n -N-C m -Prop-2-OH-CNC, where m, n are the number of carbons in the alkyl groups, each with a total of m + n = 12 , with (m , n) = (11 , 1) ; (9 , 3) ; (6 , 6) . Molecular dynamics simulations were used to assess the probable morphology of the grafted chains and the interaction potential between CNCs. Steady shear simultaneously combined with polarized light imaging and oscillatory shear rheological measurements were used to evaluate for the first time the impact of the CNC surface modifications on their dispersion flow and optical properties. Overall, the results show that the different linker topologies could effectively promote different types of aggregation morphologies based on the size of the linker, their flexibility and their most probable conformation. [ABSTRACT FROM AUTHOR]

Edge computing is essential to handle increasing data volumes and processing capacities. It provides real-time and secure data processing near data sources, like smart devices, alleviating cloud computing energy use, and saving network bandwidth. Specialized accelerators, like GPUs and FPGAs, are vital for low-latency edge computing but the requirements to customized code for different hardware and vendors suppose important compatibility issues. This paper evaluates the potential of SYCL in addressing code portability issues encountered in edge computing. We employed the Polybench suite to compare various SYCL implementations, specifically DPC++ and AdaptiveCpp, with the native solution, CUDA. The disparity between SYCL implementations was negligible, at just 5%. Furthermore, we evaluated SYCL in the context of specific edge computing applications such as video processing using three different optical flow algorithms. The results revealed a slight performance gap of 3% when transitioning from CUDA to SYCL. Upon evaluating energy consumption, the observed difference ranged from ± 10 % , depending on the application utilized. These gaps are the price one may need to pay when achieving the ability to successfully run the same code on two distinct edge boards. These findings underscore SYCL's capacity to increase productivity in terms of development costs and facilitate IoT deployment without being locked into a particular platform or manufacturer. [ABSTRACT FROM AUTHOR]

Purpose: This study aimed to investigate differences in microvasculature dropout (MvD) between the superior and inferior hemispheres in glaucoma patients. Study design: Retrospective and cross-sectional. Methods: Fifty-eight eyes of 58 open-angle glaucoma patients (age 61.12 ± 10.19 years, mean deviation − 7.32 ± 6.36 dB) were included. MvD was detected with en face images from swept-source optical coherence tomography angiography. Blood flow at the optic nerve head was measured with laser speckle flowgraphy, represented as the mean blur rate in tissue (MBRT). Logistic and linear regression models adjusted for age, intraocular pressure, axial length, and circumpapillary retinal nerve fiber layer thickness were used to investigate the relationship between various factors and MvD angle in each hemisphere. Results: The presence of inferior MvD was related to peripapillary atrophy-β area (odds ratio = 14.10 [2.49–234.00], P = 0.019). Superior MvD angle was significantly related to MBRT in the superior quadrant (β = −0.31 [− 0.60 – −0.02], P = 0.037). Inferior MvD angle was significantly related to peripapillary atrophy-β area (β = 0.49 [0.21–0.77], P = 0.001). Conclusions: Only superior MvD demonstrated a significant relationship with reduced ocular blood flow. In contrast, inferior MvD was associated with mechanical stress. These findings may suggest a potential difference in pathophysiology between superior and inferior MvD. [ABSTRACT FROM AUTHOR]

Video anomaly detection is an extremely challenging task in the field of intelligent surveillance analysis. In this paper, we propose a video anomaly detection method without any manual annotation information, which is the key limitations of existing weakly-supervised methods. Compared to existing single-clue unsupervised methods, we explore the importance of multiple cues and design a self-trained multi-cues model for video anomaly detection. In addition to appearance features, we find motion features and reconstruction error features are essential for detecting abnormal behaviors. Our method achieves the extraction and fusion of these features from video based on self-trained framework. Specifically, we use auto-encoders to generate reconstruction error maps of frames and optic flow maps respectively. Then we extract multiple cues features from frames/flow maps and the reconstruction error maps to detect abnormal events. As our model is self-trained, we do not need manually labeled training data. We conduct validation experiments on two public datasets. The experimental results show our self-trained multi-cues model outperforms existing unsupervised video anomaly detection methods and leads to good results compared with weakly-supervised methods. [ABSTRACT FROM AUTHOR]

Violence may happen anywhere. One of the ways to know and oversee the violence in some places is by installing Closed-circuit Television (CCTV). The recorded video captured by CCTV can be used as proof in a law court. Violence video classification is also one of the topics being discussed in deep learning. The latest violence video dataset is RWF-2000. That dataset contains violent and non-violent videos, 5 seconds duration, 30 frames per second, with the amount of 2000 videos. That publication also has the best accuracy of 87.25% by their proposed method. In this study, we will use a Residual Network known to have the advantage of solving the vanishing gradient problem. Beside that, we also implement transfer learning from Kinetics and Kinetics + Moments in Time pre-trained data. We also test the number of frames and the location range of the sampled frames. RGB and optical flow inputs are separately trained with different configurations. The RGB input best accuracy is 89.25% with pre-trained Kinetics + Moments in Time, using frame location of 49-149. The optical flow input best accuracy is 88.5% with pre-trained Kinetics, using 74 frames. We also try to sum the output of both inputs making accuracy of 90.5%. [ABSTRACT FROM AUTHOR]

Macro-expression spotting is an important prior step in many dynamic facial expression analysis applications. It automatically detects the onset and offset image frames of a macro-expression in the video. The state-of-the-art methods of macro-expression spotting characterize the movement of facial muscle through explicit analysis of the optical flow map and have achieved promising results. However, optical flow map estimation and expression spotting in these methods are performed in two separate and successive stages. In this paper, we propose a new dual-branch network to achieve unified optimization for expression spotting and optical flow estimation tasks. The proposed dual-branch network implicitly learns optical flow during training and enriches the feature representation with motion information. During inference, we use only the encoder of the optical flow estimation network for motion feature extraction and integrate it with expression spotting into a one-stage framework. The proposed method eliminates the need to construct optical flow maps explicitly during inference and significantly reduces the computational cost. We also apply a consistency constraint on the global- and local-level semantic features of the clip to guide the model to focus on the category-consistent regions of the video. We evaluate the proposed methods extensively on two popular facial expression spotting datasets, CAS(ME) 2 and SAMM Long Videos. The experimental results show that compared with the state-of-the-art methods, the proposed method improves the F1-scores for MaE spotting by 5.81 % and 1.57 % on the CAS(ME) 2 and SAMM Long Videos datasets respectively. [ABSTRACT FROM AUTHOR]

Style transfer aims to apply the stylistic characteristics of a reference image onto a target image or video. Existing studies on style transfer suffer from either fixed style without adjustability or unclear stylistic patterns in output results. Moreover, concerning video style transfer, issues such as discontinuity in content and time, flickering, and local distortions are common. Current research on artistic image style transfer mainly focuses on Western painting. In view of the differences between Eastern and Western painting, the existing methods cannot be directly applied to the style transfer of Chinese painting. To address the aforementioned issues, we propose a controllable style transfer method based on generative adversarial networks. The method operates directly in the feature space of style and content domains, synthesizing target images by merging style features and content features. To enhance the output stylization effect of Chinese painting, we incorporate stroke constraints and ink diffusion constraints to improve the visual quality. To mitigate issues such as blank spaces, highlights, and color confusion resulting in flickering and noise in Chinese painting style videos, we propose a flow-based stylized video optimization strategy to ensure consistency in content and time. Qualitative and quantitative experimental results show that our method outperforms state-of-the-art style transfer methods. [ABSTRACT FROM AUTHOR]

Background: Being an image-based optical technique for full-field deformation measurements, the ultimate purpose of digital image correlation (DIC) is to realize accurate, precise and pixel-wise displacement/strain measurements in a full-automatic manner without users' inputs. Objective: In this work, we propose a task-optimized neural network, called RAFT-DIC, to achieve user-independent, accurate and pixel-wise displacement field measurements. Methods: RAFT-DIC is based on the state-of-the-art optical flow architecture: Recurrent All-Pairs Field Transforms (RAFT). We make two targeted improvements that fundamentally enhanced its measurement accuracy and generalization performance. Firstly, we remove all the down-sampling operations in the encode module to improve the perception of spatial information, and reduce the number of pyramid levels of the correlation layer to increase the small displacement accuracy. By building the correlation layer to compute the similarity of pixel pairs, and iteratively updating the displacement field through a recurrent unit, RAFT-DIC introduces the prior information of DIC measurement to guide the displacement estimation with high accuracy. Secondly, we develop a novel dataset generation method to synthesize customized speckle patterns and diverse displacement fields, which facilitate the construction of a robust and adaptable dataset to improve the network generalization. Results: Both simulated and real experimental results demonstrate that the accuracy of the proposed method is approximately an order of magnitude higher than pervious deep learning-based DIC (DL-DIC). Conclusions: The proposed RAFT-DIC shows higher accuracy as well as stronger practicality and cross-dataset generalization performance over existing DL-DIC methods, and is expected to be a new standard architecture for DL-DIC. [ABSTRACT FROM AUTHOR]

The use of meteorological radars in monitoring current weather conditions is crucial regarding the observation of the evolution and dissipation of thunderstorms. Thus, Doppler velocities being measured in each radar scan and velocity vectors derived from Numerical Weather Prediction (NWP) models—that are usually not as highly resolving as radar scans—are combined, as a monitoring utility during the evolution of a convective weather event. The objective of this study is to develop a new method that allows the implementation of a thunderstorm movement velocity estimation technique combining block matching and optical flow techniques. This new method constitutes a nowcasting (NWC) application that enables the use of a single Doppler radar without the need of using NWPs. The method relies on the estimation of the thunderstorm movement vector velocities (Doppler velocity) for each constant altitude plan position indicator (CAPPI) and through correction for aliasing errors to obtain 3D vector velocity fields for convective systems. The performance of the method is evaluated for selected case studies of convective thunderstorms under different synoptic scale conditions over Greece, a geographical area with challenges in forecasting due to its sharp relief and the need for optimization of the use of radar products. [ABSTRACT FROM AUTHOR]

Video colorization is the process of assigning reasonable color values to gray videos. Most of the existing video colorization methods use an image colorization algorithm directly regardless of correlation between frames in videos. Besides, these methods need reference, but it is difficult to obtain the guidance of users or reference images in practical applications. To overcome this limitation, we propose an automatic video colorization based on contrastive learning and optical flow. Contrastive learning optimizes the temporal correlation loss between the feature vectors of the gray video frame patches and the generated color video frame patches during training, which ensures the consistency of image content at the patch level. The optical flow module maintains the temporal consistency of colored videos by minimizing temporal correlation loss. In particular, the perceptual loss is introduced to improve the subjective colorization effect. Experiments show that the proposed algorithm outperforms the existing methods in terms of subjective quality. To reproduce the results, we release the source code at: https://github.com/orange-416/CLAC-GAN. [ABSTRACT FROM AUTHOR]

Multi-Object Tracking, also known as Multi-Target Tracking, is an important area of computer vision with various applications in different domains. The advent of deep learning has had a profound impact on this field, forcing researchers to explore innovative avenues. Deep learning methods have become the cornerstone of today's state-of-the-art solutions, consistently delivering exceptional tracking results. However, the significant computational demands of deep learning models require powerful hardware resources that do not always match real-time tracking requirements, limiting their practical applicability in real-world scenarios. Thus, there is an imperative to strike a balance by merging robust deep learning strategies with conventional approaches to enable more accessible, cost-effective solutions that meet real-time requirements. This paper embarks on this endeavor by presenting a hybrid strategy for real-time multi-target tracking. It effectively combines a classical optical flow algorithm with a deep learning architecture tailored for human crowd tracking systems. This hybrid approach achieves a commendable balance between tracking accuracy and computational efficiency. The proposed architecture, subjected to extensive experimentation in various settings, demonstrated notable results, achieving a Mean Object Tracking Accuracy (MOTA) of 0.608. This level of performance placed it as the highest ranking solution on the MOT15 benchmark, surpassing the state-of-the-art benchmark of 0.549, and consistently ranked among the superior models on the MOT17 and MOT20 benchmarks. Additionally, the incorporation of the optical flow phase resulted in a substantial reduction in processing time, nearly halving the duration, while simultaneously maintaining accuracy levels comparable to established techniques. [ABSTRACT FROM AUTHOR]

FlowDenoising is a new approach to noise reduction in biological volumes obtained with three-dimensional electron microscopy (3DEM). Its abilities to enhance the structural features stem from the fact that an anisotropic Gaussian filtering is steered according to the local structures. To this end, the Optical Flow (OF) among consecutive slices is estimated, which is the most computationally expensive step in this approach. In this article, a hybrid CPU/GPU implementation of FlowDenoising is introduced and evaluated. It exploits parallel computing by distributing the workload among multiple cores and takes advantage of the massive processing in GPUs to accelerate the OF estimation. The hybrid implementation provides remarkable speed-up factors and an important reduction of the processing time, which is particularly relevant for the denoising of huge volumes typically found in 3DEM. [ABSTRACT FROM AUTHOR]

Velocity field quantification for high-speed flows is of fundamental importance to understand flow dynamics, turbulence, and flow–structure interactions. Optical velocimetry techniques commonly provide sparse information in the flows. Dense fields of velocity vectors with high spatial resolutions are indispensable for detailed analysis of complex motion patterns and accurate motion tracking within the field of view. In the present work, two-dimensional (2D) Rayleigh scattering imaging (RSI) at a rate of 10- to 100-kHz was utilized to quantify the high-speed flow velocity by employing deep learning-based optical flow analysis, along with density and temperature fields from Rayleigh scattering intensity profiles. High-speed Rayleigh scattering images are highly spatially resolved, have smooth gradients without intensity discontinuities, and precisely track key features of the flows. The deep learning-based optical flow method utilizes recurrent neural network architecture to extract the per-pixel features of both input images, calculate correlation from all pairs of the features, and get training by recurrently updating the optical flow. 2D instantaneous velocity fields of both nonreacting and reacting flows measured by RSI were obtained from deep learning-based optical flow analysis, thus extending RSI as a non-intrusive, nonseeded, and multiscalar measurement technique of high-speed nonreacting and reacting flows. [ABSTRACT FROM AUTHOR]

Optic flow provides useful information in service of spatial navigation. However, whether brain networks supporting these two functions overlap is still unclear. Here we used Activation Likelihood Estimation (ALE) to assess the correspondence between brain correlates of optic flow processing and spatial navigation and their specific neural activations. Since computational and connectivity evidence suggests that visual input from optic flow provides information mainly during egocentric navigation, we further tested the correspondence between brain correlates of optic flow processing and that of both egocentric and allocentric navigation. Optic flow processing shared activation with egocentric (but not allocentric) navigation in the anterior precuneus, suggesting its role in providing information about self-motion, as derived from the analysis of optic flow, in service of egocentric navigation. We further documented that optic flow perception and navigation are partially segregated into two functional and anatomical networks, i.e., the dorsal and the ventromedial networks. Present results point to a dynamic interplay between the dorsal and ventral visual pathways aimed at coordinating visually guided navigation in the environment. [ABSTRACT FROM AUTHOR]

Existing homography and optical flow methods are erroneous in challenging scenes, such as fog, rain, night, and snow because the basic assumptions such as brightness and gradient constancy are broken. To address this issue, we present an unsupervised learning approach that fuses gyroscope into homography and optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Second, we design a self-guided fusion module (SGF) to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. Meanwhile, we propose a homography decoder module (HD) to combine gyro field and intermediate results of SGF to produce the homography. To the best of our knowledge, this is the first deep learning framework that fuses gyroscope data and image content for both deep homography and optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-the-art methods in both regular and challenging scenes. The code and dataset are available at https://github.com/lhaippp/GyroFlowPlus. [ABSTRACT FROM AUTHOR]

The paper proposes a new approach for crowd movement type estimation in video by combining convolutional neural network and integral optical flow. At first, main notions of crowd detection and tracking are given. Secondly, crowd movement features and parameters are defined. Three rules are proposed to identify direct crowd motion. Signs are presented for identifying chaotic crowd movement. Region movement indicators are introduced to analyze the movement of a group of people or a crowd. Thirdly, an algorithm of crowd movement types estimation using convolutional neural network and integral optical flow is proposed. We calculate crowd movement trajectories and show how they can be used to analyze behavior and divide crowds into groups of people. Experimental results show that with the help of convolutional neural network and integral optical flow crowd movement parameters can be calculated more accurately and quickly. The algorithm demonstrates stronger robustness to noise and the ability to get more accurate boundaries of moving objects. [ABSTRACT FROM AUTHOR]

This paper presents a low-cost pipeline surface 3D detection method based on line structured light. The method adds 3D detection to the robot at a meager cost and change. The optical flow method is used to derive the motion information of the robot to replace the motion closed loop. Finally, the depth data for the entire surface are generated automatically only from the camera and the line laser projector, without using other devices. In addition, a novel spot centroid extraction algorithm based on the color region of interest and an adaptive threshold is presented. This method can accurately detect the laser centroid in the pipeline surface. We conducted experiments with a quadruped robot and validated algorithms. The experimental results show that the proposed method achieves 3D detection on a trackless robot at a meager cost and is superior to standard depth sensors. [ABSTRACT FROM AUTHOR]

In this study, we propose a new approach to improve the accuracy of the horizontal atmospheric motion vector (AMV) in cloud-free skies and its forecasting. We adapted the optical flow of the convolutional neural network (CNN) framework model using two 10-min interval infrared images at water vapor channels (centered at 6.3, 7.0, and 7.3 μ m ) from the Korean geostationary satellite GEO-KOMPSAT-2A (GK2A). Since all pixels had seamless AMVs calculated by CNN (CNN AMVs), we could also predict AMVs using the linear regression method. The tracking performance of the CNN-based algorithm was validated using AMVs retrieved from GK2A (GK2A AMVs) by estimating the difference between those values and the ECMWF (European Centre for Medium-Range Weather Forecasts) Reanalysis v5 (ERA5) wind data over Korea in 2022. CNN AMVs showed similar or better root-mean-square vector differences (RMSVDs) than GK2A AMVs (12.33–12.86 vs. 15.89–19.96 m/s). The RMSVDs of the forecasted AMVs were 2.74, 2.95, 3.41, and 4.79 m/s at lead times of 10, 20, 30, and 60 min, respectively. Consequently, our method showed higher accuracy for tracking motion in the production of AMVs and succeeded in forecasting AMVs. We expect that such potential improvements in computational accuracy for operational GK2A AMVs will contribute to increased accuracy when forecasting meteorological phenomena related to wind. [ABSTRACT FROM AUTHOR]

How to effectively fuse spatiotemporal clues is the key to improve the accuracy of video salient object detection. Although most existing methods have achieved great success regarding fusion strategies, the issue of reliability of spatiotemporal clues needs further investigation, and the use of unreliable spatiotemporal clues can corrupt the final saliency results. In this work, we propose a novel dynamic selection-fusion network (DSFNet) for video salient object detection, and DSFNet is jointly constructed by two branches. The one is the spatial learning network, which completes the learning of video sequences to obtain the spatial saliency of images. The other is the spatiotemporal contrast network, which creatively obtains the dynamic spatiotemporal saliency in the synchronized state by learning the video sequence and the corresponding optical flow images. To further screen and fuse the spatiotemporal clues, a series of joint modules for selection were developed, mainly including contrast transformation module (CTM), contrast analysis module (CAM) and selection guidance module (SGM) which play an important role in selecting spatiotemporal features. In addition, a fusion refinement module (FRM) is designed to further refine and enhance the input features. The experimental results show that the proposed method is significantly better than other algorithms in solving the problem of motion information distortion and spatiotemporal salient irrelevance. [ABSTRACT FROM AUTHOR]

Video anomaly detection (VAD) task can be addressed as a semi-supervised learning problem as datasets are highly biased towards normal samples. The popular reconstruction methods train the network based on only normal images. These methods detect anomaly events by comparing the input with the reconstructed image, assuming that the network can not accurately reconstruct anomalous regions. However, these methods suffer from the serious flaw that the anomaly regions are generalized sufficiently well. This problem reduces the ability of anomaly detection by narrowing the gap between reconstructed and anomaly input images. By converting the reconstruction process into an inpainting process, we introduce Inpainting-GAN, a distinctive anomaly detection model that utilizes generative adversarial networks (GANs) as the backbone network. Each input image is partially occluded in random way by a series of disjoint masks before being fed into our model, any abnormal areas in the frame will definitely be covered. Our network is trained to restore those masked regions to normal parts, which overcomes the drawback of reconstruction models. To address the temporal continuity in videos, we have incorporated optical flow as a motion constraint during the training of generator. Both qualitative and quantitative experimental results show that our proposed method shows competitive performance in VAD compared to some reconstruction methods. The experimental results on the UCSD Ped1, Ped2 and CUHK Avenue datasets have obtained AUC values of 78.6%, 94.7% and 81.4%. [ABSTRACT FROM AUTHOR]

It is a challenging and meaningful task to carry out UAV-based livestock monitoring in high-altitude (more than 4500 m on average) and cold regions (annual average – 4 °C) on the Qinghai Tibet Plateau. The purpose of artificial intelligence (AI) is to execute automated tasks and to solve practical problems in actual applications by combining the software technology with the hardware carrier to create integrated advanced devices. Only in this way, the maximum value of AI could be realized. In this paper, a real-time tracking system with dynamic target tracking ability is proposed. It is developed based on the tracking-by-detection architecture using YOLOv7 and Deep SORT algorithms for target detection and tracking, respectively. In response to the problems encountered in the tracking process of complex and dense scenes, our work (1) Uses optical flow to compensate the Kalman filter, to solve the problem of mismatch between the target bounding box predicted by the Kalman filter (KF) and the input when the target detection in the current frame is complex, thereby improving the prediction accuracy; (2) Using a low confidence trajectory filtering method to reduce false positive trajectories generated by Deep SORT, thereby mitigating the impact of unreliable detection on target tracking. (3) A visual servo controller has been designed for the Unmanned Aerial Vehicle (UAV) to reduce the impact of rapid movement on tracking and ensure that the target is always within the field of view of the UAV camera, thereby achieving automatic tracking tasks. Finally, the system was tested using Tibetan yaks on the Qinghai Tibet Plateau as tracking targets, and the results showed that the system has real-time multi tracking ability and ideal visual servo effect in complex and dense scenes. [ABSTRACT FROM AUTHOR]

In order for detection of a fire in fields, it is effective to detect smoke since it often behaves as a precursor of the fire. One preferable way for early detection is to use visual information from outdoor cameras that widely monitor the filed. There have been many attempts to detect smokes via optical sensors on digital cameras using optical flow methods, but not fully successful from practical-use aspects. It is because the area of smokes occupying on the footage by outdoor cameras is not necessarily large enough. Moreover, in case of urban cities, discrimination of the smokes from other moving objects such as cars, trees and turbines is not easy. Herein we propose a novel method to detect daytime smokes based on variance of optical flow and characteristics of HSV (hue-saturation-value) color. We apply the method to a set of footage of three days obtained in an industrial zone in Japan. Successful results are obtained as over 90% of smokes are detected. Notable is that this method is independent of solar radiation conditions on sunny and cloudy days. [ABSTRACT FROM AUTHOR]

Hierarchical concepts have proven useful in many classical and learning-based optical flow methods regarding both accuracy and robustness. In this paper we show that such concepts are still useful in the context of recent neural networks that follow RAFT's paradigm refraining from hierarchical strategies by relying on recurrent updates based on a single-scale all-pairs transform. To this end, we introduce MS-RAFT+: a novel recurrent multi-scale architecture based on RAFT that unifies several successful hierarchical concepts. It employs a coarse-to-fine estimation to enable the use of finer resolutions by useful initializations from coarser scales. Moreover, it relies on RAFT's correlation pyramid that allows to consider non-local cost information during the matching process. Furthermore, it makes use of advanced multi-scale features that incorporate high-level information from coarser scales. And finally, our method is trained subject to a sample-wise robust multi-scale multi-iteration loss that closely supervises each iteration on each scale, while allowing to discard particularly difficult samples. In combination with an appropriate mixed-dataset training strategy, our method performs favorably. It not only yields highly accurate results on the four major benchmarks (KITTI 2015, MPI Sintel, Middlebury and VIPER), it also allows to achieve these results with a single model and a single parameter setting. Our trained model and code are available at https://github.com/cv-stuttgart/MS_RAFT_plus. [ABSTRACT FROM AUTHOR]

This study presents an analysis of the electrolysis process using nickel–iron electrodes with a 30% KOH electrolyte and a 30% KOH electrolyte with an additional 2.5 g of ionic liquid (BMI.BF4). The electrodes were placed 20 mm apart, and the current density at 55, 110, and 165 A/m2. The process involved capturing and processing images of hydrogen generation to observe the influence of the ionic liquid on ionic conductivity. Results, depicted through images of vertical velocity, vorticity, and velocity field, indicate potential gains in hydrogen generation with larger volumes of bubbles and improved vertical displacement speeds of bubbles when adding ionic liquid to the electrolyte. [ABSTRACT FROM AUTHOR]

Sign language serves as a vital means of communication for individuals with hearing impairments, yet recognition resources for the over 100 distinct sign languages are severely lacking. In response, we present our work on sign language recognition using transfer learning and the domain adaptation method TA3N, which utilizes the Temporal Relational Network (TRN) module for aligning multi-scale temporal relations. Our findings highlight the superior performance of Domain Adaptation to neural network-based transfer learning, particularly in improving recognition of American Sign Language (ASL). Our research also identifies the effectiveness of aligning shorter-term temporal features between source and target domains. In addition to using RGB, we conducted experiments using Optical Flow mode for the sign language samples, ultimately determining that RGB outperforms Optical Flow in the majority of cases. Our work aims to improve accessibility and communication for individuals who rely on sign language as their primary mode of communication. [ABSTRACT FROM AUTHOR]

Current marker-based facial motion capture methods might lose the target markers in some cases, such as those with considerable occlusion and blur. Manually revising these statuses requires extensive labor-intensive work. Thus, a robust marker tracking method that provides long-term stability must be developed, thereby simplifying manual operations. In this paper, we present a new facial marker tracking system that focuses on the accuracy and stability of performance capture. The tracking system includes a synthetic analysis step with the robust optical flow tracking method and the proposed Marker-YOLO detector. To illustrate the strength of our system, a real dataset of the performance of voluntary actors was obtained, and ground truth labels were given by artists for subsequent experiments. The results showed that our approach outperforms state-of-the-art trackers such as SiamDW and ECO in specific tasks while running at a real-time speed of 38 fps. The root-mean-squared error and area under the curve results verified the improvements in the accuracy and stability of our approach. [ABSTRACT FROM AUTHOR]

To resolve high-precision deformation from low-quality speckle images, a high-order time-domain digital image correlation algorithm is proposed. This algorithm is based on the nonlinear optical flow equation for low-quality speckle images collected in extreme testing environments. A subpixel displacement image containing Gaussian white noise is established by using the simulated speckle method. The displacement of the high-order time-domain digital image correlation algorithm is solved by the least squares iteration method, and the test accuracy of the proposed algorithm is analyzed. The results show that the nonlinear optical flow equation can describe the time continuity of pixel gray level changes before and after deformation. The proposed algorithm can also analyze the velocity and acceleration term of deformation. The high-order time-domain digital image correlation algorithm is insensitive to the change in image grayscale and has high measurement accuracy. Therefore, this method can solve the problem of resolving high-precision deformation from low-quality speckle images. [ABSTRACT FROM AUTHOR]

Recent optical flow estimators usually employ deep models designed for image classification as the encoders for feature extraction and matching. However, those encoders developed for image classification may be sub-optimal for flow estimation. In contrast, the decoder design of optical flow estimators often requires meticulous design for flow estimation. The disconnect between the encoder and decoder could negatively affect optical flow estimation. To address this issue, we propose a neural architecture search method, FlowNAS, to automatically find the more suitable and stronger encoder architecture for existing flow decoders. We first design a suitable search space, including various convolutional operators, and construct a weight-sharing super-network for efficiently evaluating the candidate architectures. To better train the super-network, we present a Feature Alignment Distillation module that utilizes a well-trained flow estimator to guide the training of the super-network. Finally, a resource-constrained evolutionary algorithm is exploited to determine an optimal architecture (i.e., sub-network). Experimental results show that FlowNAS can be easily incorporated into existing flow estimators and achieves state-of-the-art performance with the trade-off between accuracy and efficiency. Furthermore, the encoder architecture discovered by FlowNAS with the weights inherited from the super-network achieves 4.67% F1-all error on KITTI, an 8.4% reduction of RAFT baseline, surpassing state-of-the-art handcrafted GMA and AGFlow models, while reducing the model complexity and latency. The source code and trained models will be released at https://github.com/VDIGPKU/FlowNAS. [ABSTRACT FROM AUTHOR]

Background: X-ray imaging addresses many challenges with visible light imaging in extreme environments, where optical diagnostics such as digital image correlation (DIC) and particle image velocimetry (PIV) suffer biases from index of refraction changes and/or cannot penetrate visibly occluded objects. However, conservation of intensity—the fundamental principle of optical image correlation algorithms—may be violated if ancillary components in the X-ray path besides the surface or fluid of interest move during the test. Objective: The test series treated in this work sought to characterize the safe use of fiber-epoxy composites in aerospace and aviation industries during fire accident scenarios. Stereo X-ray DIC was employed to measure test unit deformation in an extreme thermal environment involving a visibly occluded test unit, incident surface heating to temperatures above 600oC, and flames and soot from combusting epoxy decomposition gasses. The objective of the current work is to evaluate two solutions to resolve the violation of conservation of intensity that resulted from both the test unit and the thermal chamber deforming during the test. Methods: The first solution recovered conservation of intensity by pre-processing the path-integrated X-ray images to isolate the DIC pattern of the test unit from the thermal chamber components. These images were then correlated with standard, optical DIC software. The second solution, called Path-Integrated Digital Image Correlation (PI-DIC), modified the fundamental matching criterion of DIC to account for multiple, independently-moving components contributing to the final image intensity. The PI-DIC algorithm was extended from a 2D framework to a stereo framework and implemented in a custom DIC software. Results: Both solutions accurately measured the cylindrical shape of the undeformed test unit, recovering radii values of R = 76.20 ± 0.12 mm compared to the theoretical radius of R theor = 76.20 mm. During the test, the test unit bulged asymmetrically as decomposition gasses pressurized the interior and eventually burned in a localized jet. Both solutions measured the heterogeneous radius of this bulge, which reached a maximum of approximately R = 91 mm, with a discrepancy of 2–3% between the two solutions. Conclusions: Two solutions that resolve the violation of conservation of intensity for path-integrated X-ray images were developed. Both were successfully employed in a stereo X-ray DIC configuration to measure deformation of an aluminum-skinned, fiber-epoxy composite test unit in a fire accident scenario. [ABSTRACT FROM AUTHOR]

For visual estimation of optical flow, which is crucial for various vision analyses, unsupervised learning by view synthesis has emerged as a promising alternative to supervised methods because the ground-truth flow is not readily available in many cases. However, unsupervised learning is likely to be unstable when pixel tracking is lost via occlusion and motion blur, or pixel correspondence is impaired by variations in image content and spatial structure over time. Recognizing that dynamic occlusions and object variations usually exhibit a smooth temporal transition in natural settings, we shifted our focus to model unsupervised learning optical flow from multi-frame sequences of such dynamic scenes. Specifically, we simulated various dynamic scenarios and occlusion phenomena based on Markov property, allowing the model to extract motion laws and thus gain performance in dynamic and occluded areas, which diverges from existing methods without considering temporal dynamics. In addition, we introduced a temporal dynamic model based on a well-designed spatial-temporal dual recurrent block, resulting in a lightweight model structure with fast inference speed. Assuming the temporal smoothness of optical flow, we used the prior motions of adjacent frames to supervise the occluded regions more reliably. Experiments on several optical flow benchmarks demonstrated the effectiveness of our method, as the performance is comparable to several state-of-the-art methods with advantages in memory and computational overhead. [ABSTRACT FROM AUTHOR]

Owing to the physical and economic impacts of urban flooding, effective flood risk management is of crucial importance. Thus, it is essential to employ reliable techniques for monitoring water levels in urban creeks and detecting abrupt fluctuations in weather patterns. Ground-based cameras alongside a creek offer a cost-effective solution, since they can be deployed for determining water levels through image-based analysis. Previous research has examined the benefits of image processing and artificial intelligence techniques to achieve this goal. However, the current methods only analyze static image features and ignore the valuable motion information that may exist in adjacent frames that are captured minutes apart. In addressing this limitation, our approach involves computing dense optical flow fields from consecutive images taken by a stationary camera and integrating these representations into a deep-learning workflow. We evaluated the capacity of both our method and alternative approaches to measure not only the absolute water level (i.e., whether the water height is low, medium, high, or flooding) but also the relative water level (i.e., whether the water level is rising or falling). The results showed that optical flow-based representations significantly improved the ability to measure the relative water level, while pairs of successive grayscale images effectively determined the absolute water level. [ABSTRACT FROM AUTHOR]