Search

In video action recognition, effective spatiotemporal modeling is crucial. However, traditional two-stream methods face challenges in integrating spatial information from RGB images and temporary information from optical flow without long-range temporal modelling. To address these limitations, we propose the Deep Fusion Module (DFM), which focuses on the deep fusion of spatial and temporal information and consists of two components. First, we propose an Attention Fusion Module (AFM) to effectively fuse the shallow features obtained from a two-stream network, thereby facilitating the integration of spatial and temporal information. Next, we incorporate a SpatioTemporal Module (STM), comprising a ConvGRU and a 1×1 convolution, to model long-range temporal dependency and fuse spatial-temporal features. Experiments on the UCF101 dataset show that our method achieves 96.5% accuracy, outperforming baseline two-stream models by 0.3%. [ABSTRACT FROM AUTHOR]

Artificial images and recordings are broad on the web via different media channels such as blogs, YouTube videos, etc. These manipulated and synthesized images tend to steal the identity of individuals and majorly contribute to establishing societal disruptions such as theft, political errors, social engineering, disinformation attacks and reputation fraud. These fake visual objects gradually came to be known as deep fakes. Different deep learning techniques are used to generate deepfake images which go unnoticed by human eyes. It is essential to develop a defense mechanism that can stop the common people from being manipulated and harnessed. The objective of this work is to develop an ensemble deep learning-based system that can differentiate between fake and real images. With the use of the recommended optical flow technique, a novel approach is proposed that extracts the apparent motion of image pixels which gives more accurate results compared to other state-of-the-art. FaceForensics + + dataset is used to test the extraction algorithms and ensemble model which fetched an accuracy of 86.02% for the DeepFake subset and 85.7% for the FaceSwap subset of the dataset. To the best knowledge, no one has completely used the ensemble model- OptiFake on the optical flow derived frames, highlighting a research gap in the field of deepfake detection. [ABSTRACT FROM AUTHOR]

Facial micro-expression (ME), consisting of uncontrollable muscle movements in faces, is an important clue for revealing real people's feelings. Due to the short duration and low intensity, the salient feature representation learning is the main challenge for robust facial ME recognition. To acquire the diverse and spatial relation representation, this paper proposes a simple and yet distinctive micro-expression recognition model based on multiscale convolutional fusion and multi-channel capsule network (MCFMCN). Firstly, the apex frame in a ME clip, located by computing the pixel difference between frames, is filtered by the optical flow transformation. Secondly, a multiscale fusion module is introduced to capture diverse ME related details. Then, to further explore the subtle spatial relations between parts in the ME faces, the multi-channel capsule network is designed to improve the feature representation performance of the traditional single channel capsule network. Finally, the entire ME recognition model is trained and verified on three benchmarks (CASMEII, SAMM, and SMIC) using the associated standard evaluation protocols: unweighted average recall rate (UAR) and unweighted F1 score (UF1). ME recognition experiments indicate that our method based on MCFMCN can improve the UAR (from 75.79% to 83.58%) and UF1(from79.37% to 87.06%) in comparison with the traditional capsule network. Extensive experimental results show the performance of proposed ME recognition is superior to that of works based on pervious single channel capsule network or other state-of-the-art CNN models, which validates the finding that combination of multi-scale analysis and multi-channel capsule network is feasible and effective to improve the ME recognition performance. [ABSTRACT FROM AUTHOR]

This paper proposes a method for an automatic detection of 3D-display-friendly scenes from video sequences. Manual selection of such scenes by a human user would be extremely time consuming and would require additional evaluation of the result on 3D display. The input videos can be intentionally captured or taken from other sources, such as films. First, the input video is analyzed and the camera trajectory is estimated. The optimal frame sequence that follows defined rules, based on optical attributes of the display, is then extracted. This ensures the best visual quality and viewing comfort. The following identification of a correct focusing distance is an important step to produce a sharp and artifact-free result on a 3D display. Two novel and equally efficient focus metrics for 3D displays are proposed and evaluated. Further scene enhancements are proposed to correct the unsuitably captured video. Multiple image analysis approaches used in the proposal are compared in terms of both quality and time performance. The proposal is experimentally evaluated on a state-of-the-art 3D display by Looking Glass Factory and is suitable even for other multi-view devices. The problem of optimal scene detection, which includes the input frames extraction, resampling, and focusing, was not addressed in any previous research. Separate stages of the proposal were compared with existing methods, but the results show that the proposed scheme is optimal and cannot be replaced by other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Since digital media has become increasingly popular, video processing has expanded in recent years. Video processing systems require high levels of processing, which is one of the challenges in this field. Various approaches, such as hardware upgrades, algorithmic optimizations, and removing unnecessary information, have been suggested to solve this problem. This study proposes a video saliency map based method that identifies the critical parts of the video and improves the system's overall performance. Using an image registration algorithm, the proposed method first removes the camera's motion. Subsequently, each video frame's color, edge, and gradient information are used to obtain a spatial saliency map. Combining spatial saliency with motion information derived from optical flow and color-based segmentation can produce a saliency map containing both motion and spatial data. A nonlinear function is suggested to properly combine the temporal and spatial saliency maps, which was optimized using a multi-objective genetic algorithm. The proposed saliency map method was added as a preprocessing step in several Human Action Recognition (HAR) systems based on deep learning, and its performance was evaluated. Furthermore, the proposed method was compared with similar methods based on saliency maps, and the superiority of the proposed method was confirmed. The results show that the proposed method can improve HAR efficiency by up to 6.5% relative to HAR methods with no preprocessing step and 3.9% compared to the HAR method containing a temporal saliency map. [ABSTRACT FROM AUTHOR]

• Vertical vergence was triggered by optic flow and visual rotations in all planes. • The degree of vergence was correlated with increased instability and stress. • Visually induced vertical vergence likely reflects subcortical vestibular activity. The present study explored visually induced vertical vergence (VIVV) as non-specific motion processing response. Healthy participants (7 male, mean age 28.57 ± 2.30; 9 female, mean age 27.67 ± 3.65) were exposed to optokinetic stimuli in an HTC VIVE virtual reality headset while VIVV, pupil-size, and postural sway was recorded. The methodology was shown to produce VIVV in the roll plane at 30 deg/s. Subsequent trials consisted of 40 s optokinetic motion in yaw, pitch, and roll directions at 60 deg/s, and radial optic flow; optokinetic directions were inverted after 20 s of motion. Median VIVV amplitude changes were normalized to the clockwise roll rotation, analysed, and correlated with changes in pupil-size and body sway. VIVV, pupil-size, and body sway were all affected by changes in optokinetic direction. Post-hoc analyses showed significant VIVV responses during optokinetic yaw and pitch rotations, as well as during radial optic flow stimulations. VIVV magnitudes were universally correlated with pupil-size and body sway. In conclusion, VIVV was expressed in all tested dimensions and may consequently serve as a visual motion processing biomarker. Failing to support binocularity while responding to optokinetic directionality, VIVV may reflect an eye-movement response associated with increased postural instability and stress, similar to a dorsal light reflex. [ABSTRACT FROM AUTHOR]

Summary: While wavelength division multiplexing (WDM) technology combines several wavelengths onto a single waveguide, the technology of mode division multiplexing (MDM) allows many orthogonal modes of the same wavelength to operate simultaneously without interchannel crosstalk. Thus, the hybrid WDM and MDM network in which the two above‐mentioned techniques cooperate could give a several‐fold increase in the overall network capacity. Constructing this network requires hybrid wavelength‐and‐mode multiplexers, especially ones with high integration and complementary metal‐oxide‐semiconductor (CMOS) compatibility. In this paper, we propose a design of a four‐mode triple‐band multiplexer that is capable of multiplexing up to 12 separate optical signal flows by utilizing four eigenmodes (TE0, TE1, TE2, and TE3) and three‐wavelength windows, which center at 1310, 1490, and 1550 nm. The device is on silicon‐on‐insulator (SOI) platform, consisting of four butterfly‐shaped multimode interference (MMI) couplers, four directional couplers, and a 4×1 cascaded asymmetric Y‐junction coupler. Via numerical simulations, the proposed design is verified to be able to operate effectively on the three aforementioned bandwidth slots with an optical conversion efficiency of over 93% in all functions. Moreover, it exhibits insertion loss less than 1.5 dB and crosstalk smaller than −16 dB within 25 nm bandwidth at each wavelength window. These results can affirm the success of wavelength–mode combination, which leads to a massive improve in the channel capacity on the same optical multiplexing system for optical telecommunications and photonics on‐chip interconnections. [ABSTRACT FROM AUTHOR]

The objective of this pilot study was to assess the reliability of superb microvascular ultrasound (SMI) for the measurement of the cerebrospinal fluid (CSF) flow within VPS systems as an indirect sign for shunt dysfunction. Asymptomatic hydrocephalus patients, with a VPS system implanted between 2017 and 2021, were prospectively enrolled in the study. Using SMI, the CSF flow within the proximal and distal catheters were analysed. Before and after pumping the shunt reservoir, intraabdominal free fluid, optical nerve sheath diameter (ONSD), and papilla diameter (PD) were evaluated and correlated with the amount of valve activation. Nineteen patients were included. A flow was detectable in 100% (N = 19) patients in the proximal and in 89.5% (N = 17) in the distal catheter. The distal catheter tip was detectable in 27.7% (N = 5) patients. Free intraabdominal fluid was initially detected in 21.4% (N = 4) patients and in 57.9% (N = 11) at the end of the examination (P = 0.049). ONSD was significantly lower after pump activation (4.4 ± 0.9 mm versus 4.1 ± 0.8 mm, P = 0.049). Both peak velocity and flow volume per second were higher in proximal compared to distal catheters (32.2 ± 45.2 versus 5.6 ± 3.7 cm/sec, P = 0.015; 16.6 ± 9.5 ml/sec versus 5.1 ± 4.0 ml/sec, P = 0.001, respectively). No correlation was found between the number of pump activations and the changes in ONSD (P = 0.975) or PD (P = 0.820). SMI appears to be a very promising non-invasive diagnostic tool to assess CSF flow within the VPS systems and therefore affirm their function. Furthermore, appearance of free intraperitoneal fluid followed by repeated compression of a shunt reservoir indicates an intact functioning shunt system. [ABSTRACT FROM AUTHOR]

Complex steerable pyramid (CSP) performs well when applied to magnify subtle motions of structures for observing the dynamic characteristics of facilities. However, the impact of the types and parameters of CSP filters upon the performance of phase-based optical flow (PBOF) in measuring motion parameters has not been systematically studied. The purpose of this study is to comprehensively evaluate the impact of different CSP filter types (Octave, HalfOctave, SmoothHalfOctave, and QuarterOctave) and parameters on the performance of PBOF in measuring motion parameters. Firstly, by measuring simulated translational motion, the influence of the CSP's down-sampling rates on the displacement measurement accuracy of PBOF is analyzed to determine appropriate settings. Subsequently, the effective displacement measurement interval and accuracy of PBOF using the CSP are studied through simulated and experimental translational motion measurements. Further, the vibration parameter's accuracy is analyzed through simulated periodic vibration measurements. Finally, the characteristics of PBOF using the four kinds of CSP and practical considerations are discussed. Simulation and experimental results demonstrate that when using middle-level filters within the effective level range of HalfOctave, PBOF achieves the best overall displacement measurement performance. Additionally, this method can easily integrate with signal processing techniques in analyzing structural dynamic characteristics under field conditions. [ABSTRACT FROM AUTHOR]

Emulsification processes are often found in the process industry and their evaluation is crucial for product quality and safety. Numerous methods exist to analyze critical quality attributes (CQA) such as the droplet sizes and droplet size distribution (DSD) of an emulsification process. During the emulsification process, the optical process accessibility may be limited due to high disperse phase content of liquid-liquid systems. To overcome this challenge, a modular, optical measurement flow cell is presented to widen the application window of optical methods in emulsification processes. In this contribution, the channel geometry is subject of optimization to modify the flow characteristics and produce high optical quality. In terms of rapid prototyping, an iterative optimization procedure via SLA-3D printing was used to increase operability. The results demonstrated that the flow cell resulting from the optimization procedure provides a broad observation window for droplet detection. [ABSTRACT FROM AUTHOR]

Human action recognition from videos requires a sequence of frames to be analyzed involving high processing power and time. Moreover, subsequent frames in a video contain redundancy that does not contribute to extracting distinguishable features but rather consumes processing resources. To recognize action with lower processing overhead, researchers are focusing on key frames and have proposed different techniques based on inter-frame clustering, fix-frame interval, and inter-frame differences. Clustering and fixed interval approaches consider the predefined number of frames or fixed intervals, therefore, not equally effective for both faster and slower actions which differ in terms of frequency of change. Furthermore, all existing approaches consider pixels? Intensities only but neglect another equally important aspect of motion, i.e., direction. The three-channel optical flows are one of the motion representations that depict both the magnitude and the direction of movement between pairs of frames in the form of colors. Here we proposed a novel "KFF-algorithm" that processes a sequence of three-channel optical flows to extract key flow frames for action recognition. Being dynamic in terms of frame interval, it efficiently extracts key frames for both slower and faster actions by analyzing changes in direction, magnitude, and coverage of motion in subsequent frames. Furthermore, in comparison with other approaches, KFF-algorithm covers relatively extended motion patterns with least number of frames or frames with significant change only. Also, for the majority of classes, KFF-algorithm has achieved substantial per-class accuracy when evaluated with the 3D-ConvNet model over the UCF-101 benchmark dataset for human action recognition. [ABSTRACT FROM AUTHOR]

Convolutional neural network-based image style transfer models often suffer from temporal inconsistency when applied to video. Although several video style transfer models have been proposed to improve temporal consistency, they often trade off processing speed, perceptual style quality, and temporal consistency. In this work, we propose a novel approach for fast continuous patch-based arbitrary video style transfer that achieves high-quality transfer results while maintaining temporal coherence. Our approach begins with stylizing the first frame as a standalone single image using patch propagation within the content activation. Subsequent frames are computed based on the key insight that optical flow field evaluated from neighboring content activations provides meaningful information to preserve temporal coherence efficiently. To address the problems introduced from optical flow stage, we additionally incorporate a correction procedure as a post-process to ensure a high-quality stylized video. Finally, we demonstrate our method can transfer arbitrary styles on a set of examples and illustrate that our approach exhibits superior performance both qualitatively and quantitatively. [ABSTRACT FROM AUTHOR]

Scene-flow (SF) estimation is considered to be one of the most fundamental problems in scene understanding and autonomous control. The majority of the existing methods adopted for SF estimation suffer lack of robustness in some environments and cannot be easily applied for high-speed applications such as autonomous driving. Although some of the available methods are precise, they include high computational costs or require a GPU. The most serious challenge faced in SF estimation is its inability to strike a balance between speed, precision, robustness, and the computational costs. This paper, therefore, aims at proposing a novel sparse scene-flow (stereo-RSSF) method which is highly distinguished in terms of its faster speed, robustness, and precision using the following: stereo calibrated frames, sparse optical flow such as the LKT algorithm, a new inlier detection module based on spatial correlation analysis, epipolar geometry, and modified circular matching techniques. The comparisons made between stereo-RSSF and several advanced methods indicate that this sparse method has significantly higher accuracy than all the other state-of-the-art methods in the points it estimates. In this paper, the effects of each module and hyper-parameters of stereo-RSSF on the performance and running time are analyzed. Stereo-RSSF has also been evaluated on the KITTI test dataset, and the results have been independently verified by the reference group. The code for our implementation of stereo-RSSF is available at: https://github.com/salehierfan/Stereo-RSSF. [ABSTRACT FROM AUTHOR]

Multicomponent seismic technology utilizes the kinematic and dynamic characteristics of reflected P‐waves and converted S‐waves to reduce ambiguity in seismic exploration. The imaging and inversion accuracy of P‐SV‐converted waves are important in determining whether multicomponent seismic exploration can achieve higher exploration accuracy than conventional P‐wave exploration. Pre‐stack inversion of P‐SV‐converted waves requires precise input of P‐SV‐converted wave angle‐domain common‐image gathers. Consequently, the P‐SV‐converted wave angle‐domain common‐image gather extraction accuracy will significantly affect the P‐SV‐converted wave inversion accuracy. However, existing methods for extracting P‐SV‐converted wave angle‐domain common‐image gathers are constrained by issues such as the P‐ and S‐wave crosstalk artefacts, low‐frequency noises and inaccurate calculation of P‐wave incident angles, leading to poor imaging accuracy. We study an angle‐domain cross‐correlation imaging condition and address three key issues based on this condition: the decoupling of P‐ and S‐waves, the separation of up‐going and down‐going waves and the precise calculation of P‐wave incident angles. Our strategies facilitate high‐precision extraction of P‐SV‐converted wave angle‐domain common‐image gathers using elastic wave reverse‐time migration. In this paper, first, we employ the first‐order velocity‐dilatation‐rotation elastic wave equations to decouple P‐ and S‐waves automatically during source and receiver wavefield extrapolations. Second, we calculate the optical flow vectors of P‐ and S‐waves to ensure stable calculations of wave propagation directions. Based on this, we obtain up‐going and down‐going waves of P‐ and S‐waves. Meanwhile, we calculate the incident angle of the source P‐wave using geometric relations. Lastly, we apply the angle‐domain imaging condition to achieve high‐precision extraction of P‐SV‐converted wave angle‐domain common‐image gathers. Model examples demonstrate the effectiveness and advantages of the proposed method. [ABSTRACT FROM AUTHOR]

Background Methods Results Conclusion Hemolysis in mechanical circulatory support systems is currently determined quantitatively. To also locally resolve hemolysis, we are developing a fluorescent hemolysis detection method. This requires a translucent two‐phase blood analog fluid combined with particle image velocimetry, an optical flow field measurement. The blood analog fluid is composed of red blood cell surrogates. However, producing surrogates in sufficient volume is a challenge. We therefore present a high‐volume and high‐concentration production for our surrogates: ghost cells, hemoglobin‐depleted erythrocytes.In the ghost cell production, the hemoglobin is removed by a repeated controlled osmolar lysis. We have varied the solution mixture, centrifugation time, and centrifugation force in order to increase production efficiency. The production is characterized by measurements of output volume, hematocrit, transparency, and rheology of the blood analog fluid.The volume of produced ghost cells was significantly increased, and reproducibility was improved. An average production of 389 mL of ghost cells were achieved per day. Those ghost cells diluted in plasma have a rheology similar to blood while being permeable to light.The volume of ghost cells produced is sufficient for optical measurements as particle image velocimetry in mechanical circulatory support systems. This makes further work on experimental measurements for a locally resolved hemolysis detection possible. [ABSTRACT FROM AUTHOR]

In this study, we analyze mucociliary transport (MCT) by measuring the magnitude and identifying regions of ciliary beats using high-frame-rate microscopic videos. Our methodology, integrating dense optical flow (DOF), connected component labeling (CCL), Butterworth filter, and Fast Fourier Transform (FFT), captures ciliary movement and magnitude. We focus on region extraction, quantification of ciliary activity, and classification of power and recovery strokes in ciliary beat frequency (CBF), which are crucial for evaluating MCT efficiency. Our approach was able to extract the ciliary region semi-automatically, obtain the CBF, and visualize the ciliary movement in each frame. Despite dataset challenges and limited ground truth, our approach shows a promising result for ciliary dynamics research and medical diagnostics. We hope for future open-source datasets with ground-truth ciliary beat patterns to enable developing and evaluating automated ciliary analysis techniques, leading to improved assessment. [ABSTRACT FROM AUTHOR]

Real-time hybrid simulation has gained popularity over the last 20 years as a viable and cost-effective method of testing dynamic systems that cannot be tested using traditional methods. The emergence of multi-axial Real-time Hybrid Simulation (maRTHS) has led to an increase in the allowable fidelity of the numerical and experimental substructures. The testing community can now replicate multiple-degree-of-freedom (MDOF) responses of both substructures and thus can perform more representative tests. However, with this increased fidelity of the substructures comes an increased complexity of controlling these components. Specifically, multi-axial hydraulic actuator assemblages require nonlinear coordinate transformations to derive plant displacements as the force transducers on the actuators are not capable of performing this task directly. Recently, benchmark problems have been provided to the RTHS community in the form of virtual simulations. Virtual simulation refers to a fully virtual testing methodology where numerical and physical components are represented virtually. This approach enables the RTHS community to evaluate various control algorithms without the need to recreate physical components. This project aims to demonstrate the capability of computer vision-based displacement tracking in a realistic virtual simulation of the experimental substructure in avoiding excess nonlinear coordinate transforms. The tracking algorithm utilizing the Lucas-Kanade optical flow method is tested in the virtual simulation environment which is set up using real-time 3D creation engine, Unreal Engine 4 (UE4), and computer graphics software, Blender. This environment interfaces with MATLAB/Simulink, more specifically "Simulation Tool for v-maRTHS benchmark" developed for multi-axial tests. The result of this study establishes a novel framework for applying computer vision-based tracking algorithms and sensing in v-maRTHS simulations using simulated cameras within virtual simulation environments. A computer vision displacement tracking algorithm is developed and optimized to work in tandem with a MIMO PI controller to reduce tracking time delays within 31.25 milliseconds while tracking the nodal displacement and rotation of the frame within a normalized RMSE of 1.24 and 1.10 respectively. [ABSTRACT FROM AUTHOR]

In recent years, the increasing frequency of climate change and extreme weather events has significantly elevated the risk of levee breaches, potentially triggering large-scale floods that threaten surrounding environments and public safety. Rapid and accurate measurement of river surface velocities is crucial for developing effective emergency response plans. Video image velocimetry has emerged as a powerful new approach due to its non-invasive nature, ease of operation, and low cost. This paper introduces the Dynamic Feature Point Pyramid Lucas–Kanade (DFP-P-LK) optical flow algorithm, which employs a feature point dynamic update fusion strategy. The algorithm ensures accurate feature point extraction and reliable tracking through feature point fusion detection and dynamic update mechanisms, enhancing the robustness of optical flow estimation. Based on the DFP-P-LK, we propose a river surface velocity measurement model for rapid levee breach emergency response. This model converts acquired optical flow motion to actual flow velocities using an optical flow-velocity conversion model, providing critical data support for levee breach emergency response. Experimental results show that the method achieves an average measurement error below 15% within the velocity range of 0.43 m/s to 2.06 m/s, demonstrating high practical value and reliability. [ABSTRACT FROM AUTHOR]

In this paper, we proposed Mix-VIO, a monocular and binocular visual-inertial odometry, to address the issue where conventional visual front-end tracking often fails under dynamic lighting and image blur conditions. Mix-VIO adopts a hybrid tracking approach, combining traditional handcrafted tracking techniques with Deep Neural Network (DNN)-based feature extraction and matching pipelines. The system employs deep learning methods for rapid feature point detection, while integrating traditional optical flow methods and deep learning-based sparse feature matching methods to enhance front-end tracking performance under rapid camera motion and environmental illumination changes. In the back-end, we utilize sliding window and bundle adjustment (BA) techniques for local map optimization and pose estimation. We conduct extensive experimental validations of the hybrid feature extraction and matching methods, demonstrating the system's capability to maintain optimal tracking results under illumination changes and image blur. [ABSTRACT FROM AUTHOR]

Data and reports indicate an increasing frequency and intensity of natural disasters worldwide. Buildings play a crucial role in disaster responses and damage assessments, aiding in planning rescue efforts and evaluating losses. Despite advances in applying deep learning to building extraction, challenges remain in handling complex natural disaster scenes and reducing reliance on labeled datasets. Recent advances in satellite video are opening a new avenue for efficient and accurate building extraction research. By thoroughly mining the characteristics of disaster video data, this work provides a new semantic segmentation model for accurate and efficient building extraction based on a limited number of training data, which consists of two parts: the prediction module and the automatic correction module. The prediction module, based on a base encoder–decoder structure, initially extracts buildings using a limited amount of training data that are obtained instantly. Then, the automatic correction module takes the output of the prediction module as input, constructs a criterion for identifying pixels with erroneous semantic information, and uses optical flow values to extract the accurate corresponding semantic information on the corrected frame. The experimental results demonstrate that the proposed method outperforms other methods in accuracy and computational complexity in complicated natural disaster scenes. [ABSTRACT FROM AUTHOR]

The rapid advancement of deep learning and computer vision technologies has given rise to a concerning class of deceptive media, commonly known as deepfakes. This paper addresses emerging trends in deepfakes, including the creation of hyper-realistic facial manipulations, the incorporation of synthesized human voices, and the addition of fabricated subtitles to video content. To effectively combat these multifaceted deepfake threats, we introduce an ensemble-based deepfake detection framework called the "D-Fence" layer. The D-Fence layer consists of two uni-modal classifiers designed to identify tampered facial and vocal elements, as well as two cross-modal classifiers for interactions between Video-Audio and Audio-Text domains to detect deepfakes across multiple modalities. To evaluate the effectiveness of our framework, we introduce two novel adversarial attacks: the "Bogus-in-the-middle" attack, which strategically inserts counterfeit video frames within authentic sequences, and the "Downsampling attack", designed to create deceptive audio. A comparative study of the D-Fence layer against various state-of-the-art multi-modal deepfake detection systems is conducted, demonstrating that our ensemble architecture outperforms existing classifiers. Under diverse adversarial conditions, our D-Fence layer achieves an impressive detection accuracy of 92%, showcasing its ability to detect deepfakes efficiently and reliably. [ABSTRACT FROM AUTHOR]

The objective of this article is to examine potential techniques for suppressing solar heat flow on the optical antenna of a laser communication sensor. Firstly, the characteristics of the geosynchronous Earth orbit's (GEO) space radiation environment are analysed, and a combined passive and active thermal control solution is proposed. Secondly, the temperature distribution of the lasercom sensor under extreme operating conditions is simulated utilising IDEAS-TMG (6.8 NX Series) software, which employs Monte Carlo and radiative heat transfer numerical calculation methods. Finally, a strategy for avoiding direct sunlight around midnight is proposed. The simulation results demonstrated that the thermal control solution and solar avoidance strategy proposed in this paper achieved long-term fine-stable control of the temperature field of the optical antenna, which met the thermal permissible communication hours per daily orbit cycle in excess of 14 h per day. [ABSTRACT FROM AUTHOR]

Objectives: Pixel offset tracking (POT) for optical remote sensing imagery is widely used to invert coseismic deformation fields and monitor landslides. Traditional pixel offset tracking method estimates the displacement of the central pixel by searching for the matching window with the highest correlation, which is computationally inefficient and suffers from inaccurate deformation boundary extraction due to the decoherence effects in the region with dynamical deformation. We introduce the optical flow field model commonly used in computer vision to the pixel offset tracking problem to obtain accurate surface deformation efficiently. Methods: The optical flow field method applicable to optical remote sensing images and the improved inversion algorithm for the time series analysis are proposed to inverse the surface deformation. Experiments on the simulated coseismic deformation fields in Tajikistan are detailed to assess the feasibility and the minimum detectable deformation of the optical flow field method. The advantages of the proposed method over computational cost and deformation boundary extraction accuracy are illustrated by the co-seis‐mic deformation field of the California earthquake and the displacement of the Baige landslide. Further‐ more, the performance on estimating large gradient deformation and the robustness of the improved time series inversion algorithm are discussed by analyzing the time series deformation of the Baige landslide. Results: The results show that compared with the traditional window correlation matching method, the optical flow field method has an offset tracking accuracy of 0.032 pixel, which improves the computational efficiency by about 20 times, and the accuracy of the deformation zone is improved by 25.9%. The time series weighted inversion algorithm reduces the uncertainties in the estimation of east-west and north-south dis‐ placements of optical remote sensing images by 16.2% and 12.4%, respectively. Conclusions: The pro‐ posed method alleviates the pixel offset tracking problem in the boundary region with large gradient deformation [ABSTRACT FROM AUTHOR]

Geotechnical particle image velocimetry (GeoPIV), as a type of digital image correlation (DIC), represents the state-of-the-art methodology for non-contact full-field deformation measurement in geotechnical engineering. Yet, when applying GeoPIV on sand specimens with interests in grain level, the discontinuities detection at grain boundaries remains as a challenge for 2D GeoPIV applications. In order to facilitate the full-field measurement for microscopic study, a method is proposed in this study to realize 2D pixel-level motion calculation using supervised optical flow algorithm with deep networks. Using digital images acquired from direct shear testing, the performance of this approach is demonstrated and compared with the prevailing GeoPIV method. Two series of experiments using small and large displacement modes were conducted, respectively, to demonstrate the method's ability of revealing greater insights on soil behavior at grain level. To verify its accuracy, performance benchmarking of the approach was also conducted. Besides, a method was proposed to evaluate the errors in experimental images to ensure the accuracy and precision. It was demonstrated that the proposed method can achieve accurate pixel-level motion field calculation using images of common size and that the deformation discontinuities among particles can be clearly presented. [ABSTRACT FROM AUTHOR]

High spatial resolution and high accuracy estimation of 3D velocity fields are important for tomographic particle image velocimetry (Tomo-PIV), especially when measuring complex flow fields with delicate 3D structures. However, the widely used cross-correlation-based methods have limited spatial resolution, while the recently developed optical flow-based methods have low robustness and are sensitive to particle volume reconstruction errors. Therefore, 3D velocity estimation methods that simultaneously exhibit high resolution and robustness must be developed. In this study, we propose a novel velocity estimation method for Tomo-PIV measurement using the guided filter-based 3D hybrid variational optical flow (GF-HVOF) method to achieve high spatial resolution and highly accurate measurement of 3D flow field structure. First, we propose a novel L1-norm regularization term based on the Helmholtz decomposition theorem to preserve the divergence and vorticity of the fluid flow. Second, we propose a guided-filter-based constraint term using the result of the cross-correlation-based method as the guided flow field to improve the robustness of the optical flow method. Third, we propose a hybrid constraint term based on particle tracking velocimetry (PTV) method and a spatially weighted data term to reduce the effect of ghost particles and discrete errors generated during the reconstruction of particle volumes. The newly proposed hybrid method combines the advantages of optical-flow-based and cross-correlation-based methods and corrects the flow field using the PTV method. Velocity fields are estimated over synthetic and experimental particle volumes. The results show that the newly proposed GF-HVOF method achieves better performance and greater measurement accuracy than existing 3D fluid motion estimation methods. [ABSTRACT FROM AUTHOR]

Background: Vision-based pulmonary diagnostics present a unique approach for tracking and measuring natural breathing behaviors through remote imaging. While many existing methods correlate chest and diaphragm movements to respiratory behavior, we look at how the direct visualization of thermal CO2 exhale flow patterns can be tracked to directly measure expiratory flow. Methods: In this work, we present a novel method for isolating and extracting turbulent exhale flow signals from thermal image sequences through flow-field prediction and optical flow measurement. The objective of this work is to introduce a respiratory diagnostic tool that can be used to capture and quantify natural breathing, to identify and measure respiratory metrics such as breathing rate, flow, and volume. One of the primary contributions of this work is a method for capturing and measuring natural exhale behaviors that describe individualized pulmonary traits. By monitoring subtle individualized respiratory traits, we can perform secondary analysis to identify unique personalized signatures and abnormalities to gain insight into pulmonary function. In our study, we perform data acquisition within a clinical setting to train an inference model (FieldNet) that predicts flow-fields to quantify observed exhale behaviors over time. Results: Expiratory flow measurements capturing individualized flow signatures from our initial cohort demonstrate how the proposed flow field model can be used to isolate and analyze turbulent exhale behaviors and measure anomalous behavior. Conclusions: Our results illustrate that detailed spatial flow analysis can contribute to unique signatures for identifying patient specific natural breathing behaviors and abnormality detection. This provides the first-step towards a non-contact respiratory technology that directly captures effort-independent behaviors based on the direct measurement of imaged CO2 exhaled airflow patterns. [ABSTRACT FROM AUTHOR]

Optic flow provides information on movement direction and speed during locomotion. Changing the relationship between optic flow and walking speed via training has been shown to influence subsequent distance and hill steepness estimations. Previous research has shown that experience with slow optic flow at a given walking speed was associated with increased effort and distance overestimation in comparison to experiencing with fast optic flow at the same walking speed. Here, we investigated whether exposure to different optic flow speeds relative to gait influences perceptions of leaping and jumping ability. Participants estimated their maximum leaping and jumping ability after exposure to either fast or moderate optic flow at the same walking speed. Those calibrated to fast optic flow estimated farther leaping and jumping abilities than those calibrated to moderate optic flow. Findings suggest that recalibration between optic flow and walking speed may specify an action boundary when calibrated or scaled to actions such as leaping, and possibly, the manipulation of optic flow speed has resulted in a change in the associated anticipated effort for walking a prescribed distance, which in turn influence one's perceived action capabilities for jumping and leaping. [ABSTRACT FROM AUTHOR]

Foraging honey bees, Apis mellifera , need to interact with a range of moving objects, including flowers during windy conditions. Their ability to land on moving flowers, which they demonstrate regularly in nature, would require them to be able to detect, identify and compensate for the flowers' movements. We sought to investigate whether honey bees can distinguish between a stationary and an oscillating flower and whether they display a preference for one or the other. Different sets of individual free-flying honey bees were trained by presenting them with either a stationary or an oscillating flower-like stimulus, which were identical in shape and colour. Subsequently, when prompted to spontaneously choose between two identical flowers, one moving and the other stationary, honey bees exhibited a preference for the moving flower, regardless of whether they were previously trained on the stationary or the moving flower. In a further experiment, a separate set of bees were presented, after being trained, with a choice between stationary or oscillating flowers whose shape differed from the training flower. Here too, bees displayed a significant preference to land on the moving novel-shaped flower. These findings highlight the significance of flower movement to honey bee foraging behaviour. Moving objects like flowers could contribute additional visual salience which would enable easier detection, highlighting motion as an important descriptor used by insects to identify and interact with relevant environmental stimuli. • Honey bees prefer moving flower-like stimuli over stationary counterparts. • The preference for moving stimuli is independent of their shape. • Moving flowers demonstrate unique optic flow characteristics. • Optic flow-driven salience is a likely attractor during honey bee foraging. • Flower movement may also drive decision making in honey bees. [ABSTRACT FROM AUTHOR]

Static and moving objects often occur in real-life videos. Most video object segmentation methods only focus on extracting and exploiting motion cues to perceive moving objects. Once faced with the frames of static objects, the moving object predictors may predict failed results caused by uncertain motion information, such as low-quality optical flow maps. Besides, different sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only consider either the RGB or RGB and optical flow. In this paper, we propose a novel adaptive multi-source predictor for zero-shot video object segmentation (ZVOS). In the static object predictor, the RGB source is converted to depth and static saliency sources, simultaneously. In the moving object predictor, we propose the multi-source fusion structure. First, the spatial importance of each source is highlighted with the help of the interoceptive spatial attention module (ISAM). Second, the motion-enhanced module (MEM) is designed to generate pure foreground motion attention for improving the representation of static and moving features in the decoder. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By using the ISAM, MEM and FPM, the multi-source features are effectively fused. In addition, we put forward an adaptive predictor fusion network (APF) to evaluate the quality of the optical flow map and fuse the predictions from the static object predictor and the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Experiments show that the proposed model outperforms the state-of-the-art methods on three challenging ZVOS benchmarks. And, the static object predictor precisely predicts a high-quality depth map and static saliency map at the same time. [ABSTRACT FROM AUTHOR]

The occlusion problem remains a crucial challenge in optical flow estimation (OFE). Despite the recent significant progress brought about by deep learning, most existing deep learning OFE methods still struggle to handle occlusions; in particular, those based on two frames cannot correctly handle occlusions because occluded regions have no visual correspondences. However, there is still hope in multi-frame settings, which can potentially mitigate the occlusion issue in OFE. Unfortunately, multi-frame OFE (MOFE) remains underexplored, and the limited studies on it are mainly specially designed for pyramid backbones or else obtain the aligned previous frame's features, such as correlation volume and optical flow, through time-consuming backward flow calculation or non-differentiable forward warping transformation. This study proposes an efficient MOFE framework named SplatFlow to address these shortcomings. SplatFlow introduces the differentiable splatting transformation to align the previous frame's motion feature and designs a Final-to-All embedding method to input the aligned motion feature into the current frame's estimation, thus remodeling the existing two-frame backbones. The proposed SplatFlow is efficient yet more accurate, as it can handle occlusions properly. Extensive experimental evaluations show that SplatFlow substantially outperforms all published methods on the KITTI2015 and Sintel benchmarks. Especially on the Sintel benchmark, SplatFlow achieves errors of 1.12 (clean pass) and 2.07 (final pass), with surprisingly significant 19.4% and 16.2% error reductions, respectively, from the previous best results submitted. The code for SplatFlow is available at https://github.com/wwsource/SplatFlow. [ABSTRACT FROM AUTHOR]

When analyzing microscopic time-lapse observations, frame alignment is an essential task to visually understand the morphological and translation dynamics of cells and tissue. While in traditional single-sample microscopy, the region of interest (RoI) is fixed, multi-sample microscopy often uses a single microscope that scans multiple samples over a long period of time by laterally relocating the sample stage. Hence, the relocation of the optics induces a statistical RoI offset and can introduce jitter as well as drift, which results in a misaligned RoI for each sample's time-lapse observation (stage drift). We introduce a robust approach to automatically align all frames within a time-lapse observation and compensate for frame drift. In this study, we present a sub-pixel precise alignment approach based on recurrent all-pairs field transforms (RAFT); a deep network architecture for optical flow. We show that the RAFT model pre-trained on the Sintel dataset performed with near perfect precision for registration tasks on a set of ten contextually unrelated time-lapse observations containing 250 frames each. Our approach is robust for elastically undistorted and translation displaced (x,y) microscopic time-lapse observations and was tested on multiple samples with varying cell density, obtained using different devices. The approach only performed well for registration and not for tracking of the individual image components like cells and contaminants. We provide an open-source command-line application that corrects for stage drift and jitter. [ABSTRACT FROM AUTHOR]

Purpose: Wearable ultrasound devices can be used to continuously monitor muscle activity. One possible application is to provide real-time feedback during physiotherapy, to show a patient whether an exercise is performed correctly. Algorithms which automatically analyze the data can be of importance to overcome the need for manual assessment and annotations and speed up evaluations especially when considering real-time video sequences. They even could be used to present feedback in an understandable manner to patients in a home-use scenario. The following work investigates three deep learning based segmentation approaches for abdominal muscles in ultrasound videos during a segmental stabilizing exercise. The segmentations are used to automatically classify the contraction state of the muscles. Methods: The first approach employs a simple 2D network, while the remaining two integrate the time information from the videos either via additional tracking or directly into the network architecture. The contraction state is determined by comparing measures such as muscle thickness and center of mass between rest and exercise. A retrospective analysis is conducted but also a real-time scenario is simulated, where classification is performed during exercise. Results: Using the proposed segmentation algorithms, 71% of the muscle states are classified correctly in the retrospective analysis in comparison to 90% accuracy with manual reference segmentation. For the real-time approach the majority of given feedback during exercise is correct when the retrospective analysis had come to the correct result, too. Conclusion: Both retrospective and real-time analysis prove to be feasible. While no substantial differences between the algorithms were observed regarding classification, the networks incorporating the time information showed temporally more consistent segmentations. Limitations of the approaches as well as reasons for failing cases in segmentation, classification and real-time assessment are discussed and requirements regarding image quality and hardware design are derived. [ABSTRACT FROM AUTHOR]

This study systematically investigates the flow birefringence of cellulose nanocrystal (CNC) suspensions. The aim is to clarify the importance of the stress component along the camera's optical axis in the stress-optic law (SOL), which describes the relationship between birefringence, the retardation of transmitted polarized light, and the stress field. More than 100 datasets pertaining to the retardation of CNC suspensions (concentrations of 0.1, 0.3, 0.5, and 1.0 wt%) in a laminar flow field within a rectangular channel (aspect ratios of 0.1, 1, and 3) are systematically obtained. The measured retardation data are compared with the predictions given by the conventional SOL excluding the stress component along the camera's optical axis and by the SOL including these components as second-order terms (2nd-order SOL). The results show that the 2nd-order SOL gives a significantly better agreement with the measurements. Based on the 2nd-order SOL, the retardation at the center of the channel, where the effect of the stress component along the camera's optical axis is most pronounced, is predicted to be proportional to the square of the flow rate, which agrees with the experimental data. The results confirm the importance of considering the stress component along the camera's optical axis in the flow birefringence of CNC suspensions at high flow rates, even for quasi-two-dimensional channel flow. [ABSTRACT FROM AUTHOR]

High-speed turbulence induces significant aero-optical effects that severely disrupt the functionality of imaging systems of hypersonic vehicles. In this study, the aero-optical correction of various jet cooling modes is investigated using a Terminal High Altitude Area Defense (THAAD)-like seeker model and the imaging impact of high-speed flow field and flow control on the optical window is analyzed by the Delayed Detached Eddy Simulation (DDES) method. The findings reveal that a jet mode parallel to the window exhibits better cooling effectiveness compared to a perpendicular jet mode along the body axis; however, it introduces additional wavefront distortion, leading to degraded imaging quality. Although micro-vortex generators (MVGs) can reduce density fluctuations near the window from a refractive index perspective, they do not effectively mitigate wavefront distortion or improve window cooling efficiency. Finally, incorporating suction control, a comprehensive flow control solution, significantly improves the flow field structure near the window, resulting in a more uniform temperature distribution and reduced wavefront distortion. Applying this flow control method results in a 14.7% reduction in wavefront distortion at 3 Ma and an approximately 20% maximum value reduction at 5 Ma. This study proposes a novel and comprehensive flow control method to effectively mitigate the aero-optical effect in hypersonic flows, providing a new avenue for subsequent researchers in this field. [ABSTRACT FROM AUTHOR]

Learned video compression has drawn great attention and shown promising compression performance recently. In this article, we focus on the two components in the learned video compression framework, the conditional entropy model and quality enhancement module, to improve compression performance. Specifically, we propose an adaptive spatial-temporal entropy model for image, motion, and residual compression, which introduces a temporal prior to reduce temporal redundancy of latents and an additional modulated mask to evaluate the similarity and perform refinement. In addition, a quality enhancement module is proposed for predicted frame and reconstructed frame to improve frame quality and reduce the bitrate cost of residual coding. The module reuses decoded optical flow as a motion prior and utilizes deformable convolution to mine high-quality information from the reference frame in a bit-free manner. The two proposed coding tools are integrated into a pixel-domain residual coding–based compression framework to evaluate their effectiveness. Experimental results demonstrate that our framework achieves competitive compression performance in the low-delay scenario compared with recent learning-based methods and traditional H.265/HEVC in terms of Peak Signal-to-Noise Ratio (PSNR) and Multi-Scale Structural Similarity Index (MS-SSIM). The code is available at OpenLVC. [ABSTRACT FROM AUTHOR]

Multi-modal human action segmentation is a critical and challenging task with a wide range of applications. Nowadays, the majority of approaches concentrate on the fusion of dense signals (i.e., RGB, optical flow, and depth maps). However, the potential contributions of sparse IoT sensor signals, which can be crucial for achieving accurate recognition, have not been fully explored. To make up for this, we introduce a Sparse signal-guided Transformer (SigFormer) to combine both dense and sparse signals. We employ mask attention to fuse localized features by constraining cross-attention within the regions where sparse signals are valid. However, since sparse signals are discrete, they lack sufficient information about the temporal action boundaries. Therefore, in SigFormer, we propose to emphasize the boundary information at two stages to alleviate this problem. In the first feature extraction stage, we introduce an intermediate bottleneck module to jointly learn both category and boundary features of each dense modality through the inner loss functions. After the fusion of dense modalities and sparse signals, we then devise a two-branch architecture that explicitly models the interrelationship between action category and temporal boundary. Experimental results demonstrate that SigFormer outperforms the state-of-the-art approaches on a multi-modal action segmentation dataset from real industrial environments, reaching an outstanding F1 score of 0.958. The codes and pre-trained models have been made available at https://github.com/LIUQI-creat/SigFormer. [ABSTRACT FROM AUTHOR]

In minimally invasive surgery, endoscopes or laparoscopes equipped with miniature cameras and tools are used to enter the human body for therapeutic purposes through small incisions or natural cavities. However, in clinical operating environments, endoscopic images often suffer from challenges such as low texture, uneven illumination, and non-rigid structures, which affect feature observation and extraction. This can severely impact surgical navigation or clinical diagnosis due to missing feature points in endoscopic images, leading to treatment and postoperative recovery issues for patients. To address these challenges, this paper introduces, for the first time, a Cross-Channel Multi-Modal Adaptive Spatial Feature Fusion (ASFF) module based on the lightweight architecture of EfficientViT. Additionally, a novel lightweight feature extraction and matching network based on attention mechanism is proposed. This network dynamically adjusts attention weights for cross-modal information from grayscale images and optical flow images through a dual-branch Siamese network. It extracts static and dynamic information features ranging from low-level to high-level, and from local to global, ensuring robust feature extraction across different widths, noise levels, and blur scenarios. Global and local matching are performed through a multi-level cascaded attention mechanism, with cross-channel attention introduced to simultaneously extract low-level and high-level features. Extensive ablation experiments and comparative studies are conducted on the HyperKvasir, EAD, M2caiSeg, CVC-ClinicDB, and UCL synthetic datasets. Experimental results demonstrate that the proposed network improves upon the baseline EfficientViT-B3 model by 75.4% in accuracy (Acc), while also enhancing runtime performance and storage efficiency. When compared with the complex DenseDescriptor feature extraction network, the difference in Acc is less than 7.22%, and IoU calculation results on specific datasets outperform complex dense models. Furthermore, this method increases the F1 score by 33.2% and accelerates runtime by 70.2%. It is noteworthy that the speed of CMMCAN surpasses that of comparative lightweight models, with feature extraction and matching performance comparable to existing complex models but with faster speed and higher cost-effectiveness. [ABSTRACT FROM AUTHOR]

Micro-expressions are instantaneous flashes of facial expressions that reveal a person's true feelings and emotions. Micro-expression recognition (MER) is challenging due to its low motion intensity, short duration, and the limited number of publicly available samples. Although the present MER methods have achieved great progress, they face the problems of a large number of training parameters and insufficient feature extraction ability. In this paper, we propose a lightweight network MFE-Net with Res-blocks to extract multi-scale features for MER. To extract more valuable features, we incorporate Squeeze-and-Excitation attention and multi-headed self-attention mechanisms in our MFE-Net. The proposed network is used for learning features from three optical flow features (i.e. optical strain, horizontal and vertical optical flow images) which are calculated from the onset and apex frames. We employ the LOSO cross-validation strategy to conduct experiments on CASME II and the composite dataset selected by MEGC2019, respectively. The extensive experimental results demonstrate the viability and effectiveness of our method. [ABSTRACT FROM AUTHOR]

Underwater target motion estimation is a challenge for ocean military and scientific research. In this work, we propose a method based on the combination of polarization imaging and optical flow for turbid underwater target detection. Polarization imaging can reduce the influence of backscattered light and obtain high-quality images underwater. The optical flow shows the motion and structural information of the target. We use polarized optical flow to obtain the optical flow field and estimate the target motion. The experimental results of different targets under varying water turbidity levels illustrate that our method is realizable and robust. The precision is verified by comparing the results with the precise displacement data and calculating two error measures. The proposed method based on polarized optical flow can obtain accurate displacement information and a good recognition effect. Moving target segmentation based on the Otsu method further proves the superiority of the polarized optical flow under turbid water. This study is valuable for target detection and motion estimation in scattering environments. [ABSTRACT FROM AUTHOR]

Convective dynamics in a supercell thunderstorm, a volcanic eruption, and two pyrocumulonimbus (pyroCb) events are compared by computing cloud‐top divergence (CTD) with an optical flow technique called Deepflow. Visible 0.64‐μm imagery sequences from Geostationary Operational Environmental Satellites (GOES)‐R series Advanced Baseline Imager (ABI) are used as input into the optical flow algorithm. CTD is computed after post‐processing of the retrieved motions. Analysis is performed on specific image times, as well as the full time series of each case. Multiple CTD‐based parameters, such as the maximum and the two‐dimensional area exceeding a specified CTD threshold, are examined along with the optical flow‐retrieved wind speed. CTD is shown to accurately and quantitatively represent the behavior and magnitude of different deep convective phenomena, including distinguishing between convective pulses within each individual event. CTD captures updraft intensification as well as differences in convective activity between two pyroCb events and individual updraft pulses occurring within a single pyroCb event. Finally, the characteristics of high‐altitude smoke plumes injected by two separate pyroCb pulses are linked to CTD using ultraviolet aerosol index and satellite imagery. Optical flow‐derived parameters can therefore be applied to individual pyroCbs in real‐time, with potential to characterize pyroCb smoke source inputs for downstream smoke modeling applications and to facilitate future tools supporting air quality modeling and firefighting efforts. Plain Language Summary: Under certain weather conditions, wildfires can generate pyrocumulonimbus, which are deep, convective storms that inject smoke particles high into the atmosphere, resulting in significant aviation and climate impacts. This research tracks cloud‐top motion using computer vision algorithms and satellite data to compare the dynamics of pyrocumulonimbus with a volcanic eruption and supercell thunderstorm, which represent similar deep convective phenomena. The primary goal is to quantify updraft magnitude, duration, and evolution from space. Results show that cloud‐top motion accurately captures variations in updraft magnitude over time and distinguishes between different types of updrafts. Cloud‐top motion is likely more effective than traditional techniques at distinguishing individual, high‐altitude smoke injections from pyrocumulonimbus activity over specific fires. This research lays a foundation for improved smoke source inputs for downstream smoke modeling applications. Key Points: General dynamical characteristics for pyrocumulonimbus (pyroCb), volcano, and supercell cases are compared using satellite‐retrieved cloud‐top divergenceCloud‐top divergence time series analysis distinguishes individual convective pulses of pyroCb and volcano casesCloud‐top divergence is potentially related to the magnitude of pyroCb smoke plume injection height and mass [ABSTRACT FROM AUTHOR]

This research proposes a sensor for tracking the motion of a human head via optical tactile sensing. It implements the use of a fibrescope a non-metal alternative to a webcam. Previous works have included robotics grippers to mimic the sensory features of human skin, that used monochrome cameras and depth cameras. Tactile sensing has shown advantages in feedback-based interactions between robots and their environment. The methodology in this paper is utilised to track motion of objects in physical contact with these sensors to replace external camera based motion capture systems. Our immediate application is related to detection of human head motion during radiotherapy procedures. The motion was analysed in two degrees of freedom, respective to the tactile sensor (translational in z-axis, and rotational around y-axis), to produce repeatable and accurate results. The movements were stimulated by a robot arm, which also provided ground truth values from its end- effector. The fibrescope was implemented to ensure the device's compatibility with electromagnetic waves. The cameras and the ground truth values were time synchronised using robotics operating systems tools. Image processing methods were compared between grayscale and binary image sequences, followed by motion tracking estimation using deterministic approaches. These included Lukas-Kanade Optical Flow and Simple Blob Detection, by OpenCV. The results showed that the grayscale image processing along with the Lukas- Kanade algorithm for motion tracking can produce better tracking abilities, although further exploration to improve the accuracy is still required. [ABSTRACT FROM AUTHOR]

The present study discusses the effect of symmetric and asymmetric grafting on the surface of CNCs (cellulose nanocrystals) on their dispersion properties using dialkyl azetidinium salts. Three dialkylamine of different size and chain length were successfully grafted to the sulfate groups on the surface of CNCs by conjugation of azetidinium salts. The coupling process resulted in the formation of 2-hydroxypropyl-N-dialkylamine conjugated to the CNC sulfate groups abbreviated as C n -N-C m -Prop-2-OH-CNC, where m, n are the number of carbons in the alkyl groups, each with a total of m + n = 12 , with (m , n) = (11 , 1) ; (9 , 3) ; (6 , 6) . Molecular dynamics simulations were used to assess the probable morphology of the grafted chains and the interaction potential between CNCs. Steady shear simultaneously combined with polarized light imaging and oscillatory shear rheological measurements were used to evaluate for the first time the impact of the CNC surface modifications on their dispersion flow and optical properties. Overall, the results show that the different linker topologies could effectively promote different types of aggregation morphologies based on the size of the linker, their flexibility and their most probable conformation. [ABSTRACT FROM AUTHOR]

A core challenge in perception is recognizing objects across the highly variable retinal input that occurs when objects are viewed from different directions (e.g. front versus side views). It has long been known that certain views are of particular importance, but it remains unclear why. We reasoned that characterizing the computations underlying visual comparisons between objects could explain the privileged status of certain qualitatively special views. We measured pose discrimination for a wide range of objects, finding large variations in performance depending on the object and the viewing angle, with front and back views yielding particularly good discrimination. Strikingly, a simple and biologically plausible computational model based on measuring the projected three-dimensional optical flow between views of objects accurately predicted both successes and failures of discrimination performance. This provides a computational account of why certain views have a privileged status. [ABSTRACT FROM AUTHOR]

Event-based cameras, also known as biologically inspired visual sensors, are capable of capturing real-time scene changes efficiently. Unlike traditional frame-based cameras, event cameras solely report triggered pixel-level brightness changes which are referred to as events. Event-based cameras show many advantages such as high temporal resolution, low latency, and high dynamic range, making them very attractive in robotics and computer vision, especially in challenging scenarios that are too demanding for traditional cameras. In this paper, we provide a comprehensive overview of the emerging field of event-based vision, focusing on the operation principle, sampling mechanisms, and algorithms that take advantage of their superior features. We also delve into the various tasks for which event cameras are utilized, such as object tracking, optical flow estimation, 3D reconstruction, SLAM, image reconstruction, and recognition. Additionally, we highlight the challenges and future opportunities for event cameras, seeking a more efficient way for machines to perceive and interact with the world. [ABSTRACT FROM AUTHOR]

Edge computing is essential to handle increasing data volumes and processing capacities. It provides real-time and secure data processing near data sources, like smart devices, alleviating cloud computing energy use, and saving network bandwidth. Specialized accelerators, like GPUs and FPGAs, are vital for low-latency edge computing but the requirements to customized code for different hardware and vendors suppose important compatibility issues. This paper evaluates the potential of SYCL in addressing code portability issues encountered in edge computing. We employed the Polybench suite to compare various SYCL implementations, specifically DPC++ and AdaptiveCpp, with the native solution, CUDA. The disparity between SYCL implementations was negligible, at just 5%. Furthermore, we evaluated SYCL in the context of specific edge computing applications such as video processing using three different optical flow algorithms. The results revealed a slight performance gap of 3% when transitioning from CUDA to SYCL. Upon evaluating energy consumption, the observed difference ranged from ± 10 % , depending on the application utilized. These gaps are the price one may need to pay when achieving the ability to successfully run the same code on two distinct edge boards. These findings underscore SYCL's capacity to increase productivity in terms of development costs and facilitate IoT deployment without being locked into a particular platform or manufacturer. [ABSTRACT FROM AUTHOR]

Purpose: This study aimed to investigate differences in microvasculature dropout (MvD) between the superior and inferior hemispheres in glaucoma patients. Study design: Retrospective and cross-sectional. Methods: Fifty-eight eyes of 58 open-angle glaucoma patients (age 61.12 ± 10.19 years, mean deviation − 7.32 ± 6.36 dB) were included. MvD was detected with en face images from swept-source optical coherence tomography angiography. Blood flow at the optic nerve head was measured with laser speckle flowgraphy, represented as the mean blur rate in tissue (MBRT). Logistic and linear regression models adjusted for age, intraocular pressure, axial length, and circumpapillary retinal nerve fiber layer thickness were used to investigate the relationship between various factors and MvD angle in each hemisphere. Results: The presence of inferior MvD was related to peripapillary atrophy-β area (odds ratio = 14.10 [2.49–234.00], P = 0.019). Superior MvD angle was significantly related to MBRT in the superior quadrant (β = −0.31 [− 0.60 – −0.02], P = 0.037). Inferior MvD angle was significantly related to peripapillary atrophy-β area (β = 0.49 [0.21–0.77], P = 0.001). Conclusions: Only superior MvD demonstrated a significant relationship with reduced ocular blood flow. In contrast, inferior MvD was associated with mechanical stress. These findings may suggest a potential difference in pathophysiology between superior and inferior MvD. [ABSTRACT FROM AUTHOR]

Parallax processing and structure preservation have long been important and challenging tasks in image stitching. In this paper, an image stitching method based on sliding camera to eliminate perspective deformation and asymmetric optical flow to solve parallax is proposed. By maintaining the viewpoint of two input images in the mosaic non-overlapping area and creating a virtual camera by interpolation in the overlapping area, the viewpoint is gradually transformed from one to another so as to complete the smooth transition of the two image viewpoints and reduce perspective deformation. Two coarsely aligned warped images are generated with the help of a global projection plane. After that, the optical flow propagation and gradient descent method are used to quickly calculate the bidirectional asymmetric optical flow between the two warped images, and the optical-flow-based method is used to further align the two warped images to reduce parallax. In the image blending, the softmax function and registration error are used to adjust the width of the blending area, further eliminating ghosting and reducing parallax. Finally, by comparing our method with APAP, AANAP, SPHP, SPW, TFT, and REW, it has been proven that our method can not only effectively solve perspective deformation, but also gives more natural transitions between images. At the same time, our method can robustly reduce local misalignment in various scenarios, with higher structural similarity index. A scoring method combining subjective and objective evaluations of perspective deformation, local alignment and runtime is defined and used to rate all methods, where our method ranks first. [ABSTRACT FROM AUTHOR]

The consecutive submesoscale sea surface processes observed by an unmanned aerial vehicle (UAV) were used to decompose into spatial waves and current features. For the image decomposition, the Fast and Adaptive Multidimensional Empirical Mode Decomposition (FA-MEMD) method was employed to disintegrate multicomponent signals identified in sea surface optical images into modulated signals characterized by their amplitudes and frequencies. These signals, referred to as Bidimensional Intrinsic Mode Functions (BIMFs), represent the inherent two-dimensional oscillatory patterns within sea surface optical data. The BIMFs, separated into seven modes and a residual component, were subsequently reconstructed based on the physical frequencies. A two-dimensional Fast Fourier Transform (2D FFT) for each high-frequency mode was used for surface wave analysis to illustrate the wave characteristics. Wavenumbers (Kx, Ky) ranging between 0.01–0.1 radm−1 and wave directions predominantly in the northeastward direction were identified from the spectral peak ranges. The Optical Flow (OF) algorithm was applied to the remaining consecutive low-frequency modes as the current signal under 0.1 Hz for surface current analysis and to estimate a current field with a 1 m spatial resolution. The accuracy of currents in the overall region was validated with in situ drifter measurements, showing an R-squared (R2) value of 0.80 and an average root-mean-square error (RMSE) of 0.03 ms−1. This study proposes a novel framework for analyzing individual sea surface dynamical processes acquired from high-resolution UAV imagery using a multidimensional signal decomposition method specialized in nonlinear and nonstationary data analysis. [ABSTRACT FROM AUTHOR]

Background: The placenta is a transient organ critical for fetal development. Disruptions of normal placental functions can impact health throughout an individual's entire life. Although being recognized by the NIH Human Placenta Project as an important organ, the placenta remains understudied, partly because of a lack of non-invasive tools for longitudinally evaluation for key aspects of placental functionalities. Objective: Our goal is to create a non-invasive preclinical imaging pipeline that can longitudinally probe murine placental health in vivo. We use advanced imaging processing schemes to establish functional biomarkers for non-invasive longitudinal evaluation of placental development. Methodology: We implement dynamic contrast enhancement magnetic resonance imaging (DCE-MRI) and analysis pipeline to quantify uterine contraction and placental perfusion dynamics. We use optic flow and time-frequency analysis to quantify and characterize contraction-related placental motion. Our novel imaging and analysis pipeline uses subcutaneous administration of gadolinium for steepest slope-based perfusion evaluation, enabling non-invasive longitudinal monitoring. Results: We demonstrate that the placenta exhibits spatially asymmetric contractile motion that develops from E14.5 to E17.5. Additionally, we see that placental perfusion, perfusion delivery rate, and substrate delivery all increase from E14.5 to E17.5, with the High Perfusion Chamber (HPC) leading the placental changes that occur from E14.5 to E17.5 Discussion: We advance the placental perfusion chamber paradigm with a novel, physiologically based threshold model for chamber localization and demonstrate spatially varying placental chambers using multiple functional metrics that assess mouse placental development and remodeling throughout gestation. Conclusion: Our pipeline enables the non-invasive, longitudinal assessment of multiple placenta functions from a single imaging session. Our pipeline serves as a key toolbox for advancing research in mouse models of placental disease and disorder. [ABSTRACT FROM AUTHOR]

Manual intervention in the self‐assembly of soft matter to obtain a desired defect structure is a complex but significant project. Specifically, the flow and optical properties of liquid crystal (LC) materials have become the most important platform for studying topological structures in ordered physical systems. The regulation of liquid crystals (LCs) is largely limited to established means such as temperature, electric field, magnetic field, etc., however, a new effective strategy by mechanical stress is desirable but elusive. Here, using disclination controlled by stress, topologically continuous superstructure is generated to apply to touch display and wideband vector beam generator. This work demonstrates a new approach for direct controllable manipulation of disclination, unlocking the long‐sought full potential of topological defect, thus paving the way for optical and dynamic applications. [ABSTRACT FROM AUTHOR]