Search

Multi-Object Tracking, also known as Multi-Target Tracking, is an important area of computer vision with various applications in different domains. The advent of deep learning has had a profound impact on this field, forcing researchers to explore innovative avenues. Deep learning methods have become the cornerstone of today's state-of-the-art solutions, consistently delivering exceptional tracking results. However, the significant computational demands of deep learning models require powerful hardware resources that do not always match real-time tracking requirements, limiting their practical applicability in real-world scenarios. Thus, there is an imperative to strike a balance by merging robust deep learning strategies with conventional approaches to enable more accessible, cost-effective solutions that meet real-time requirements. This paper embarks on this endeavor by presenting a hybrid strategy for real-time multi-target tracking. It effectively combines a classical optical flow algorithm with a deep learning architecture tailored for human crowd tracking systems. This hybrid approach achieves a commendable balance between tracking accuracy and computational efficiency. The proposed architecture, subjected to extensive experimentation in various settings, demonstrated notable results, achieving a Mean Object Tracking Accuracy (MOTA) of 0.608. This level of performance placed it as the highest ranking solution on the MOT15 benchmark, surpassing the state-of-the-art benchmark of 0.549, and consistently ranked among the superior models on the MOT17 and MOT20 benchmarks. Additionally, the incorporation of the optical flow phase resulted in a substantial reduction in processing time, nearly halving the duration, while simultaneously maintaining accuracy levels comparable to established techniques. [ABSTRACT FROM AUTHOR]

The growing use of Unmanned Aerial Vehicles (UAVs) raises the need to improve their autonomous navigation capabilities. Visual odometry allows for dispensing positioning systems, such as GPS, especially on indoor flights. This paper reports an effort toward UAV autonomous navigation by proposing a translational velocity observer based on inertial and visual measurements for a quadrotor. The proposed observer complementarily fuses available measurements from different domains and is synthesized following the Immersion and Invariance observer design technique. A formal Lyapunov-based observer error convergence to zero is provided. The proposed observer algorithm is evaluated using numerical simulations in the Parrot Mambo Minidrone App from Simulink-Matlab. [ABSTRACT FROM AUTHOR]

Compressive sensing provides a promising sampling paradigm for video acquisition for resource‐limited sensor applications. However, the reconstruction of original video signals from sub‐sampled measurements is still a great challenge. To exploit the temporal redundancies within videos during the recovery, previous works tend to perform alignment on initial reconstructions, which are too coarse to provide accurate motion estimations. To solve this problem, the authors propose a novel reconstruction network, named TSRN, for compressive video sensing. Specifically, the authors utilise a number of stacked temporal shift reconstruction blocks (TSRBs) to enhance the initial reconstruction progressively. Each TSRB could learn the temporal structures by exchanging information with last and next time step, and no additional computations is imposed on the network compared to regular 2D convolutions due to the high efficiency of temporal shift operations. After the enhancement, a bidirectional alignment module to build accurate temporal dependencies directly with the help of optical flows is employed. Different from previous methods that only extract supplementary information from the key frames, the proposed alignment module can receive temporal information from the whole video sequence via bidirectional propagations, thus yielding better performance. Experimental results verify the superiority of the proposed method over other state‐of‐the‐art approaches quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

FlowDenoising is a new approach to noise reduction in biological volumes obtained with three-dimensional electron microscopy (3DEM). Its abilities to enhance the structural features stem from the fact that an anisotropic Gaussian filtering is steered according to the local structures. To this end, the Optical Flow (OF) among consecutive slices is estimated, which is the most computationally expensive step in this approach. In this article, a hybrid CPU/GPU implementation of FlowDenoising is introduced and evaluated. It exploits parallel computing by distributing the workload among multiple cores and takes advantage of the massive processing in GPUs to accelerate the OF estimation. The hybrid implementation provides remarkable speed-up factors and an important reduction of the processing time, which is particularly relevant for the denoising of huge volumes typically found in 3DEM. [ABSTRACT FROM AUTHOR]

Existing homography and optical flow methods are erroneous in challenging scenes, such as fog, rain, night, and snow because the basic assumptions such as brightness and gradient constancy are broken. To address this issue, we present an unsupervised learning approach that fuses gyroscope into homography and optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Second, we design a self-guided fusion module (SGF) to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. Meanwhile, we propose a homography decoder module (HD) to combine gyro field and intermediate results of SGF to produce the homography. To the best of our knowledge, this is the first deep learning framework that fuses gyroscope data and image content for both deep homography and optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-the-art methods in both regular and challenging scenes. The code and dataset are available at https://github.com/lhaippp/GyroFlowPlus. [ABSTRACT FROM AUTHOR]

Optic flow provides useful information in service of spatial navigation. However, whether brain networks supporting these two functions overlap is still unclear. Here we used Activation Likelihood Estimation (ALE) to assess the correspondence between brain correlates of optic flow processing and spatial navigation and their specific neural activations. Since computational and connectivity evidence suggests that visual input from optic flow provides information mainly during egocentric navigation, we further tested the correspondence between brain correlates of optic flow processing and that of both egocentric and allocentric navigation. Optic flow processing shared activation with egocentric (but not allocentric) navigation in the anterior precuneus, suggesting its role in providing information about self-motion, as derived from the analysis of optic flow, in service of egocentric navigation. We further documented that optic flow perception and navigation are partially segregated into two functional and anatomical networks, i.e., the dorsal and the ventromedial networks. Present results point to a dynamic interplay between the dorsal and ventral visual pathways aimed at coordinating visually guided navigation in the environment. [ABSTRACT FROM AUTHOR]

Velocity field quantification for high-speed flows is of fundamental importance to understand flow dynamics, turbulence, and flow–structure interactions. Optical velocimetry techniques commonly provide sparse information in the flows. Dense fields of velocity vectors with high spatial resolutions are indispensable for detailed analysis of complex motion patterns and accurate motion tracking within the field of view. In the present work, two-dimensional (2D) Rayleigh scattering imaging (RSI) at a rate of 10- to 100-kHz was utilized to quantify the high-speed flow velocity by employing deep learning-based optical flow analysis, along with density and temperature fields from Rayleigh scattering intensity profiles. High-speed Rayleigh scattering images are highly spatially resolved, have smooth gradients without intensity discontinuities, and precisely track key features of the flows. The deep learning-based optical flow method utilizes recurrent neural network architecture to extract the per-pixel features of both input images, calculate correlation from all pairs of the features, and get training by recurrently updating the optical flow. 2D instantaneous velocity fields of both nonreacting and reacting flows measured by RSI were obtained from deep learning-based optical flow analysis, thus extending RSI as a non-intrusive, nonseeded, and multiscalar measurement technique of high-speed nonreacting and reacting flows. [ABSTRACT FROM AUTHOR]

Background: Threat and individual differences in threat-processing bias perception of stimuli in the environment. Yet, their effect on perception of one's own (body-based) self-motion in space is unknown. Here, we tested the effects of threat on self-motion perception using a multisensory motion simulator with concurrent threatening or neutral auditory stimuli. Results: Strikingly, threat had opposite effects on vestibular and visual self-motion perception, leading to overestimation of vestibular, but underestimation of visual self-motions. Trait anxiety tended to be associated with an enhanced effect of threat on estimates of self-motion for both modalities. Conclusions: Enhanced vestibular perception under threat might stem from shared neural substrates with emotional processing, whereas diminished visual self-motion perception may indicate that a threatening stimulus diverts attention away from optic flow integration. Thus, threat induces modality-specific biases in everyday experiences of self-motion. [ABSTRACT FROM AUTHOR]

Accurate estimation of global winds is crucial for various scientific and practical applications, such as global chemical transport modeling and numerical weather prediction. One valuable source of wind measurements is atmospheric motion vectors (AMVs), which play a vital role in the global observing system and numerical weather prediction models. However, errors in AMV retrievals need to be addressed before their assimilation into data assimilation systems, as they can affect the accuracy of outputs. An assessment of the bias and uncertainty in passive-sensor AMVs can be done by comparing them with information from independent sources such as active-sensor winds. In this paper, we examine the benefit and performance of a colocation scheme using independent and sparse lidar wind observations as a dependent variable in a supervised machine learning model. We demonstrate the feasibility and performance of this approach in an observing system simulation experiment (OSSE) framework, with reference geophysical state data obtained from high-resolution Weather Research and Forecasting (WRF) model simulations of three different weather events. Lidar wind data are typically available in only one direction, and our study demonstrates that this single component of wind in high-precision active-sensor data can be leveraged (via a machine learning algorithm to model the conditional mean) to reduce the bias in the passive-sensor winds. Further, this active-sensor wind information can be leveraged through an algorithm that models the conditional quantiles to produce stable estimates of the prediction intervals, which are helpful in the design and application of error analysis, such as quality filters. [ABSTRACT FROM AUTHOR]

The sky may seem big enough for two flying vehicles to collide, but the facts show that mid-air collisions still occur occasionally and are a significant concern. Pilots learn manual tactics to avoid collisions, such as see-and-avoid, but these rules have limitations. Automated solutions have reduced collisions, but these technologies are not mandatory in all countries or airspaces, and they are expensive. These problems have prompted researchers to continue the search for low-cost solutions. One attractive solution is to use computer vision to detect obstacles in the air due to its reduced cost and weight. A well-trained deep learning solution is appealing because object detection is fast in most cases, but it relies entirely on the training data set. The algorithm chosen for this study is optical flow. The optical flow vectors can help us to separate the motion caused by camera motion from the motion caused by incoming objects without relying on training data. This paper describes the development of an optical flow-based airborne obstacle detection algorithm to avoid mid-air collisions. The approach uses the visual information from a monocular camera and detects the obstacles using morphological filters, optical flow, focus of expansion, and a data clustering algorithm. The proposal was evaluated using realistic vision data obtained with a self-developed simulator. The simulator provides different environments, trajectories, and altitudes of flying objects. The results showed that the optical flow-based algorithm detected all incoming obstacles along their trajectories in the experiments. The results showed an F-score greater than 75% and a good balance between precision and recall. [ABSTRACT FROM AUTHOR]

It is a challenging and meaningful task to carry out UAV-based livestock monitoring in high-altitude (more than 4500 m on average) and cold regions (annual average – 4 °C) on the Qinghai Tibet Plateau. The purpose of artificial intelligence (AI) is to execute automated tasks and to solve practical problems in actual applications by combining the software technology with the hardware carrier to create integrated advanced devices. Only in this way, the maximum value of AI could be realized. In this paper, a real-time tracking system with dynamic target tracking ability is proposed. It is developed based on the tracking-by-detection architecture using YOLOv7 and Deep SORT algorithms for target detection and tracking, respectively. In response to the problems encountered in the tracking process of complex and dense scenes, our work (1) Uses optical flow to compensate the Kalman filter, to solve the problem of mismatch between the target bounding box predicted by the Kalman filter (KF) and the input when the target detection in the current frame is complex, thereby improving the prediction accuracy; (2) Using a low confidence trajectory filtering method to reduce false positive trajectories generated by Deep SORT, thereby mitigating the impact of unreliable detection on target tracking. (3) A visual servo controller has been designed for the Unmanned Aerial Vehicle (UAV) to reduce the impact of rapid movement on tracking and ensure that the target is always within the field of view of the UAV camera, thereby achieving automatic tracking tasks. Finally, the system was tested using Tibetan yaks on the Qinghai Tibet Plateau as tracking targets, and the results showed that the system has real-time multi tracking ability and ideal visual servo effect in complex and dense scenes. [ABSTRACT FROM AUTHOR]

In order for detection of a fire in fields, it is effective to detect smoke since it often behaves as a precursor of the fire. One preferable way for early detection is to use visual information from outdoor cameras that widely monitor the filed. There have been many attempts to detect smokes via optical sensors on digital cameras using optical flow methods, but not fully successful from practical-use aspects. It is because the area of smokes occupying on the footage by outdoor cameras is not necessarily large enough. Moreover, in case of urban cities, discrimination of the smokes from other moving objects such as cars, trees and turbines is not easy. Herein we propose a novel method to detect daytime smokes based on variance of optical flow and characteristics of HSV (hue-saturation-value) color. We apply the method to a set of footage of three days obtained in an industrial zone in Japan. Successful results are obtained as over 90% of smokes are detected. Notable is that this method is independent of solar radiation conditions on sunny and cloudy days. [ABSTRACT FROM AUTHOR]

To solve the problem of poor localization accuracy and robustness of visual simultaneous localization and mapping (SLAM) systems in highly dynamic environments, this paper proposes a dynamic visual SLAM algorithm called FFD-SLAM that fuses the target detection network with the optical flow method. The algorithm considers ORB-SLAM2 as the basic framework, joins the semantic thread in parallel with its tracking thread, initially obtains the set of feature points through the real-time detection of dynamic objects in the environment through YOLOv5 in the semantic thread, then filters the set of feature points obtained in the semantic thread through the optical flow module, and finally utilizes the remaining static feature points for the matching calculation. Experiments showed that the proposed algorithm showed an improvement of approximately 97% in the localization accuracy compared with the ORB-SLAM2 algorithm in a highly dynamic environment, which effectively improves the localization accuracy and robustness of the system. The proposed algorithm also showed a higher real-time performance compared with some excellent dynamic SLAM algorithms. [ABSTRACT FROM AUTHOR]

Motion object detection (MOD) with freely moving cameras is a challenging task in computer vision. To extract moving objects, most studies have focused on the difference in motion features between foreground and background, which works well for dynamic scenes with relatively regular movements and variations. However, abrupt illumination changes and occlusions often occur in real-world scenes, and the camera may also pan, tilt, rotate, and jitter, etc., resulting in local irregular variations and global discontinuities in motion features. Such complex and changing scenes bring great difficulty in detecting moving objects. To solve this problem, this paper proposes a new MOD method that effectively leverages local and global visual information for foreground/background segmentation. Specifically, on the global side, to support a wider range of camera motion, the relative inter-frame transformations are optimized to absolute transformations referenced to intermediate frames in a global form after enriching the inter-frame matching pairs. The global transformation is fine-tuned using the spatial transformer network (STN). On the local side, to address the problem of dynamic background scenes, foreground object detection is optimized by utilizing the pixel differences between the current frame and the local background model, as well as the consistency of local spatial variations. Then, the spatial information is combined using optical flow segmentation methods, enhancing the precision of the object information. The experimental results show that our method achieves a detection accuracy improvement of over 1.5% compared with the state-of-the-art methods on the datasets of CDNET2014, FBMS-59, and CBD. It demonstrates significant effectiveness in challenging scenarios such as shadows, abrupt changes in illumination, camera jitter, occlusion, and moving backgrounds. [ABSTRACT FROM AUTHOR]

With the development of simultaneous positioning and mapping technology in the field of automatic driving, the current simultaneous localization and mapping scheme is no longer limited to a single sensor and is developing in the direction of multi-sensor fusion to enhance the robustness and accuracy. In this study, a localization and mapping scheme named LVI-fusion based on multi-sensor fusion of camera, lidar and IMU is proposed. Different sensors have different data acquisition frequencies. To solve the problem of time inconsistency in heterogeneous sensor data tight coupling, the time alignment module is used to align the time stamp between the lidar, camera and IMU. The image segmentation algorithm is used to segment the dynamic target of the image and extract the static key points. At the same time, the optical flow tracking based on the static key points are carried out and a robust feature point depth recovery model is proposed to realize the robust estimation of feature point depth. Finally, lidar constraint factor, IMU pre-integral constraint factor and visual constraint factor together construct the error equation that is processed with a sliding window-based optimization module. Experimental results show that the proposed algorithm has competitive accuracy and robustness. [ABSTRACT FROM AUTHOR]

Facial action unit (AU) recognition and optical flow estimation are two highly correlated tasks, since optical flow can provide motion information of facial muscles to facilitate AU recognition. However, most existing AU recognition methods handle the two tasks independently by offline extracting optical flow as auxiliary information or directly ignoring the use of optical flow. In this paper, we propose a novel end-to-end joint framework of AU recognition and optical flow estimation, in which the two tasks contribute to each other. Moreover, due to the lack of optical flow annotations in AU datasets, we propose to estimate optical flow in a self-supervised manner. To regularize the self-supervised estimation of optical flow, we propose an identical mapping constraint for the optical flow guided image warping process, in which the estimated optical flow between two same images is required to not change the image during warping. Experiments demonstrate that our framework (i) outperforms most of the state-of-the-art AU recognition methods on the challenging BP4D and GFT benchmarks, and (ii) also achieves competitive self-supervised optical flow estimation performance. • An end-to-end joint framework of AU recognition and self-supervised optical flow estimation. • An identity mapping constraint to ensure the reliability of self-supervised optical flow estimation. • An AU occurrence probability map regression loss for exploiting AU location information and facial spatial information. • Our approach outperforms most of the state-of-the-art AU recognition methods. • Our approach achieves comparable self-supervised optical flow estimation performance. [ABSTRACT FROM AUTHOR]

Hierarchical concepts have proven useful in many classical and learning-based optical flow methods regarding both accuracy and robustness. In this paper we show that such concepts are still useful in the context of recent neural networks that follow RAFT's paradigm refraining from hierarchical strategies by relying on recurrent updates based on a single-scale all-pairs transform. To this end, we introduce MS-RAFT+: a novel recurrent multi-scale architecture based on RAFT that unifies several successful hierarchical concepts. It employs a coarse-to-fine estimation to enable the use of finer resolutions by useful initializations from coarser scales. Moreover, it relies on RAFT's correlation pyramid that allows to consider non-local cost information during the matching process. Furthermore, it makes use of advanced multi-scale features that incorporate high-level information from coarser scales. And finally, our method is trained subject to a sample-wise robust multi-scale multi-iteration loss that closely supervises each iteration on each scale, while allowing to discard particularly difficult samples. In combination with an appropriate mixed-dataset training strategy, our method performs favorably. It not only yields highly accurate results on the four major benchmarks (KITTI 2015, MPI Sintel, Middlebury and VIPER), it also allows to achieve these results with a single model and a single parameter setting. Our trained model and code are available at https://github.com/cv-stuttgart/MS_RAFT_plus. [ABSTRACT FROM AUTHOR]

In current times, reconstruction of remote sensing images using super-resolution is a prominent topic of study. Remote sensing data have a complex spatial distribution. Compared with natural pictures, remote sensing pictures often contain subtler and more complicated information. Most super-resolution reconstruction algorithms cannot restore all the information contained in remote sensing images when reconstructing them. The content of some areas in the reconstructed images may be too smooth, and some areas may even have color changes, resulting in lower quality reconstructed images. In response to the problems presenting in current reconstruction algorithms about super-resolution, this article proposes the SRGAN-MSAM-DRC model (SRGAN model with multi-scale attention mechanism and dense residual connection). This model roots in generative adversarial networks and incorporates multi-scale attention mechanisms and dense residual connections into the generator. Furthermore, residual blocks are incorporated into the discriminator. We use some remote sensing image datasets of real-world data to evaluate this model, and the results indicate the SRGAN-MSAM-DRC model has shown enhancements in three evaluation metrics for reconstructed images about super-resolution. Compared to the basic SRGAN model, the SSIM (structural similarity), PSNR (peak signal-to-noise ratio), and IE (image entropy) increase by 5.0%, 4.0%, and 4.1%, respectively. From the results, we know the quality of the reconstructed images of remote sensing using the SRGAN-MSAM-DRC model is better than basic SRGAN model, and verifies that the model has good applicability and performance in reconstruction of remote sensing images using super-resolution. [ABSTRACT FROM AUTHOR]

Flight accidents caused by spatial disorientation (SD) greatly affect flight safety. Few studies have been devoted to the evaluation of SD. 10 pilots and 10 non-pilots were recruited for the experimental induction of SD. Videos for giving optical flow stimuli were played at two different flow speeds to induce SD. Subjective judgment and center of foot pressure (CoP) data were collected from the tests. The data were combined to determine the occurrence of SD and analyze the SD types. The number of self-reported SD events was slightly smaller in the pilots than in the non-pilots. The average upper bound of the confidence interval for the standard deviation of CoP was 0.32 ± 0.09 cm and 0.38 ± 0.12 cm in the pilots and non-pilots, respectively. This indicator was significantly lower in the pilots than in the non-pilots (P= 0.03). The success rate of the experimental induction of unrecognized SD was 26.7% and 45.0% in the pilots and non-pilots, respectively. The method offered a new to analyze unrecognized SD. We could determine the occurrence unrecognized SD. This is an essential means of reducing flight accidents caused by unrecognized SD. [ABSTRACT FROM AUTHOR]

To resolve high-precision deformation from low-quality speckle images, a high-order time-domain digital image correlation algorithm is proposed. This algorithm is based on the nonlinear optical flow equation for low-quality speckle images collected in extreme testing environments. A subpixel displacement image containing Gaussian white noise is established by using the simulated speckle method. The displacement of the high-order time-domain digital image correlation algorithm is solved by the least squares iteration method, and the test accuracy of the proposed algorithm is analyzed. The results show that the nonlinear optical flow equation can describe the time continuity of pixel gray level changes before and after deformation. The proposed algorithm can also analyze the velocity and acceleration term of deformation. The high-order time-domain digital image correlation algorithm is insensitive to the change in image grayscale and has high measurement accuracy. Therefore, this method can solve the problem of resolving high-precision deformation from low-quality speckle images. [ABSTRACT FROM AUTHOR]

Purpose: We employed automated analysis of scanning laser ophthalmoscopy (SLO) to determine if mechanical strains imposed on disc, and retinal and choroidal vessels during horizontal duction in children differ from those of adults. Methods: Thirty-one children aged 11.3 ± 2.7 (standard deviation) years underwent SLO in central gaze, and 35° ab- and adduction. Automated registration with deep learning-based optical flow analysis quantified vessel deformations as horizontal, vertical, shear, and equivalent strains. Choroidal vessel displacements in lightly pigmented fundi, and central disc vessel displacements, were also observed. Results: As in adults, strain in vessels during horizontal duction was greatest at the disc and decreased with distance from it. Strain in the pediatric disc was similar to published values in young adults,1 but in the peripapillary region was greater and propagated significantly more peripherally to at least three disc radii from it. During adduction in children, the nasal disc was compressed and disc vessels distorted, but the temporal half experienced tensile strain, while peripapillary tissues were compressed. The pattern was similar but strains were less in abduction (p <.001). Choroidal vessels were visualized in 24 of the 62 eyes and shifted directionally opposite overlying retinal vessels. Conclusions: Horizontal duction deforms the normal pediatric optic disc, central retinal vessels, peripapillary retina, and choroid, shearing the inner retina over the choroid. These mechanical effects occur at the sites of remodeling of the disc, sclera, and choroid associated with typical adult features that later emerge later, including optic cup enlargement, temporal disc tilting, and peripapillary atrophy. [ABSTRACT FROM AUTHOR]

Video super-resolution (VSR) remains challenging for real-world applications due to complex and unknown degradations. Existing methods lack the flexibility to handle video sequences with different degradation levels, thus failing to reflect real-world scenarios. To address this problem, we propose a degradation-adaptive video super-resolution network (DAVSR) based on a bidirectional propagation network. Specifically, we adaptively employ three distinct degradation levels to process input video sequences, aiming to obtain training pairs that reflect a variety of real-world corrupted images. We also equip the network with a pre-cleaning module to reduce noise and artifacts in the low-quality video sequences prior to information propagation. Additionally, compared to previous flow-based methods, we employ an unsupervised optical flow estimator to acquire a more precise optical flow to guide inter-frame alignment. Meanwhile, while maintaining network performance, we streamline the propagation network branches and the structure of the reconstruction module of the baseline network. Experiments are conducted on datasets with diverse degradation types to validate the effectiveness of DAVSR. Our method exhibits an average improvement of 0.18 dB over a recent SOTA approach (DBVSR) in terms of the PSNR metric. Extensive experiments demonstrate the effectiveness of our network in handling real-world video sequences with different degradation levels. [ABSTRACT FROM AUTHOR]

This paper proposes a new sensor using optical flow to stabilize a quadrotor when a GPS signal is not available. Normally, optical flow varies with the attitude of the aerial vehicle. This produces positive feedback on the attitude control that destabilizes the orientation of the vehicle. To avoid this, we propose a novel sensor using an optical flow camera with a 6DoF IMU (Inertial Measurement Unit) mounted on a two-axis anti-shake stabilizer mobile aerial gimbal. We also propose a robust algorithm based on Sliding Mode Control for stabilizing the optical flow sensor downwards independently of the aerial vehicle attitude. This method improves the estimation of the position and velocity of the quadrotor. We present experimental results to show the performance of the proposed sensor and algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes a new enterprise intelligentization framework, by making the transition from process transformation to artificial intelligence (AI) transformation. The novel transformation framework can be decomposed into the conceptual model of AI strategic planning, the procedural model, the operational model, and the analytics model. For leading-edge microchip production, a new AI transformation project regarding the reticle SMIF pod (RSP) transport system designed by a medium-sized semiconductor tool vendor in Taiwan is presented. The technical advantages, gained from the implementation of the presented AI transformation project, over the existing RSP systems are manifold. The throughput and yield rate significantly increase on a semiconductor-fabrication-plant basis. The clean room construction costs less by approximately 3 million dollars per FAB, mainly attributed to the redesigned automatic optical inspection flow. The proposed model-based framework proves to be a viable tool from the process transformation to the AI transformation in the semiconductor manufacturing. [ABSTRACT FROM AUTHOR]

Current marker-based facial motion capture methods might lose the target markers in some cases, such as those with considerable occlusion and blur. Manually revising these statuses requires extensive labor-intensive work. Thus, a robust marker tracking method that provides long-term stability must be developed, thereby simplifying manual operations. In this paper, we present a new facial marker tracking system that focuses on the accuracy and stability of performance capture. The tracking system includes a synthetic analysis step with the robust optical flow tracking method and the proposed Marker-YOLO detector. To illustrate the strength of our system, a real dataset of the performance of voluntary actors was obtained, and ground truth labels were given by artists for subsequent experiments. The results showed that our approach outperforms state-of-the-art trackers such as SiamDW and ECO in specific tasks while running at a real-time speed of 38 fps. The root-mean-squared error and area under the curve results verified the improvements in the accuracy and stability of our approach. [ABSTRACT FROM AUTHOR]

We propose an adaptive fractional multi-scale optimization optical flow algorithm, which for the first time improves the over-smoothing of optical flow estimation under the total variation model from the perspective of global feature and local texture balance, and solves the problem that the convergence of fractional optical flow algorithms depends on the order parameter. Specifically, a fractional-order discrete L1-regularization Total Variational Optical Flow model is constructed. On this basis, the Ant Lion algorithm is innovatively used to realize the iterative calculation of the optical flow equation, and the fractional order is dynamically adjusted to obtain an adaptive optimization algorithm with strong search accuracy and high efficiency. In this paper, the flexibility of optical flow estimation in weak gradient texture scenes is increased, and the optical flow extraction rate of target features at multiple scales is greatly improved. We show excellent recognition performance and stability under the MPI_Sintel and Middlebury benchmarks. [ABSTRACT FROM AUTHOR]

Optical flow estimation is an important hot research in computer vision. Although existing methods had got a considerable progress in improving their performance, they still have drawbacks, such as heavily computational burden, inaccurate pixel-level offset estimation, and poor interpretability. To address these issues, this letter proposes a pyramidal deep Taylor expansion (PDTE) framework, including: First, we seriously interpreted the relationship between optical flow computation and Taylor expansion. Then, the proposed PDTE is constructed by employing global motion aggregation (GMA) to calculate each derivative part which contributes to the final estimated optical flow. Quantitative and qualitative results on the Sintel and KITTI datasets validate that the proposed PDTE scheme is effective and outperforms the state-of-the-art optical flow estimation methods. The results of extensive experiments in the ablation study demonstrate that PDTE performs well on shape preservation and the accuracy improvement of optical flow estimation, even pixel-level offset calculation. • A novel pyramidal deep Taylor expansion (PDTE) framework for optical flow estimation. • Each derivative part calculated by a global motion aggregation (GMA) method. • Both quantitative and qualitative results verify the effectiveness of our PDTE. [ABSTRACT FROM AUTHOR]

• Design and train 55-layer CNN model on the CIFAR-100 dataset. • Build two-stream network to extract spatial and temporal features from the images. • Feature optimization is performed using a Genetic algorithm. Human Gait Recognition (HGR) is referred to as a biometric tactic that is broadly used for the recognition of an individual by using the pattern of walking. There are some key factors such as angle variation, clothing variation, foot shadows, and carrying conditions that affect the human gait. In this work, a new approach is proposed for the HGR that contains five major steps. In the first step, the video data is converted into image frames. In the second step, RGB to GRAY conversion is carried out. After that, a two-stream network is designed by using a 55-layer CNN model called CNN-55 trained on CIFAR-100. The CNN-55 is designed from scratch and trained on the CIFAR-100 dataset by selecting hyperparameters. This pre-trained CNN-55 is used to build a two-stream network. In Stream-1 the optical flow frames are obtained by Horn and Schunk algorithm. These frames are fed into a CNN-55 to extract temporal features. In Stream-2 the GRAY frames are fed to the CNN-55 model for extraction of spatial features. After that, both vectors are serially fused. In the fourth step, the fused feature vector is fed into the Genetic Algorithm for optimization. Finally, the feature vector is fed into the One-Versus-All SVM classifier for recognition. The system is tested on all CASIA-B angles such as 000, 180, 360, 540, 720, 900, 1080, 1260, 1440, 1620, and 1800 which provides accuracy of 97.10%, 96.80%, 94.60%, 98.0%, 98.30%, 96.80%, 97.60, 96.90%, 99.60%, 96.80%, and 97.60%, respectively. The proposed method produces better outcomes compared to recent techniques. [ABSTRACT FROM AUTHOR]

n smart cities, sanitation workers are the key to urban construction. Accurate targeting of sanitation workers can help managers better monitor and manage them. Pedestrian detection is the core and key component of object detection technology. The difficulty of pedestrian detection in the actual feature recognition is still how to quickly and accurately identify the identity in the complex video scene. To realize the effective detection of sanitation workers, the study designs an identity identification scheme conducive to the friendly management of smart urban management. Since the optical flow feature extraction method and HSV color space extraction method can not meet the actual detection efficiency, this study innovatively integrates the two methods to improve the detection accuracy of the mode. Meanwhile, the study also introduces PCA algorithm to identify the specific identity of sanitation workers. In the actual detection of sanitation workers, the identification rate of two sanitation workers is high, and the similarity is 98.61%. This technology greatly reduces the false detection rate of actual detection and improves the detection accuracy. [ABSTRACT FROM AUTHOR]

Background: X-ray imaging addresses many challenges with visible light imaging in extreme environments, where optical diagnostics such as digital image correlation (DIC) and particle image velocimetry (PIV) suffer biases from index of refraction changes and/or cannot penetrate visibly occluded objects. However, conservation of intensity—the fundamental principle of optical image correlation algorithms—may be violated if ancillary components in the X-ray path besides the surface or fluid of interest move during the test. Objective: The test series treated in this work sought to characterize the safe use of fiber-epoxy composites in aerospace and aviation industries during fire accident scenarios. Stereo X-ray DIC was employed to measure test unit deformation in an extreme thermal environment involving a visibly occluded test unit, incident surface heating to temperatures above 600oC, and flames and soot from combusting epoxy decomposition gasses. The objective of the current work is to evaluate two solutions to resolve the violation of conservation of intensity that resulted from both the test unit and the thermal chamber deforming during the test. Methods: The first solution recovered conservation of intensity by pre-processing the path-integrated X-ray images to isolate the DIC pattern of the test unit from the thermal chamber components. These images were then correlated with standard, optical DIC software. The second solution, called Path-Integrated Digital Image Correlation (PI-DIC), modified the fundamental matching criterion of DIC to account for multiple, independently-moving components contributing to the final image intensity. The PI-DIC algorithm was extended from a 2D framework to a stereo framework and implemented in a custom DIC software. Results: Both solutions accurately measured the cylindrical shape of the undeformed test unit, recovering radii values of R = 76.20 ± 0.12 mm compared to the theoretical radius of R theor = 76.20 mm. During the test, the test unit bulged asymmetrically as decomposition gasses pressurized the interior and eventually burned in a localized jet. Both solutions measured the heterogeneous radius of this bulge, which reached a maximum of approximately R = 91 mm, with a discrepancy of 2–3% between the two solutions. Conclusions: Two solutions that resolve the violation of conservation of intensity for path-integrated X-ray images were developed. Both were successfully employed in a stereo X-ray DIC configuration to measure deformation of an aluminum-skinned, fiber-epoxy composite test unit in a fire accident scenario. [ABSTRACT FROM AUTHOR]

Recent optical flow estimators usually employ deep models designed for image classification as the encoders for feature extraction and matching. However, those encoders developed for image classification may be sub-optimal for flow estimation. In contrast, the decoder design of optical flow estimators often requires meticulous design for flow estimation. The disconnect between the encoder and decoder could negatively affect optical flow estimation. To address this issue, we propose a neural architecture search method, FlowNAS, to automatically find the more suitable and stronger encoder architecture for existing flow decoders. We first design a suitable search space, including various convolutional operators, and construct a weight-sharing super-network for efficiently evaluating the candidate architectures. To better train the super-network, we present a Feature Alignment Distillation module that utilizes a well-trained flow estimator to guide the training of the super-network. Finally, a resource-constrained evolutionary algorithm is exploited to determine an optimal architecture (i.e., sub-network). Experimental results show that FlowNAS can be easily incorporated into existing flow estimators and achieves state-of-the-art performance with the trade-off between accuracy and efficiency. Furthermore, the encoder architecture discovered by FlowNAS with the weights inherited from the super-network achieves 4.67% F1-all error on KITTI, an 8.4% reduction of RAFT baseline, surpassing state-of-the-art handcrafted GMA and AGFlow models, while reducing the model complexity and latency. The source code and trained models will be released at https://github.com/VDIGPKU/FlowNAS. [ABSTRACT FROM AUTHOR]

• Propose the recurrent motion attention network (RMA-Net) to extract spatial-temporal image features for object detection in drone vision. • A recurrent fashion is designed to effectively and efficiently learn motion features in long sequences. • Only requiring a few frames of optical flows for initialization. • Not only boosts the mean average precision by 6.99 % but also reaches 31 frames per second. In recent years, object detection in drone videos has garnered widespread attention. Conventional image-based detectors have encountered a performance bottleneck for drone images due to the severe appearance deterioration of targets and numerous background clutters. Video object detection (VOD) is a promising paradigm for such problems as it can effectively enhance target appearance by aggregating features from nearby frames in drone videos. However, struggling with the feature alignment problem which is particularly important for drone images where the background varies frequently. Following the VOD philosophy, we propose the Recurrent Motion Attention Network (RMA-Net) to extract spatial-temporal image features for object detection in drone vision. A recurrent fashion is designed to effectively and efficiently learn motion features in long sequences, only requiring a few frames of optical flows for initialization. Experiments demonstrate that the proposed method could easily promote existing object detectors to achieve high performance on drone videos. Specifically, our RMA adapted YOLOv7 outperforms the state-of-the-arts by a large margin on the challenging VisDrone2019-VID dataset, not only boosts the mean average precision by 6.99 % but also reaches a batch-1 inference speed of 31 frames per second. [ABSTRACT FROM AUTHOR]

Edge computing is essential to handle increasing data volumes and processing capacities. It provides real-time and secure data processing near data sources, like smart devices, alleviating cloud computing energy use, and saving network bandwidth. Specialized accelerators, like GPUs and FPGAs, are vital for low-latency edge computing but the requirements to customized code for different hardware and vendors suppose important compatibility issues. This paper evaluates the potential of SYCL in addressing code portability issues encountered in edge computing. We employed the Polybench suite to compare various SYCL implementations, specifically DPC++ and AdaptiveCpp, with the native solution, CUDA. The disparity between SYCL implementations was negligible, at just 5%. Furthermore, we evaluated SYCL in the context of specific edge computing applications such as video processing using three different optical flow algorithms. The results revealed a slight performance gap of 3% when transitioning from CUDA to SYCL. Upon evaluating energy consumption, the observed difference ranged from ± 10 % , depending on the application utilized. These gaps are the price one may need to pay when achieving the ability to successfully run the same code on two distinct edge boards. These findings underscore SYCL's capacity to increase productivity in terms of development costs and facilitate IoT deployment without being locked into a particular platform or manufacturer. [ABSTRACT FROM AUTHOR]

Estimating depth, ego-motion, and optical flow from consecutive frames is a critical task in robot navigation and has received significant attention in recent years. In this study, we propose PDF-Former, an unsupervised joint estimation network comprising a full transformer-based framework, as well as a competition and cooperation mechanism. The transformer framework captures global feature dependencies and is customized for different task types, thereby improving the performance of sequential tasks. The competition and cooperation mechanisms enable the network to obtain additional supervisory information at different training stages. Specifically, the competition mechanism is implemented early in training to achieve iterative optimization of 6 DOF poses (rotation and translation information from the target image to the two reference images), the depth of target image, and optical flow (from the target image to the two reference images) estimation in a competitive manner. In contrast, the cooperation mechanism is implemented later in training to facilitate the transmission of results among the three networks and mutually optimize the estimation results. We conducted experiments on the KITTI dataset, and the results indicate that PDF-Former has significant potential to enhance the accuracy and robustness of sequential tasks in robot navigation. • A transformer-based framework for joint pose, depth, and optical flow estimation is proposed. • The competition–cooperation mechanisms are proposed to promote performance improvement. • Achieved notable accuracy in public datasets. [ABSTRACT FROM AUTHOR]

Particle image velocimetry (PIV) and optical flow velocimetry (OFV) are important velocity measurement methods in the field of fluid dynamics. Nevertheless, the conventional cross correlation-based PIV method is beset by diminished resolution, while the OFV method exhibits computational sluggishness and susceptibility to noise. These constraints have somewhat delimited the applicability of PIV and OFV techniques. Recent attempts have introduced deep learning-based methods for analyzing PIV images, offering high-resolution velocity fields with computational efficiency, but their accuracy needs improvement. This study proposes four neural networks based on the well-established FlowNetS. They incorporate two distinct velocity constraints, namely, first-order velocity smoothing regularization and second-order grad (curl)–grad (div) regularization. In the networks, these constraints are used either independently or in combination with optical flow conservation (OFC). The performances of the networks have been assessed on six different flow configurations, and the results show that the network with the second-order regularization markedly outperforms the original network across all flows, demonstrating an enhanced capacity to capture larger-scale vortices. The network with the first-order regularization also exhibits superior performance compared to the original network except in the case of cylinder flow. Unexpectedly, the introduction of the OFC constraints results in a decline in network performance. This anomaly may stem from the network's inherent capability to capture optical flow features, rendering the OFC constraint less effective in providing guidance. In summary, this study underscores the substantial potential of neural networks incorporated with judicious physical constraints in PIV applications, enabling the determination of high-resolution, high-accuracy flow fields. [ABSTRACT FROM AUTHOR]

Due to the complexity of real optical flow capture, the existing research still has not performed real optical flow capture of infrared (IR) images with the production of an optical flow based on IR images, which makes the research and application of deep learning-based optical flow computation limited to the field of RGB images only. Therefore, in this paper, we propose a method to produce an optical flow dataset of IR images. We utilize the RGB-IR cross-modal image transformation network to rationally transform existing RGB image optical flow datasets. The RGB-IR cross-modal image transformation is based on the improved Pix2Pix implementation, and in the experiments, the network is validated and evaluated using the RGB-IR aligned bimodal dataset M3FD. Then, RGB-IR cross-modal transformation is performed on the existing RGB optical flow dataset KITTI, and the optical flow computation network is trained using the IR images generated by the transformation. Finally, the computational results of the optical flow computation network before and after training are analyzed based on the RGB-IR aligned bimodal data. [ABSTRACT FROM AUTHOR]

Motion information from image sequences (videos) plays a key role in solving a number of real-world problems such as surveillance, traffic management, fire identification, weather prediction, and COVID-19 detection, etc. Generally, this motion is estimated in terms of optical flow between two consecutive image frames. Optical flow is a 2D vector field that illustrates object motion behavior. This paper presents a nonlinear fractional order variational model for estimating optical flow. The objective of this work is to provide dense and discontinuity preserving optical flow for different spectra and make the model robust against outliers. In particular, the presented model generalizes the integer order variational models for fractional order (0, 1). For this purpose, the proposed model is formulated with the help of Charbonnier norm and Caputo fractional derivative. Charbonnier norm is nonlinear in nature, which makes the model robust against noise and outliers, whereas Caputo fractional derivatives are well-capable to deal with discontinuous functions such as images, and therefore, preserve motion discontinuities. The Caputo derivative also manifests long-term memory effect and allows to choose the optimal value of the fractional order that corresponds to a stable solution. The numerical implementation of the formulated variational functional is performed by discretizing the fractional derivative using the Grünwald–Letnikov derivative. The resulting system of equations is solved using multi-variable fixed point iteration scheme. The entire framework is embedded in coarse-to-fine warping strategy, which helps in finding the global extremal. The experimental results are carried out on 20 different application oriented datasets such as fire and smoke, fluid, CXR, etc. The performance of the model is tested using different error measures and demonstrated against several outliers. Error concentration is shown through 3D histograms. The edge preserving nature is also realized through intensity distribution in RGB color channels. A detailed convergence analysis is also provided for the presented model. The validity of the proposed model is also verified through comparisons with other existing models. [ABSTRACT FROM AUTHOR]

This paper solves the task of unsupervised video object segmentation (UVOS) that segments the objects of interest through the entire videos without any annotation. In recent years, many unsupervised video object segmentation (UVOS) methods have been proposed. Although these methods perform well, they rely on networks with heavy weights, often leading to large model size. In order to reduce the model size while keeping a competitive performance, we propose a saliency-based dual-attention (SDA) method for UVOS in this paper. In our method, we take optical flow and video frames as inputs and extract the appearance information and motion information from optical flow and video frames. We design a two-branch network with appearance information and motion information. The information from these two branches is fused via a saliency-based dual-attention module to segment the primary object in one path. The saliency-based dual-attention module is composed of saliency attention and saliency-based reverse attention. To demonstrate the effectiveness of our network, we tested it on the DAVIS-2016 and SegtrackV2 datasets. Experimental results demonstrate that our method can achieve competitive results in terms of accuracy and model size. [ABSTRACT FROM AUTHOR]

With the goal of solving the problem of feature extractors lacking strong supervision training and insufficient time information concerning single‐sequence model learning, a hierarchical sequence memory network with a multi‐level iterative optimisation strategy is proposed for continuous sign language recognition. This method uses the spatial‐temporal fusion convolution network (STFC‐Net) to extract the spatial‐temporal information of RGB and Optical flow video frames to obtain the multi‐modal visual features of a sign language video. Then, in order to enhance the temporal relationships of visual feature maps, the hierarchical memory sequence network is used to capture local utterance features and global context dependencies across time dimensions to obtain sequence features. Finally, the decoder decodes the final sentence sequence. In order to enhance the feature extractor, the authors adopted a multi‐level iterative optimisation strategy to fine‐tune STFC‐Net and the utterance feature extractor. The experimental results on the RWTH‐Phoenix‐Weather multi‐signer 2014 dataset and the Chinese sign language dataset show the effectiveness and superiority of this method. [ABSTRACT FROM AUTHOR]

Multi-View Stereo (MVS) algorithms remain a significant challenge in reconstructing a 3D model with high completeness due to the difficulty in recovering weakly textured regions and detailed parts of large-scale urban scenes. Although the Image Pyramid Structure is a popular approach for dealing with weakly textured regions, it also leads to the loss of detailed information. The proposed method solves these problems with three new strategies: (1) We propose the optical flow consistency for recovering details. The optical flow consistency improved the sensitivity of the image pyramid structure to details by estimating the motion vector of each pixel point. We proposed a novel detail restorer based on optical flow consistency which improves the link between adjacent scales in the image pyramid structure. (2) Geometric consistency based on epipolar line constraints is proposed to recover weakly textured regions. The proposed epipolar line constraints improve the robustness of traditional geometric consistency, which avoids the problem of mismatching in weakly textured regions. (3) A depth-filling strategy is utilized to fill the loss of depth information of weakly textured regions. Image gradient is utilized to fill the gap of depth information. The filled result is utilized as the priori information to smooth the depth of weakly textured regions. Experimental results on the ETH3D , UDD5 and SenseFly benchmark datasets demonstrate that the proposed method outperforms three state-of-the-art methods (ACMMP, EPNet, DeepC-MVS), significantly improving the completeness of the 3D models. The source code of the develop method is available at https://github.com/Liaoyongjian1/HC-MVS.git. [ABSTRACT FROM AUTHOR]

The interest of science community towards planet Mars has progressively increased in past decades. Rovers are already doing an excellent job in exploring the Martian surface environment and showed that the Red Planet is not suited for life. However, there is a possibility of enabling human exploration by having astronauts take shelter in habitats located within lava tubes, which can provide protection against radiation and dust storms. These tubes are extremely hard to explore with rovers, thus a flying vehicle will be more suitable for such tasks. INGENUITY (Martian helicopter) had already demonstrated that it is possible to fly on Mars by doing small and simple manoeuvres. Based on NASA's design, this article will introduce the development of guidance, navigation, and control operations of a Martian Inspection Drone (MID) that will be capable of finding the lava tube and scan it in real time using a RealSense camera. Linux based system was chosen in the development process, which allow ROS catkin workspace to connect ArduPilot and Gazebo. The navigation is accomplished using a combination of an inertial measurement unit, a camera, and a laser rangefinder. The flight computer runs an optical flow algorithm for control and navigation purpose. Also, a convolutional neural network is used to perform lava tube entrance detection and inspection. • Martian flying drone simulated with ArduPilot, Gazebo and ROS. • Martian environment simulation. • Martian lava tube detection using convolutional neural networks. • Martian lava tube inspection using a realsense camera. [ABSTRACT FROM AUTHOR]

The realization and modulation of high-order topological states in three-dimensional (3D) photonic crystals (PCs) hold great significance for practical applications in optical communication, optical information processing, and optical computing. However, they have encountered technological challenges due to the difficulty in achieving a complete bandgap in 3D PCs. To address this issue, we introduce a design for a 3D PC based on the Su–Schrieffer–Heeger (SSH) model that exhibits distinctive topological surface states, topological hinge states, and topological corner states. Furthermore, these three types of topological states can be interconverted through the modulation of magnetic field because the topological states are closed and extremely sensitive to the structure, which are modulated by the magnetic field. This intriguing capability holds potential applications in the manipulation of optical flow, optical signal storage, and magnetic field sensors. [ABSTRACT FROM AUTHOR]

The spatter generated by the interaction between laser and powder during Powder Bed Fusion-Laser Melting (PBF-LM) can significantly affect the quality of printed parts. A high-speed camera is used to observe the dynamic process of spatter's behavior under different layer thickness and laser powers during the printing process, and to analyze the printed samples' surface roughness, microstructure, and mechanical properties. In terms of spatter image processing, employing an optical flow approach to track and quantify the number of spatters efficiently eliminates statistical redundancy and improves statistical correctness. It is found that under the same laser power, the number of spatters produced by the laser scan direction with the gas flow (LSD-W) is more than that by the laser scan direction against the gas flow (LSD-A), and the number of spatters produced increases with the increase of laser power. Analyzing the mechanical properties and surface roughness of the printed samples under different process parameters quantitatively reveals that differences in the spatter amount generated under different process parameters in the PBF-LM process is not the determining factor affecting the difference in tensile strength of printed parts. During LSD-W, the number of spatters generated at laser power of 170 W and layer thickness of 0.03 mm is 87, and the tensile strength of the printed sample is 618 MPa. During LSD-W, the number of spatters generated at laser power of 320 W and layer thickness of 0.05 mm is 211, and the tensile strength of the printed sample is 680 MPa. Instead, spatter generation has a more direct impact on the surface roughness of printed parts. The layer thickness is 0.03 mm, the laser power is 170 W, and (Ra = 2.372 μm) is the surface roughness of the sample. The layer thickness is 0.05 mm, the laser power is 320 W, and (Ra = 8.163 μm) is the surface roughness of the sample. [ABSTRACT FROM AUTHOR]

In this paper, we address the challenge of detecting small moving targets in dynamic environments characterized by the concurrent movement of both platform and sensor. In such cases, simple image-based frame registration and optical flow analysis cannot be used to detect moving targets. To tackle this, it is necessary to use sensor and platform meta-data in addition to image analysis for temporal and spatial anomaly detection. To this end, we investigate techniques that utilize inertial data to enhance frame-to-frame registration, consistently yielding improved detection outcomes when compared against purely feature-based techniques. For cases where image registration is not possible even with metadata, we propose single-frame spatial anomaly detection and then estimate the range to the target using the platform velocity. The behavior of the estimated range over time helps us to discern targets from clutter. Finally, we show that a KNN classifier can be used to further reduce the false alarm rate without a significant reduction in detection performance. The proposed strategies offer a robust solution for the detection of moving targets in dynamically challenging settings. [ABSTRACT FROM AUTHOR]

Optical fiber tweezers offer a simple, low-cost and portable solution for non-invasive trapping and manipulation of particles. However, single-fiber tweezers require fiber tip modification (tapering, lensing, etc.) and the dual-fiber approach demands strict alignment and positioning of fibers for robust trapping of particles. In addition, both tweezing techniques offer a limited range of particle manipulation and operate in low flow velocity regimes (a few 100 μm s−1) when integrated with microfluidic devices. In this paper, we report a novel opto-hydrodynamic fiber tweezers (OHT) platform that exploits the balance between the hydrodynamic drag force and optical scattering forces to trap and manipulate single or multiple particles of various shapes, sizes, and material compositions in a microfluidic channel. 3D hydrodynamic flow focusing offers an easy and dynamic alignment of the particle trajectories with the optical axis of the fiber, which enables robust trapping of particles with high efficiency of >70% and throughput of 14 particles per minute (operating flow velocity: 1000 μm s−1) without the need for precision stages or complex fabrication. By regulating the optical power and flow rates, we were able to trap single particles at desired positions in the channel with a precision of ±10 μm as well as manipulate them over a long range upstream or downstream with a maximum distance of 500 μm. Our opto-hydrodynamic tweezers offer an alternative to conventional optical fiber tweezers for several applications in physics, biology, medicine, etc. [ABSTRACT FROM AUTHOR]

Currently, shrimp fry vitality detection mainly relies on manual observation of shrimp fry swimming, but the evaluation results are susceptible to subjective influence. To address this problem, this study proposes a shrimp fry vitality detection method that combines the dense optical flow method and statistics. Firstly, shrimp fry with different vitality were obtained by rearing them in different water temperatures, and shrimp fry videos were collected; then the inter-frame motion features of shrimp fry in the videos were extracted using the Gunner Farneback optical flow method, which represents the instantaneous swimming state of the shrimp fry, and contains two dimensions, namely, magnitude and angle, and statistical analyses were performed at the same time; finally, we extracted the motion features between all the frames of the fixed-length shrimp video and applied the improved information entropy and mutual information to detect the vitality of the shrimp fry. After testing, the precision and recall of the two detection methods exceeded 98%, which can accurately and objectively realize the shrimp fry vitality detection. This study provides ideas and methods for automated shrimp fry quality detection, which has application value in aquaculture industry. [ABSTRACT FROM AUTHOR]

North East Monsoon (NEM) is the major source of rainfall for the south-eastern parts of peninsular India. Short time rainfall prediction data (i.e., nowcasting) are based on the observations from Doppler weather radars which has a high spatial and temporal resolution. This study focuses on the short-term ensemble prediction system using weather radar data to predict precipitation during the NEM and is the first of its kind in the Indian region to make an assessment of the operational performance of the prediction system. Six rainfall events have been studied for the assessment of short-term prediction system where the precipitation systems are different and include a tropical storm observed over different days during the 2022 NEM season. To assess the performance of the system, Fractional Skill Scores (FSS) at a 1 km window have been computed for a lead time of 0–2 h for all the rainfall events with more than 750 samples using different optical flow methods and ensemble sizes. The best average skill score and maximum skill score obtained at a 2 h lead time is 0.65 and 0.78 for tropical storms, 0.5 and 0.78 for stratiform and 0.15 and 0.38 for convective precipitation. It has found that the performance of the model is best for precipitation systems that are widespread and have a longer life period. [ABSTRACT FROM AUTHOR]

Accuracy and temporal resolution of flow meters are often unacceptable below the microliter per minute scale, limiting their ability to evaluate the real-time performance of many microfluidic devices. For conventional flow meters, this problem arises from uncertainties that depend on physical effects, such as evaporation, whose relative impacts scale inversely with flow rate. More advanced techniques that can measure nanoliter per minute flows are often not dynamic and require specialized equipment. Herein, we report on new experimental and theoretical results that overcome both limitations using an optofluidic flow meter. Previously, we showed that this device can measure flow rates as low as 1 nl/min with roughly 5% relative uncertainty by leveraging the photobleaching rate of a fluorescent dye. We now extend that work by determining the flow meter's relaxation time over a wide range of flow rates and incident irradiances. Using a simplified analytical model, we deduce that this time constant arises from the interplay between the photobleaching rate and transit time of the dye through the optical interrogation region. This motivates us to consider a more general model of the device, which, surprisingly, implies that all time constants are related by a simple scaling relationship depending only on the flow rate and optical irradiance. We experimentally validate this relationship to within 5% uncertainty down to 1 nl/min. Additionally, we measure a relaxation time of the flow meter on the order of 100 ms for 1 nl/min flows, demonstrating the ability to make dynamic measurements of small flows with unprecedented accuracy. [ABSTRACT FROM AUTHOR]

The 3D geometry understanding of dynamic scenes captured by moving cameras is one of the cornerstones of 3D scene understanding. Optical flow estimation, visual odometry, and depth estimation are the three most basic tasks in 3D geometry understanding. In this work, we present a unified framework for joint self-supervised learning of optical flow estimation, visual odometry, and depth estimation with two- and three-view geometric constraints. As we all know, visual odometry and depth estimation are more sensitive to dynamic objects, while optical flow estimation is more difficult to estimate the boundary area moved out of the image. To this end, we use estimated optical flow to help visual odometry and depth estimation process dynamic objects and use a rigid flow synthesized by the estimated pose and depth to help learn the optical flow of the area that moves out of the boundary due to camera motion. In order to further improve the consistency of cross-tasks, we introduce three-view geometric constraints and propose a three-view consistency loss. Finally, experiments on the KITTI data set show that our method can effectively improve the performance of the occluded boundary area and the dynamic object area. Moreover, our method achieves comparable or better performance than other monocular self-supervised state-of-the-art methods in these three subtasks. [ABSTRACT FROM AUTHOR]