Search

The growing use of Unmanned Aerial Vehicles (UAVs) raises the need to improve their autonomous navigation capabilities. Visual odometry allows for dispensing positioning systems, such as GPS, especially on indoor flights. This paper reports an effort toward UAV autonomous navigation by proposing a translational velocity observer based on inertial and visual measurements for a quadrotor. The proposed observer complementarily fuses available measurements from different domains and is synthesized following the Immersion and Invariance observer design technique. A formal Lyapunov-based observer error convergence to zero is provided. The proposed observer algorithm is evaluated using numerical simulations in the Parrot Mambo Minidrone App from Simulink-Matlab. [ABSTRACT FROM AUTHOR]

Compressive sensing provides a promising sampling paradigm for video acquisition for resource‐limited sensor applications. However, the reconstruction of original video signals from sub‐sampled measurements is still a great challenge. To exploit the temporal redundancies within videos during the recovery, previous works tend to perform alignment on initial reconstructions, which are too coarse to provide accurate motion estimations. To solve this problem, the authors propose a novel reconstruction network, named TSRN, for compressive video sensing. Specifically, the authors utilise a number of stacked temporal shift reconstruction blocks (TSRBs) to enhance the initial reconstruction progressively. Each TSRB could learn the temporal structures by exchanging information with last and next time step, and no additional computations is imposed on the network compared to regular 2D convolutions due to the high efficiency of temporal shift operations. After the enhancement, a bidirectional alignment module to build accurate temporal dependencies directly with the help of optical flows is employed. Different from previous methods that only extract supplementary information from the key frames, the proposed alignment module can receive temporal information from the whole video sequence via bidirectional propagations, thus yielding better performance. Experimental results verify the superiority of the proposed method over other state‐of‐the‐art approaches quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

FlowDenoising is a new approach to noise reduction in biological volumes obtained with three-dimensional electron microscopy (3DEM). Its abilities to enhance the structural features stem from the fact that an anisotropic Gaussian filtering is steered according to the local structures. To this end, the Optical Flow (OF) among consecutive slices is estimated, which is the most computationally expensive step in this approach. In this article, a hybrid CPU/GPU implementation of FlowDenoising is introduced and evaluated. It exploits parallel computing by distributing the workload among multiple cores and takes advantage of the massive processing in GPUs to accelerate the OF estimation. The hybrid implementation provides remarkable speed-up factors and an important reduction of the processing time, which is particularly relevant for the denoising of huge volumes typically found in 3DEM. [ABSTRACT FROM AUTHOR]

Existing homography and optical flow methods are erroneous in challenging scenes, such as fog, rain, night, and snow because the basic assumptions such as brightness and gradient constancy are broken. To address this issue, we present an unsupervised learning approach that fuses gyroscope into homography and optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Second, we design a self-guided fusion module (SGF) to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. Meanwhile, we propose a homography decoder module (HD) to combine gyro field and intermediate results of SGF to produce the homography. To the best of our knowledge, this is the first deep learning framework that fuses gyroscope data and image content for both deep homography and optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-the-art methods in both regular and challenging scenes. The code and dataset are available at https://github.com/lhaippp/GyroFlowPlus. [ABSTRACT FROM AUTHOR]

Background: Threat and individual differences in threat-processing bias perception of stimuli in the environment. Yet, their effect on perception of one's own (body-based) self-motion in space is unknown. Here, we tested the effects of threat on self-motion perception using a multisensory motion simulator with concurrent threatening or neutral auditory stimuli. Results: Strikingly, threat had opposite effects on vestibular and visual self-motion perception, leading to overestimation of vestibular, but underestimation of visual self-motions. Trait anxiety tended to be associated with an enhanced effect of threat on estimates of self-motion for both modalities. Conclusions: Enhanced vestibular perception under threat might stem from shared neural substrates with emotional processing, whereas diminished visual self-motion perception may indicate that a threatening stimulus diverts attention away from optic flow integration. Thus, threat induces modality-specific biases in everyday experiences of self-motion. [ABSTRACT FROM AUTHOR]

Accurate estimation of global winds is crucial for various scientific and practical applications, such as global chemical transport modeling and numerical weather prediction. One valuable source of wind measurements is atmospheric motion vectors (AMVs), which play a vital role in the global observing system and numerical weather prediction models. However, errors in AMV retrievals need to be addressed before their assimilation into data assimilation systems, as they can affect the accuracy of outputs. An assessment of the bias and uncertainty in passive-sensor AMVs can be done by comparing them with information from independent sources such as active-sensor winds. In this paper, we examine the benefit and performance of a colocation scheme using independent and sparse lidar wind observations as a dependent variable in a supervised machine learning model. We demonstrate the feasibility and performance of this approach in an observing system simulation experiment (OSSE) framework, with reference geophysical state data obtained from high-resolution Weather Research and Forecasting (WRF) model simulations of three different weather events. Lidar wind data are typically available in only one direction, and our study demonstrates that this single component of wind in high-precision active-sensor data can be leveraged (via a machine learning algorithm to model the conditional mean) to reduce the bias in the passive-sensor winds. Further, this active-sensor wind information can be leveraged through an algorithm that models the conditional quantiles to produce stable estimates of the prediction intervals, which are helpful in the design and application of error analysis, such as quality filters. [ABSTRACT FROM AUTHOR]

The sky may seem big enough for two flying vehicles to collide, but the facts show that mid-air collisions still occur occasionally and are a significant concern. Pilots learn manual tactics to avoid collisions, such as see-and-avoid, but these rules have limitations. Automated solutions have reduced collisions, but these technologies are not mandatory in all countries or airspaces, and they are expensive. These problems have prompted researchers to continue the search for low-cost solutions. One attractive solution is to use computer vision to detect obstacles in the air due to its reduced cost and weight. A well-trained deep learning solution is appealing because object detection is fast in most cases, but it relies entirely on the training data set. The algorithm chosen for this study is optical flow. The optical flow vectors can help us to separate the motion caused by camera motion from the motion caused by incoming objects without relying on training data. This paper describes the development of an optical flow-based airborne obstacle detection algorithm to avoid mid-air collisions. The approach uses the visual information from a monocular camera and detects the obstacles using morphological filters, optical flow, focus of expansion, and a data clustering algorithm. The proposal was evaluated using realistic vision data obtained with a self-developed simulator. The simulator provides different environments, trajectories, and altitudes of flying objects. The results showed that the optical flow-based algorithm detected all incoming obstacles along their trajectories in the experiments. The results showed an F-score greater than 75% and a good balance between precision and recall. [ABSTRACT FROM AUTHOR]

It is a challenging and meaningful task to carry out UAV-based livestock monitoring in high-altitude (more than 4500 m on average) and cold regions (annual average – 4 °C) on the Qinghai Tibet Plateau. The purpose of artificial intelligence (AI) is to execute automated tasks and to solve practical problems in actual applications by combining the software technology with the hardware carrier to create integrated advanced devices. Only in this way, the maximum value of AI could be realized. In this paper, a real-time tracking system with dynamic target tracking ability is proposed. It is developed based on the tracking-by-detection architecture using YOLOv7 and Deep SORT algorithms for target detection and tracking, respectively. In response to the problems encountered in the tracking process of complex and dense scenes, our work (1) Uses optical flow to compensate the Kalman filter, to solve the problem of mismatch between the target bounding box predicted by the Kalman filter (KF) and the input when the target detection in the current frame is complex, thereby improving the prediction accuracy; (2) Using a low confidence trajectory filtering method to reduce false positive trajectories generated by Deep SORT, thereby mitigating the impact of unreliable detection on target tracking. (3) A visual servo controller has been designed for the Unmanned Aerial Vehicle (UAV) to reduce the impact of rapid movement on tracking and ensure that the target is always within the field of view of the UAV camera, thereby achieving automatic tracking tasks. Finally, the system was tested using Tibetan yaks on the Qinghai Tibet Plateau as tracking targets, and the results showed that the system has real-time multi tracking ability and ideal visual servo effect in complex and dense scenes. [ABSTRACT FROM AUTHOR]

To solve the problem of poor localization accuracy and robustness of visual simultaneous localization and mapping (SLAM) systems in highly dynamic environments, this paper proposes a dynamic visual SLAM algorithm called FFD-SLAM that fuses the target detection network with the optical flow method. The algorithm considers ORB-SLAM2 as the basic framework, joins the semantic thread in parallel with its tracking thread, initially obtains the set of feature points through the real-time detection of dynamic objects in the environment through YOLOv5 in the semantic thread, then filters the set of feature points obtained in the semantic thread through the optical flow module, and finally utilizes the remaining static feature points for the matching calculation. Experiments showed that the proposed algorithm showed an improvement of approximately 97% in the localization accuracy compared with the ORB-SLAM2 algorithm in a highly dynamic environment, which effectively improves the localization accuracy and robustness of the system. The proposed algorithm also showed a higher real-time performance compared with some excellent dynamic SLAM algorithms. [ABSTRACT FROM AUTHOR]

Motion object detection (MOD) with freely moving cameras is a challenging task in computer vision. To extract moving objects, most studies have focused on the difference in motion features between foreground and background, which works well for dynamic scenes with relatively regular movements and variations. However, abrupt illumination changes and occlusions often occur in real-world scenes, and the camera may also pan, tilt, rotate, and jitter, etc., resulting in local irregular variations and global discontinuities in motion features. Such complex and changing scenes bring great difficulty in detecting moving objects. To solve this problem, this paper proposes a new MOD method that effectively leverages local and global visual information for foreground/background segmentation. Specifically, on the global side, to support a wider range of camera motion, the relative inter-frame transformations are optimized to absolute transformations referenced to intermediate frames in a global form after enriching the inter-frame matching pairs. The global transformation is fine-tuned using the spatial transformer network (STN). On the local side, to address the problem of dynamic background scenes, foreground object detection is optimized by utilizing the pixel differences between the current frame and the local background model, as well as the consistency of local spatial variations. Then, the spatial information is combined using optical flow segmentation methods, enhancing the precision of the object information. The experimental results show that our method achieves a detection accuracy improvement of over 1.5% compared with the state-of-the-art methods on the datasets of CDNET2014, FBMS-59, and CBD. It demonstrates significant effectiveness in challenging scenarios such as shadows, abrupt changes in illumination, camera jitter, occlusion, and moving backgrounds. [ABSTRACT FROM AUTHOR]

With the development of simultaneous positioning and mapping technology in the field of automatic driving, the current simultaneous localization and mapping scheme is no longer limited to a single sensor and is developing in the direction of multi-sensor fusion to enhance the robustness and accuracy. In this study, a localization and mapping scheme named LVI-fusion based on multi-sensor fusion of camera, lidar and IMU is proposed. Different sensors have different data acquisition frequencies. To solve the problem of time inconsistency in heterogeneous sensor data tight coupling, the time alignment module is used to align the time stamp between the lidar, camera and IMU. The image segmentation algorithm is used to segment the dynamic target of the image and extract the static key points. At the same time, the optical flow tracking based on the static key points are carried out and a robust feature point depth recovery model is proposed to realize the robust estimation of feature point depth. Finally, lidar constraint factor, IMU pre-integral constraint factor and visual constraint factor together construct the error equation that is processed with a sliding window-based optimization module. Experimental results show that the proposed algorithm has competitive accuracy and robustness. [ABSTRACT FROM AUTHOR]

Hierarchical concepts have proven useful in many classical and learning-based optical flow methods regarding both accuracy and robustness. In this paper we show that such concepts are still useful in the context of recent neural networks that follow RAFT's paradigm refraining from hierarchical strategies by relying on recurrent updates based on a single-scale all-pairs transform. To this end, we introduce MS-RAFT+: a novel recurrent multi-scale architecture based on RAFT that unifies several successful hierarchical concepts. It employs a coarse-to-fine estimation to enable the use of finer resolutions by useful initializations from coarser scales. Moreover, it relies on RAFT's correlation pyramid that allows to consider non-local cost information during the matching process. Furthermore, it makes use of advanced multi-scale features that incorporate high-level information from coarser scales. And finally, our method is trained subject to a sample-wise robust multi-scale multi-iteration loss that closely supervises each iteration on each scale, while allowing to discard particularly difficult samples. In combination with an appropriate mixed-dataset training strategy, our method performs favorably. It not only yields highly accurate results on the four major benchmarks (KITTI 2015, MPI Sintel, Middlebury and VIPER), it also allows to achieve these results with a single model and a single parameter setting. Our trained model and code are available at https://github.com/cv-stuttgart/MS_RAFT_plus. [ABSTRACT FROM AUTHOR]

In current times, reconstruction of remote sensing images using super-resolution is a prominent topic of study. Remote sensing data have a complex spatial distribution. Compared with natural pictures, remote sensing pictures often contain subtler and more complicated information. Most super-resolution reconstruction algorithms cannot restore all the information contained in remote sensing images when reconstructing them. The content of some areas in the reconstructed images may be too smooth, and some areas may even have color changes, resulting in lower quality reconstructed images. In response to the problems presenting in current reconstruction algorithms about super-resolution, this article proposes the SRGAN-MSAM-DRC model (SRGAN model with multi-scale attention mechanism and dense residual connection). This model roots in generative adversarial networks and incorporates multi-scale attention mechanisms and dense residual connections into the generator. Furthermore, residual blocks are incorporated into the discriminator. We use some remote sensing image datasets of real-world data to evaluate this model, and the results indicate the SRGAN-MSAM-DRC model has shown enhancements in three evaluation metrics for reconstructed images about super-resolution. Compared to the basic SRGAN model, the SSIM (structural similarity), PSNR (peak signal-to-noise ratio), and IE (image entropy) increase by 5.0%, 4.0%, and 4.1%, respectively. From the results, we know the quality of the reconstructed images of remote sensing using the SRGAN-MSAM-DRC model is better than basic SRGAN model, and verifies that the model has good applicability and performance in reconstruction of remote sensing images using super-resolution. [ABSTRACT FROM AUTHOR]

To resolve high-precision deformation from low-quality speckle images, a high-order time-domain digital image correlation algorithm is proposed. This algorithm is based on the nonlinear optical flow equation for low-quality speckle images collected in extreme testing environments. A subpixel displacement image containing Gaussian white noise is established by using the simulated speckle method. The displacement of the high-order time-domain digital image correlation algorithm is solved by the least squares iteration method, and the test accuracy of the proposed algorithm is analyzed. The results show that the nonlinear optical flow equation can describe the time continuity of pixel gray level changes before and after deformation. The proposed algorithm can also analyze the velocity and acceleration term of deformation. The high-order time-domain digital image correlation algorithm is insensitive to the change in image grayscale and has high measurement accuracy. Therefore, this method can solve the problem of resolving high-precision deformation from low-quality speckle images. [ABSTRACT FROM AUTHOR]

Video super-resolution (VSR) remains challenging for real-world applications due to complex and unknown degradations. Existing methods lack the flexibility to handle video sequences with different degradation levels, thus failing to reflect real-world scenarios. To address this problem, we propose a degradation-adaptive video super-resolution network (DAVSR) based on a bidirectional propagation network. Specifically, we adaptively employ three distinct degradation levels to process input video sequences, aiming to obtain training pairs that reflect a variety of real-world corrupted images. We also equip the network with a pre-cleaning module to reduce noise and artifacts in the low-quality video sequences prior to information propagation. Additionally, compared to previous flow-based methods, we employ an unsupervised optical flow estimator to acquire a more precise optical flow to guide inter-frame alignment. Meanwhile, while maintaining network performance, we streamline the propagation network branches and the structure of the reconstruction module of the baseline network. Experiments are conducted on datasets with diverse degradation types to validate the effectiveness of DAVSR. Our method exhibits an average improvement of 0.18 dB over a recent SOTA approach (DBVSR) in terms of the PSNR metric. Extensive experiments demonstrate the effectiveness of our network in handling real-world video sequences with different degradation levels. [ABSTRACT FROM AUTHOR]

This paper proposes a new sensor using optical flow to stabilize a quadrotor when a GPS signal is not available. Normally, optical flow varies with the attitude of the aerial vehicle. This produces positive feedback on the attitude control that destabilizes the orientation of the vehicle. To avoid this, we propose a novel sensor using an optical flow camera with a 6DoF IMU (Inertial Measurement Unit) mounted on a two-axis anti-shake stabilizer mobile aerial gimbal. We also propose a robust algorithm based on Sliding Mode Control for stabilizing the optical flow sensor downwards independently of the aerial vehicle attitude. This method improves the estimation of the position and velocity of the quadrotor. We present experimental results to show the performance of the proposed sensor and algorithms. [ABSTRACT FROM AUTHOR]

We propose an adaptive fractional multi-scale optimization optical flow algorithm, which for the first time improves the over-smoothing of optical flow estimation under the total variation model from the perspective of global feature and local texture balance, and solves the problem that the convergence of fractional optical flow algorithms depends on the order parameter. Specifically, a fractional-order discrete L1-regularization Total Variational Optical Flow model is constructed. On this basis, the Ant Lion algorithm is innovatively used to realize the iterative calculation of the optical flow equation, and the fractional order is dynamically adjusted to obtain an adaptive optimization algorithm with strong search accuracy and high efficiency. In this paper, the flexibility of optical flow estimation in weak gradient texture scenes is increased, and the optical flow extraction rate of target features at multiple scales is greatly improved. We show excellent recognition performance and stability under the MPI_Sintel and Middlebury benchmarks. [ABSTRACT FROM AUTHOR]

Background: X-ray imaging addresses many challenges with visible light imaging in extreme environments, where optical diagnostics such as digital image correlation (DIC) and particle image velocimetry (PIV) suffer biases from index of refraction changes and/or cannot penetrate visibly occluded objects. However, conservation of intensity—the fundamental principle of optical image correlation algorithms—may be violated if ancillary components in the X-ray path besides the surface or fluid of interest move during the test. Objective: The test series treated in this work sought to characterize the safe use of fiber-epoxy composites in aerospace and aviation industries during fire accident scenarios. Stereo X-ray DIC was employed to measure test unit deformation in an extreme thermal environment involving a visibly occluded test unit, incident surface heating to temperatures above 600oC, and flames and soot from combusting epoxy decomposition gasses. The objective of the current work is to evaluate two solutions to resolve the violation of conservation of intensity that resulted from both the test unit and the thermal chamber deforming during the test. Methods: The first solution recovered conservation of intensity by pre-processing the path-integrated X-ray images to isolate the DIC pattern of the test unit from the thermal chamber components. These images were then correlated with standard, optical DIC software. The second solution, called Path-Integrated Digital Image Correlation (PI-DIC), modified the fundamental matching criterion of DIC to account for multiple, independently-moving components contributing to the final image intensity. The PI-DIC algorithm was extended from a 2D framework to a stereo framework and implemented in a custom DIC software. Results: Both solutions accurately measured the cylindrical shape of the undeformed test unit, recovering radii values of R = 76.20 ± 0.12 mm compared to the theoretical radius of R theor = 76.20 mm. During the test, the test unit bulged asymmetrically as decomposition gasses pressurized the interior and eventually burned in a localized jet. Both solutions measured the heterogeneous radius of this bulge, which reached a maximum of approximately R = 91 mm, with a discrepancy of 2–3% between the two solutions. Conclusions: Two solutions that resolve the violation of conservation of intensity for path-integrated X-ray images were developed. Both were successfully employed in a stereo X-ray DIC configuration to measure deformation of an aluminum-skinned, fiber-epoxy composite test unit in a fire accident scenario. [ABSTRACT FROM AUTHOR]

Recent optical flow estimators usually employ deep models designed for image classification as the encoders for feature extraction and matching. However, those encoders developed for image classification may be sub-optimal for flow estimation. In contrast, the decoder design of optical flow estimators often requires meticulous design for flow estimation. The disconnect between the encoder and decoder could negatively affect optical flow estimation. To address this issue, we propose a neural architecture search method, FlowNAS, to automatically find the more suitable and stronger encoder architecture for existing flow decoders. We first design a suitable search space, including various convolutional operators, and construct a weight-sharing super-network for efficiently evaluating the candidate architectures. To better train the super-network, we present a Feature Alignment Distillation module that utilizes a well-trained flow estimator to guide the training of the super-network. Finally, a resource-constrained evolutionary algorithm is exploited to determine an optimal architecture (i.e., sub-network). Experimental results show that FlowNAS can be easily incorporated into existing flow estimators and achieves state-of-the-art performance with the trade-off between accuracy and efficiency. Furthermore, the encoder architecture discovered by FlowNAS with the weights inherited from the super-network achieves 4.67% F1-all error on KITTI, an 8.4% reduction of RAFT baseline, surpassing state-of-the-art handcrafted GMA and AGFlow models, while reducing the model complexity and latency. The source code and trained models will be released at https://github.com/VDIGPKU/FlowNAS. [ABSTRACT FROM AUTHOR]

Edge computing is essential to handle increasing data volumes and processing capacities. It provides real-time and secure data processing near data sources, like smart devices, alleviating cloud computing energy use, and saving network bandwidth. Specialized accelerators, like GPUs and FPGAs, are vital for low-latency edge computing but the requirements to customized code for different hardware and vendors suppose important compatibility issues. This paper evaluates the potential of SYCL in addressing code portability issues encountered in edge computing. We employed the Polybench suite to compare various SYCL implementations, specifically DPC++ and AdaptiveCpp, with the native solution, CUDA. The disparity between SYCL implementations was negligible, at just 5%. Furthermore, we evaluated SYCL in the context of specific edge computing applications such as video processing using three different optical flow algorithms. The results revealed a slight performance gap of 3% when transitioning from CUDA to SYCL. Upon evaluating energy consumption, the observed difference ranged from ± 10 % , depending on the application utilized. These gaps are the price one may need to pay when achieving the ability to successfully run the same code on two distinct edge boards. These findings underscore SYCL's capacity to increase productivity in terms of development costs and facilitate IoT deployment without being locked into a particular platform or manufacturer. [ABSTRACT FROM AUTHOR]

Due to the complexity of real optical flow capture, the existing research still has not performed real optical flow capture of infrared (IR) images with the production of an optical flow based on IR images, which makes the research and application of deep learning-based optical flow computation limited to the field of RGB images only. Therefore, in this paper, we propose a method to produce an optical flow dataset of IR images. We utilize the RGB-IR cross-modal image transformation network to rationally transform existing RGB image optical flow datasets. The RGB-IR cross-modal image transformation is based on the improved Pix2Pix implementation, and in the experiments, the network is validated and evaluated using the RGB-IR aligned bimodal dataset M3FD. Then, RGB-IR cross-modal transformation is performed on the existing RGB optical flow dataset KITTI, and the optical flow computation network is trained using the IR images generated by the transformation. Finally, the computational results of the optical flow computation network before and after training are analyzed based on the RGB-IR aligned bimodal data. [ABSTRACT FROM AUTHOR]

Motion information from image sequences (videos) plays a key role in solving a number of real-world problems such as surveillance, traffic management, fire identification, weather prediction, and COVID-19 detection, etc. Generally, this motion is estimated in terms of optical flow between two consecutive image frames. Optical flow is a 2D vector field that illustrates object motion behavior. This paper presents a nonlinear fractional order variational model for estimating optical flow. The objective of this work is to provide dense and discontinuity preserving optical flow for different spectra and make the model robust against outliers. In particular, the presented model generalizes the integer order variational models for fractional order (0, 1). For this purpose, the proposed model is formulated with the help of Charbonnier norm and Caputo fractional derivative. Charbonnier norm is nonlinear in nature, which makes the model robust against noise and outliers, whereas Caputo fractional derivatives are well-capable to deal with discontinuous functions such as images, and therefore, preserve motion discontinuities. The Caputo derivative also manifests long-term memory effect and allows to choose the optimal value of the fractional order that corresponds to a stable solution. The numerical implementation of the formulated variational functional is performed by discretizing the fractional derivative using the Grünwald–Letnikov derivative. The resulting system of equations is solved using multi-variable fixed point iteration scheme. The entire framework is embedded in coarse-to-fine warping strategy, which helps in finding the global extremal. The experimental results are carried out on 20 different application oriented datasets such as fire and smoke, fluid, CXR, etc. The performance of the model is tested using different error measures and demonstrated against several outliers. Error concentration is shown through 3D histograms. The edge preserving nature is also realized through intensity distribution in RGB color channels. A detailed convergence analysis is also provided for the presented model. The validity of the proposed model is also verified through comparisons with other existing models. [ABSTRACT FROM AUTHOR]

This paper solves the task of unsupervised video object segmentation (UVOS) that segments the objects of interest through the entire videos without any annotation. In recent years, many unsupervised video object segmentation (UVOS) methods have been proposed. Although these methods perform well, they rely on networks with heavy weights, often leading to large model size. In order to reduce the model size while keeping a competitive performance, we propose a saliency-based dual-attention (SDA) method for UVOS in this paper. In our method, we take optical flow and video frames as inputs and extract the appearance information and motion information from optical flow and video frames. We design a two-branch network with appearance information and motion information. The information from these two branches is fused via a saliency-based dual-attention module to segment the primary object in one path. The saliency-based dual-attention module is composed of saliency attention and saliency-based reverse attention. To demonstrate the effectiveness of our network, we tested it on the DAVIS-2016 and SegtrackV2 datasets. Experimental results demonstrate that our method can achieve competitive results in terms of accuracy and model size. [ABSTRACT FROM AUTHOR]

With the goal of solving the problem of feature extractors lacking strong supervision training and insufficient time information concerning single‐sequence model learning, a hierarchical sequence memory network with a multi‐level iterative optimisation strategy is proposed for continuous sign language recognition. This method uses the spatial‐temporal fusion convolution network (STFC‐Net) to extract the spatial‐temporal information of RGB and Optical flow video frames to obtain the multi‐modal visual features of a sign language video. Then, in order to enhance the temporal relationships of visual feature maps, the hierarchical memory sequence network is used to capture local utterance features and global context dependencies across time dimensions to obtain sequence features. Finally, the decoder decodes the final sentence sequence. In order to enhance the feature extractor, the authors adopted a multi‐level iterative optimisation strategy to fine‐tune STFC‐Net and the utterance feature extractor. The experimental results on the RWTH‐Phoenix‐Weather multi‐signer 2014 dataset and the Chinese sign language dataset show the effectiveness and superiority of this method. [ABSTRACT FROM AUTHOR]

The interest of science community towards planet Mars has progressively increased in past decades. Rovers are already doing an excellent job in exploring the Martian surface environment and showed that the Red Planet is not suited for life. However, there is a possibility of enabling human exploration by having astronauts take shelter in habitats located within lava tubes, which can provide protection against radiation and dust storms. These tubes are extremely hard to explore with rovers, thus a flying vehicle will be more suitable for such tasks. INGENUITY (Martian helicopter) had already demonstrated that it is possible to fly on Mars by doing small and simple manoeuvres. Based on NASA's design, this article will introduce the development of guidance, navigation, and control operations of a Martian Inspection Drone (MID) that will be capable of finding the lava tube and scan it in real time using a RealSense camera. Linux based system was chosen in the development process, which allow ROS catkin workspace to connect ArduPilot and Gazebo. The navigation is accomplished using a combination of an inertial measurement unit, a camera, and a laser rangefinder. The flight computer runs an optical flow algorithm for control and navigation purpose. Also, a convolutional neural network is used to perform lava tube entrance detection and inspection. • Martian flying drone simulated with ArduPilot, Gazebo and ROS. • Martian environment simulation. • Martian lava tube detection using convolutional neural networks. • Martian lava tube inspection using a realsense camera. [ABSTRACT FROM AUTHOR]

The spatter generated by the interaction between laser and powder during Powder Bed Fusion-Laser Melting (PBF-LM) can significantly affect the quality of printed parts. A high-speed camera is used to observe the dynamic process of spatter's behavior under different layer thickness and laser powers during the printing process, and to analyze the printed samples' surface roughness, microstructure, and mechanical properties. In terms of spatter image processing, employing an optical flow approach to track and quantify the number of spatters efficiently eliminates statistical redundancy and improves statistical correctness. It is found that under the same laser power, the number of spatters produced by the laser scan direction with the gas flow (LSD-W) is more than that by the laser scan direction against the gas flow (LSD-A), and the number of spatters produced increases with the increase of laser power. Analyzing the mechanical properties and surface roughness of the printed samples under different process parameters quantitatively reveals that differences in the spatter amount generated under different process parameters in the PBF-LM process is not the determining factor affecting the difference in tensile strength of printed parts. During LSD-W, the number of spatters generated at laser power of 170 W and layer thickness of 0.03 mm is 87, and the tensile strength of the printed sample is 618 MPa. During LSD-W, the number of spatters generated at laser power of 320 W and layer thickness of 0.05 mm is 211, and the tensile strength of the printed sample is 680 MPa. Instead, spatter generation has a more direct impact on the surface roughness of printed parts. The layer thickness is 0.03 mm, the laser power is 170 W, and (Ra = 2.372 μm) is the surface roughness of the sample. The layer thickness is 0.05 mm, the laser power is 320 W, and (Ra = 8.163 μm) is the surface roughness of the sample. [ABSTRACT FROM AUTHOR]

In this paper, we address the challenge of detecting small moving targets in dynamic environments characterized by the concurrent movement of both platform and sensor. In such cases, simple image-based frame registration and optical flow analysis cannot be used to detect moving targets. To tackle this, it is necessary to use sensor and platform meta-data in addition to image analysis for temporal and spatial anomaly detection. To this end, we investigate techniques that utilize inertial data to enhance frame-to-frame registration, consistently yielding improved detection outcomes when compared against purely feature-based techniques. For cases where image registration is not possible even with metadata, we propose single-frame spatial anomaly detection and then estimate the range to the target using the platform velocity. The behavior of the estimated range over time helps us to discern targets from clutter. Finally, we show that a KNN classifier can be used to further reduce the false alarm rate without a significant reduction in detection performance. The proposed strategies offer a robust solution for the detection of moving targets in dynamically challenging settings. [ABSTRACT FROM AUTHOR]

North East Monsoon (NEM) is the major source of rainfall for the south-eastern parts of peninsular India. Short time rainfall prediction data (i.e., nowcasting) are based on the observations from Doppler weather radars which has a high spatial and temporal resolution. This study focuses on the short-term ensemble prediction system using weather radar data to predict precipitation during the NEM and is the first of its kind in the Indian region to make an assessment of the operational performance of the prediction system. Six rainfall events have been studied for the assessment of short-term prediction system where the precipitation systems are different and include a tropical storm observed over different days during the 2022 NEM season. To assess the performance of the system, Fractional Skill Scores (FSS) at a 1 km window have been computed for a lead time of 0–2 h for all the rainfall events with more than 750 samples using different optical flow methods and ensemble sizes. The best average skill score and maximum skill score obtained at a 2 h lead time is 0.65 and 0.78 for tropical storms, 0.5 and 0.78 for stratiform and 0.15 and 0.38 for convective precipitation. It has found that the performance of the model is best for precipitation systems that are widespread and have a longer life period. [ABSTRACT FROM AUTHOR]

The critical goal of gait recognition is to acquire the inter-frame walking habit representation from the gait sequences. The relations between frames, however, have not received adequate attention in comparison to the intra-frame features. In this paper, motivated by optical flow, the bilateral motion-oriented block is proposed to explore motion description at the feature level. It can allow the classic 2D convolutional structure to have the capability to directly portray gait movement patterns while preventing costly computations on the estimation of optical flow. Based on such features, we develop a set of multi-scale temporal representations that force the motion context to be richly described at various levels of temporal resolution. Furthermore, the dynamic information is sensitive to inaccurate segmentation on the edge, so a correction block is devised to eliminate the segmentation noise of silhouettes for getting more precise gait modality. Subsequently, the temporal feature set and the spatial features are combined to comprehensively characterize gait processes. Extensive experiments are conducted on CASIA-B and OU-MVLP datasets, and the results achieve an outstanding identification performance, which has demonstrated the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

To accurately describe the thermal comfort of indoor occupant used to regulate heating, ventilation and air conditioning (HVAC) system is a key goal to reduce energy consumption in intelligent buildings. In this paper, we propose a noncontact measurement of occupant thermal discomfort behavior as an index of thermal comfort. The method takes the existing monitoring image data in the building as the input to infer the occupant thermal discomfort directly, which saves the cost because there is no need to install new sensors in the building and the occupant does not need to wear additional equipment. The framework combined three channels of body posture, motion and performance information to infer occupant thermal discomfort behavior, it consists of a human detection and crop module, a posture analysis module, an optical flow extraction module and a 3D convolutional neural network module. The three channels describe the actions from different perspectives, and the proposed method makes full use of the complementarity of the three modalities to identify the occupant's thermal discomfort behaviors. Sixteen postures related to thermal discomfort were identified through a questionnaire, and 14,800 video clips containing these postures were collected for experimental evaluation. The results demonstrate the superior performances of our approach to the state-of-the-art techniques. The framework achieves noninvasive, cost-effective thermal comfort evaluation, and has potential value in improving energy efficiency of HVAC systems. [ABSTRACT FROM AUTHOR]

In this paper, we focus on exploring effective methods for faster and accurate semantic segmentation. A common practice to improve the performance is to attain high-resolution feature maps with strong semantic representation. Two strategies are widely used: atrous convolutions and feature pyramid fusion, while both are either computationally intensive or ineffective. Inspired by the Optical Flow for motion alignment between adjacent video frames, we propose a Flow Alignment Module (FAM) to learn Semantic Flow between feature maps of adjacent levels and broadcast high-level features to high-resolution features effectively and efficiently. Furthermore, integrating our FAM to a standard feature pyramid structure exhibits superior performance over other real-time methods, even on lightweight backbone networks, such as ResNet-18 and DFNet. Then to further speed up the inference procedure, we also present a novel Gated Dual Flow Alignment Module to directly align high-resolution feature maps and low-resolution feature maps where we term the improved version network as SFNet-Lite. Extensive experiments are conducted on several challenging datasets, where results show the effectiveness of both SFNet and SFNet-Lite. In particular, when using Cityscapes test set, the SFNet-Lite series achieve 80.1 mIoU while running at 60 FPS using ResNet-18 backbone and 78.8 mIoU while running at 120 FPS using STDC backbone on RTX-3090. Moreover, we unify four challenging driving datasets (i.e., Cityscapes, Mapillary, IDD, and BDD) into one large dataset, which we named Unified Driving Segmentation (UDS) dataset. It contains diverse domain and style information. We benchmark several representative works on UDS. Both SFNet and SFNet-Lite still achieve the best speed and accuracy trade-off on UDS, which serves as a strong baseline in such a challenging setting. The code and models are publicly available at https://github.com/lxtGH/SFSegNets. [ABSTRACT FROM AUTHOR]

We propose a deep multi-stream model for left ventricular ejection fraction (LVEF) prediction in 2D echocardiographic (2DE) examinations. We use four standard 2DE views as model input, which are automatically selected from the full 2DE examination. The LVEF prediction model processes eight streams of data (images + optical flow) and consists of convolutional neural networks terminated with transformer layers. The model is made robust to missing, misclassified and duplicate views via pre-training, sampling strategies and parameter sharing. The model is trained and evaluated on an existing clinical dataset (12,648 unique examinations) with varying properties in terms of quality, examining physician, and ultrasound system. We report R 2 = 0.84 and mean absolute error = 4.0% points for the test set. When evaluated on two public benchmarks, the model performs on par or better than all previous attempts on fully automatic LVEF prediction. Code and trained models are available on a public project repository. [ABSTRACT FROM AUTHOR]

Detecting violent behavior in videos to ensure public safety and security poses a significant challenge. Precisely identifying and categorizing instances of violence in real-life closed-circuit television, which vary across specifications and locations, requires comprehensive understanding and processing of the sequential information embedded in these videos. This study aims to introduce a model that adeptly grasps the spatiotemporal context of videos within diverse settings and specifications of violent scenarios. We propose a method to accurately capture spatiotemporal features linked to violent behaviors using optical flow and RGB data. The approach leverages a Conv3D-based ResNet-3D model as the foundational network, capable of handling high-dimensional video data. The efficiency and accuracy of violence detection are enhanced by integrating an attention mechanism, which assigns greater weight to the most crucial frames within the RGB and optical-flow sequences during instances of violence. Our model was evaluated on the UBI-Fight, Hockey, Crowd, and Movie-Fights datasets; the proposed method outperformed existing state-of-the-art techniques, achieving area under the curve scores of 95.4, 98.1, 94.5, and 100.0 on the respective datasets. Moreover, this research not only has the potential to be applied in real-time surveillance systems but also promises to contribute to a broader spectrum of research in video analysis and understanding. [ABSTRACT FROM AUTHOR]

Satellite time-series data contain information in three dimensions—spatial, spectral, and temporal—and are widely used for monitoring, simulating, and evaluating Earth activities. However, some time-phase images in the satellite time series data are missing due to satellite sensor malfunction or adverse atmospheric conditions, which prevents the effective use of the data. Therefore, we need to complement the satellite time series data with sequence image interpolation. Linear interpolation methods and deep learning methods that have been applied to sequence image interpolation lead to large errors between the interpolation results and the real images due to the lack of accurate estimation of pixel positions and the capture of changes in objects. Inspired by video frame interpolation, we combine optical flow estimation and deep learning and propose a method named Multi-Scale Optical Flow-Intermediate Feature Joint Network. This method learns pixel occlusion and detailed compensation information for each channel and jointly refines optical flow and intermediate features at different scales through an end-to-end network together. In addition, we set a spectral loss function to optimize the network's learning of the spectral features of satellite images. We have created a time-series dataset using Landsat-8 satellite data and Sentinel-2 satellite data and then conducted experiments on this dataset. Through visual and quantitative evaluation of the experimental results, we discovered that the interpolation results of our method retain better spectral and spatial consistency with the real images, and that the results of our method on the test dataset have a 7.54% lower Root Mean Square Error than other approaches. [ABSTRACT FROM AUTHOR]

This work presents a novel RGB-D dynamic Simultaneous Localisation and Mapping (SLAM) method that improves the precision, stability, and efficiency of localisation while relying on lightweight deep learning in a dynamic environment compared to the traditional static feature-based visual SLAM algorithm. Based on ORB-SLAM3, the GCNv2-tiny network instead of the ORB method, improves the reliability of feature extraction and matching and the accuracy of position estimation; then, the semantic segmentation thread employs the lightweight YOLOv5s object detection algorithm based on the GSConv network combined with a depth image to determine potentially dynamic regions of the image. Finally, to guarantee that the static feature points are used for position estimation, dynamic probability is employed to determine the true dynamic feature points based on the optical flow, semantic labels, and the state in last frame. We have performed experiments on the TUM datasets to verify the feasibility of the algorithm. Compared with the classical dynamic visual SLAM algorithm, the experimental results demonstrate that the absolute trajectory error is greatly reduced in dynamic environments, and that the computing efficiency is improved by 31.54% compared with the real-time dynamic visual SLAM algorithm with close accuracy, demonstrating the superiority of DLD-SLAM in accuracy, stability, and efficiency. [ABSTRACT FROM AUTHOR]

We investigated the assessment of blood flow within polypoidal lesions using optical coherence tomography angiography (OCTA) to determine intravitreal brolucizumab (IVBr) efficacy for treating polypoidal choroidal vasculopathy (PCV). We retrospectively studied 46 eyes with PCV that completed 1-year IVBr treatment. Blood flow signals within polypoidal lesions were evaluated using OCTA after loading-phase treatment, and 1-year outcomes were compared between eyes in which blood flow signals disappeared versus persisting. After loading-phase treatment, blood flow signals within polypoidal lesions disappeared in 31 eyes and persisted in 15. In the former group, visual acuity improved significantly throughout the year (P < 0.01), while in the latter there was no significant difference between baseline and after 1 year. The total number of injections was significantly lower with than without disappearance of blood flow signals (6.0 vs. 6.9, P < 0.01). The intended injection interval at the last visit was significantly longer in the former than in the latter group (15.7 weeks vs. 12.5 weeks, P < 0.01). These results indicate that PCV cases showing disappearance of blood flow signals within polypoidal lesions by OCTA after loading-phase treatment had favorable 1-year outcomes of IVBr. Therefore, evaluating blood flow within polypoidal lesions by OCTA may allow noninvasive prediction of PCV treatment outcomes. [ABSTRACT FROM AUTHOR]

The ability to predict the future based on past experience lies at the core of the brain's ability to adapt behavior. However, the neural mechanisms that participate in generating and updating predictions are not clearly understood. Further, the evolutionary antecedents and the prevalence of predictive processing among vertebrates are even less explored. Here, we show evidence of predictive processing via the involvement of cerebellar circuits in larval zebrafish. We presented stereotyped optic flow stimuli to larval zebrafish to evoke swims and discovered that lesioning the cerebellum abolished prediction-dependent modulation of swim latency. When expectations of optic flow direction did not match with reality, error signals arrive at Purkinje cells via the olivary climbing fibers, whereas granule cells and Purkinje cells encode signals of expectation. Strong neural representations of expectation correlate with faster swim responses and vice versa. In sum, our results show evidence for predictive processing in nonmammalian vertebrates with the involvement of cerebellum, an evolutionarily conserved brain structure. [ABSTRACT FROM AUTHOR]

Forecasting heavy precipitation accurately is a challenging task for most deep learning (DL)-based models. To address this, we present a novel DL architecture called "multi-scale feature fusion" (MFF) that can forecast precipitation with a lead time of up to 3 h. The MFF model uses convolution kernels with varying sizes to create multi-scale receptive fields. This helps to capture the movement features of precipitation systems, such as their shape, movement direction, and speed. Additionally, the architecture utilizes the mechanism of discrete probability to reduce uncertainties and forecast errors, enabling it to predict heavy precipitation even at longer lead times. For model training, we use 4 years of radar echo data from 2018 to 2021 and 1 year of data from 2022 for model testing. We compare the MFF model with three existing extrapolative models: time series residual convolution (TSRC), optical flow (OF), and UNet. The results show that MFF achieves superior forecast skills with high probability of detection (POD), low false alarm rate (FAR), small mean absolute error (MAE), and high structural similarity index (SSIM). Notably, MFF can predict high-intensity precipitation fields at 3 h lead time, while the other three models cannot. Furthermore, MFF shows improvement in the smoothing effect of the forecast field, as observed from the results of radially averaged power spectral (RAPS). Our future work will focus on incorporating multi-source meteorological variables, making structural adjustments to the network, and combining them with numerical models to further improve the forecast skills of heavy precipitations at longer lead times. [ABSTRACT FROM AUTHOR]

The ability to perceive biological motion is crucial for human survival, social interactions, and communication. Over the years, researchers have studied the mechanisms and neurobiological substrates that enable this ability. In a previous study, we proposed a descriptive Bayesian simulation model to represent the dorsal pathway of the visual system, which processes motion information. The model was inspired by recent studies that questioned the impact of dynamic form cues in biological motion perception and was trained to distinguish the direction of a soccer ball from a set of complex biological motion soccer-kick stimuli. However, the model was unable to simulate the reaction times of the athletes in a credible manner, and a few subjects could not be simulated. In this current work, we implemented a novel disremembering strategy to incorporate neural adaptation at the decision-making level, which improved the model's ability to simulate the athletes' reaction times. We also introduced receptive fields to detect rotational optic flow patterns not considered in the previous model to simulate a new subject and improve the correlation between the simulation and experimental data. The findings suggest that rotational optic flow plays a critical role in the decision-making process and sheds light on how different individuals perform at different levels. The correlation analysis of human versus simulation data shows a significant, almost perfect correlation between experimental and simulated angular thresholds and slopes, respectively. The analysis also reveals a strong relation between the average reaction times of the athletes and the simulations. [ABSTRACT FROM AUTHOR]

Video anomaly detection by reconstruction is a challenging task. One of its challenges is related to the volume of input data frames needed to be processed to detect anomalies. The challenge usually manifests itself as increased training and especially testing time. The proposed architecture boosts performance while maintaining the same test time as our previously introduced AnoDetNet architecture. The proposed architecture is a cascaded framework that is a succession of reconstruction and an auxiliary network. Upon training, the auxiliary network acts as guidance through the use of combined loss. The combined training of the networks results in a performance increase compared with the reconstruction case alone. Considering that the auxiliary network's results are not used in the test phase, the overall anomaly detection test time does not change compared with the non-cascaded architecture. Two possible auxiliary networks, namely edge detection and optical flow estimation are studied. The proposed architecture results in state-of-the-art results on the Ped2 and Avenue datasets. [ABSTRACT FROM AUTHOR]

A space-time variational method is developed for retrieving upper-level vortex winds from geostationary satellite rapid infrared scans over hurricanes. In this method, new vortex-flow-dependent correlation functions are formulated for the radial and tangential components of the vortex wind. These correlation functions are used to construct the background error covariance matrix and its square root matrix. The resulting square root matrix is then employed to precondition the cost function, constrained by an advection equation formulated for rapidly scanned infrared image movements. This newly formulated and preconditioned cost function is more suitable for deriving upper-level vortex winds from GOES-16 rapid infrared scans over hurricanes than the cost function in the recently adopted optical flow technique. The new method was applied to band-13 (10.3 µm) brightness temperature images scanned every min from GOES-16 over Hurricanes Laura on 27 August 2020 and Hurricanes Ida on 29 August 2021. The retrieved vortex winds were shown to not only be much denser than operationally produced atmospheric motion vectors (AMVs) but also more rotational and better organized around the eyewall than the super-high-resolution AMVs derived from optical-flow technique. By comparing their component velocities (projected along radar beams) with limited radar velocity observations available near the cloud top, the vortex winds retrieved using the new method were also shown to be more accurate than the super-high-resolution AMVs derived from the optical-flow technique. The new method is computationally efficient for real-time applications and potentially useful for hurricane wind nowcasts. Furthermore, the combined use of VF-dependent covariance functions and imagery advection equation is not only novel but was also found to be critically important for the improved performance of the method. This finding implies that similar combined approaches can be developed with improved performance for retrieving vortex flows rapidly scanned using other types of remote sensing on different scales, such as tornadic mesocyclones rapidly scanned by phased-array radars. [ABSTRACT FROM AUTHOR]

The ability to detect and assess world‐relative object‐motion is a critical computation performed by the visual system. This computation, however, is greatly complicated by the observer's movements, which generate a global pattern of motion on the observer's retina. How the visual system implements this computation is poorly understood. Since we are potentially able to detect a moving object if its motion differs in velocity (or direction) from the expected optic flow generated by our own motion, here we manipulated the relative motion velocity between the observer and the object within a stationary scene as a strategy to test how the brain accomplishes object‐motion detection. Specifically, we tested the neural sensitivity of brain regions that are known to respond to egomotion‐compatible visual motion (i.e., egomotion areas: cingulate sulcus visual area, posterior cingulate sulcus area, posterior insular cortex [PIC], V6+, V3A, IPSmot/VIP, and MT+) to a combination of different velocities of visually induced translational self‐ and object‐motion within a virtual scene while participants were instructed to detect object‐motion. To this aim, we combined individual surface‐based brain mapping, task‐evoked activity by functional magnetic resonance imaging, and parametric and representational similarity analyses. We found that all the egomotion regions (except area PIC) responded to all the possible combinations of self‐ and object‐motion and were modulated by the self‐motion velocity. Interestingly, we found that, among all the egomotion areas, only MT+, V6+, and V3A were further modulated by object‐motion velocities, hence reflecting their possible role in discriminating between distinct velocities of self‐ and object‐motion. We suggest that these egomotion regions may be involved in the complex computation required for detecting scene‐relative object‐motion during self‐motion. [ABSTRACT FROM AUTHOR]

A good understanding of geometrical concepts as well as a broad familiarity with objects lead to excellent human perception of moving objects. The human ability to detect and segment moving objects works in the presence of multiple objects, complex background geometry, motion of the observer and even camouflage. How we perceive moving objects so reliably is a longstanding research question in computer vision and borrows findings from related areas such as psychology, cognitive science and physics. One approach to the problem is to teach a deep network to model all of these effects. This is in contrast with the strategy used by human vision, where cognitive processes and body design are tightly coupled and each is responsible for certain aspects of correctly identifying moving objects. Similarly, from the computer vision perspective there is evidence that classical, geometry-based techniques are better suited to the "motion-based" parts of the problem, while deep networks are more suitable for modeling appearance. In this work, we argue that the coupling of camera rotation and camera translation can create complex motion fields that are difficult for a deep network to untangle directly. We present a novel probabilistic model to estimate the camera's rotation given the motion field. We then rectify the flow field to obtain a rotation-compensated motion field for subsequent segmentation. This strategy of first estimating camera motion, and then allowing a network to learn the remaining parts of the problem, yields improved results on the widely used DAVIS benchmark as well as the more recent motion segmentation data set MoCA (Moving Camouflaged Animals). [ABSTRACT FROM AUTHOR]

Industrial activities such as metallurgy, coal and oil combustion, cement production, and petrochemistry release aerosol particles into the atmosphere. We propose analyzing the aerosol composition of plumes emitted by different industrial stacks using PRISMA (PRecursore IperSpettrale della Missione Applicativa) satellite hyperspectral observations. Three industrial sites have been observed: a coal-fired power plant in Matla, South Africa (imaged on 25 September 2021); a steel plant in Wuhan, China (24 March 2021); and gas flaring at an oil extraction site in Hassi Messaoud, Algeria (9 July 2021). Below-plume surface reflectances are constrained using a combination of PRISMA and Sentinel-2/MSI images. Radiative transfer simulations are performed for each scene including the surface, background atmosphere, and plume optical properties. The plume aerosol optical thickness (AOT), particle radius, volume of coarse-mode aerosol, and soot are then retrieved within the plumes following an optimal estimation framework. The mean plume retrieved AOT at 500 nm ranges between 0.27 and 1.27 and the median radius between 0.10 and 0.12 µ m. We found a volume fraction of soot of 3.6 % and 10.4 % in the sinter plant and coal-fired plant plumes, respectively. The mass flow rate of particulate matter at a point source estimated by an integrated mass enhancement method varies from 840 ± 155 g s -1 for the flaring emission to 1348 ± 570 g s -1 at the coal-fired plant. [ABSTRACT FROM AUTHOR]

Infrared small target detection plays a crucial role in maritime security. However, detecting small targets within heavy sea clutter environments remains challenging. Existing methods often fail to deliver satisfactory performance in the presence of substantial clutter interference. This paper analyzes the spatial–temporal appearance characteristics of small targets and sea clutter. Based on this analysis, we propose a novel detection method based on the appearance stable isotropy measure (ASIM). First, the original images are processed using the Top-Hat transformation to obtain the salient regions. Next, a preliminary threshold operation is employed to extract the candidate targets from these salient regions, forming a candidate target array image. Third, to distinguish between small targets and sea clutter, we introduce two characteristics: the gradient histogram equalization measure (GHEM) and the local optical flow consistency measure (LOFCM). GHEM evaluates the isotropy of the candidate targets by examining their gradient histogram equalization, while LOFCM assesses their appearance stability based on local optical flow consistency. To effectively combine the complementary information provided by GHEM and LOFCM, we propose ASIM as a fusion characteristic, which can effectively enhance the real target. Finally, a threshold operation is applied to determine the final targets. Experimental results demonstrate that our proposed method exhibits superior comprehensive performance compared to baseline methods. [ABSTRACT FROM AUTHOR]

SLAM (simultaneous localization and mapping) plays a crucial role in autonomous robot navigation. A challenging aspect of visual SLAM systems is determining the 3D camera orientation of the motion trajectory. In this paper, we introduce an end-to-end network structure, InertialNet, which establishes the correlation between the image sequence and the IMU signals. Our network model is built upon inertial measurement learning and is employed to predict the camera's general motion pose. By incorporating an optical flow substructure, InertialNet is independent of the appearance of training sets and can be adapted to new environments. It maintains stable predictions even in the presence of image blur, changes in illumination, and low-texture scenes. In our experiments, we evaluated InertialNet on the public EuRoC dataset and our dataset, demonstrating its feasibility with faster training convergence and fewer model parameters for inertial measurement prediction. [ABSTRACT FROM AUTHOR]

Foundation models trained on vast quantities of data have demonstrated impressive performance in capturing complex nonlinear relationships and accurately predicting neuronal responses. Due to the fact that deep learning neural networks depend on massive amounts of data samples and high energy consumption, foundation models based on spiking neural networks (SNNs) have the potential to significantly reduce calculation costs by training on neuromorphic hardware. In this paper, a visually inspired computational model composed of an SNN and echo state network (ESN) is proposed for the recognition of optic flow. The visually inspired SNN model serves as a foundation model that is trained using spike-timing-dependent plasticity (STDP) for extracting core features. The ESN model makes readout decisions for recognition tasks using the linear regression method. The results show that STDP can perform similar functions as non-negative matrix decomposition (NMF), i.e., generating sparse and linear superimposed readouts based on basis flow fields. Once the foundation model is fully trained from enough input samples, it can considerably reduce the training samples required for ESN readout learning. Our proposed SNN-based foundation model facilitates efficient and cost-effective task learning and could also be adapted to new stimuli that are not included in the training of the foundation model. Moreover, compared with the NMF algorithm, the foundation model trained using STDP does not need to be retrained during the testing procedure, contributing to a more efficient computational performance. [ABSTRACT FROM AUTHOR]

Pig husbandry constitutes a significant segment within the broader framework of livestock farming, with porcine well-being emerging as a paramount concern due to its direct implications on pig breeding and production. An easily observable proxy for assessing the health of pigs lies in their daily patterns of movement. The daily movement patterns of pigs can be used as an indicator of their health, in which more active pigs are usually healthier than those who are not active, providing farmers with knowledge of identifying pigs' health state before they become sick or their condition becomes life-threatening. However, the conventional means of estimating pig mobility largely rely on manual observations by farmers, which is impractical in the context of contemporary centralized and extensive pig farming operations. In response to these challenges, multi-object tracking and pig behavior methods are adopted to monitor pig health and welfare closely. Regrettably, these existing methods frequently fall short of providing precise and quantified measurements of movement distance, thereby yielding a rudimentary metric for assessing pig health. This paper proposes a novel approach that integrates optical flow and a multi-object tracking algorithm to more accurately gauge pig movement based on both qualitative and quantitative analyses of the shortcomings of solely relying on tracking algorithms. The optical flow records accurate movement between two consecutive frames and the multi-object tracking algorithm offers individual tracks for each pig. By combining optical flow and the tracking algorithm, our approach can accurately estimate each pig's movement. Moreover, the incorporation of optical flow affords the capacity to discern partial movements, such as instances where only the pig's head is in motion while the remainder of its body remains stationary. The experimental results show that the proposed method has superiority over the method of solely using tracking results, i.e., bounding boxes. The reason is that the movement calculated based on bounding boxes is easily affected by the size fluctuation while the optical flow data can avoid these drawbacks and even provide more fine-grained motion information. The virtues inherent in the proposed method culminate in the provision of more accurate and comprehensive information, thus enhancing the efficacy of decision-making and management processes within the realm of pig farming. [ABSTRACT FROM AUTHOR]

Results of studying stabilization of a homogeneous hydrogen–air flame on an optical discharge plasma in a high-velocity flow are reported. The main aspect of experiments is providing stable combustion behind the region of laser beam focusing without any mechanical flame holders. The laser radiation parameters are sufficient for creating a quasi-steady plasma in the flow. It is shown that the optical discharge stabilizes the flame front in a wide range of equivalence ratios for flow velocities up to m/s. The laser radiation parameters within the range of their variation from one experiment to another exert a minor effect on the turbulent flame velocity. Flame stabilization behind the optical discharge region has some specific features. An important parameter is heat release due to hydrogen combustion. A dimensionless criterion is derived: the turbulent flame velocity is a linear function of this criterion. [ABSTRACT FROM AUTHOR]

Accurate building extraction holds paramount importance in various applications such as urbanization rate calculations, urban planning, and resource allocation. In response to the escalating demand for precise low-altitude unmanned aerial vehicle (UAV) building segmentation in intricate scenarios, this study introduces a semi-supervised methodology to alleviate the labor-intensive process of procuring pixel-level annotations. Within the framework of adversarial networks, we employ a dual-channel parallel generator strategy that amalgamates the morphology-driven optical flow estimation channel with an enhanced multilayer sensing Deeplabv3+ module. This approach aims to comprehensively capture both the morphological attributes and textural intricacies of buildings while mitigating the dependency on annotated data. To further enhance the network's capability to discern building features, we introduce an adaptive attention mechanism via a feature fusion module. Additionally, we implement a composite loss function to augment the model's sensitivity to building structures. Across two distinct low-altitude UAV datasets within the domain of UAV-based building segmentation, our proposed method achieves average mean pixel intersection-over-union (mIoU) ratios of 82.69% and 79.37%, respectively, with unlabeled data constituting 70% of the overall dataset. These outcomes signify noteworthy advancements compared with contemporaneous networks, underscoring the robustness of our approach in tackling intricate building segmentation challenges in the domain of UAV-based architectural analysis. [ABSTRACT FROM AUTHOR]