Search

Facial expressions (FEs) spotting aims to split long videos into intervals of neutral expression, macro-expression, or micro-expression. Recent works mainly focus on feature descriptor or optical flow methods, suffering from difficulty capturing subtle facial motion and efficient temporal aggregation. This paper proposes a novel end-to-end network, named FESNet (Facial Expression Spotting Network), to solve the above challenges. The main idea is to model the subtle facial motion as local spatial discrepancy and incorporate temporal correlation by multi-scale temporal convolution. The FESNet comprises a local spatial discrepancy module (LSDM) and a multi-scale temporal aggregation module (MTAM). The LSDM first extracts the static spatial features from each frame by residual convolution and learns the inner spatial correlation by multi-head attention. Moreover, the subtle facial motion of facial expression is modeled as the discrepancy between the first frame and the current frame of the input interval, making frame-wise spatial proposals. Using the local spatial discrepancy features and proposals as input, the MTAM incorporates the temporal correlation by multi-scale temporal convolution and performs cascade refinement to make the final prediction. Furthermore, this paper proposes a smooth loss to ensure the temporal consistency of the cascade refined proposals from MTAM. Comprehensive experiments show that FESNet achieves competitive performance compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The research work is devoted to the quantitative determination of heavy metals in fallen leaves of different zones of their growth. Today, the problem of anthropogenic pollution of cities is acute in the world. Numerous wastes accumulate in plants and can become harmful in their further circulation in the ecosystem. Heavy metals tend to accumulate very well in plants, remaining in them for a long time. The purpose is to determine Cobalt, Iron, and Lead ions in fallen leaves from different urban areas and to know if urban fallen leaves are suitable for ecological composting. The experiment used a spectrophotometric method to determine heavy metals by their colored complex compounds and the transmission of optical flow through a sample. The results show that Iron, Cobalt, and Lead accumulate most in the leaves of roadside trees. Poplar leaves accumulate the most; maple leaves accumulate the least metals. Leaves that are gathered in parks are suitable for ecological composting. However, the roadside municipal leaves are definitely not suitable for composting because of the high concentration of heavy metal ions. [ABSTRACT FROM AUTHOR]

Surveillance video analytics encounters unprecedented challenges in 5G and IoT environments, including complex intra-class variations, short-term and long-term temporal dynamics, and variable video quality. This study introduces Edge-Enhanced TempoFuseNet, a cutting-edge framework that strategically reduces spatial resolution to allow the processing of low-resolution images. A dual upscaling methodology based on bicubic interpolation and an encoder–bank–decoder configuration is used for anomaly classification. The two-stream architecture combines the power of a pre-trained Convolutional Neural Network (CNN) for spatial feature extraction from RGB imagery in the spatial stream, while the temporal stream focuses on learning short-term temporal characteristics, reducing the computational burden of optical flow. To analyze long-term temporal patterns, the extracted features from both streams are combined and routed through a Gated Recurrent Unit (GRU) layer. The proposed framework (TempoFuseNet) outperforms the encoder–bank–decoder model in terms of performance metrics, achieving a multiclass macro average accuracy of 92.28%, an F1-score of 69.29%, and a false positive rate of 4.41%. This study presents a significant advancement in the field of video anomaly recognition and provides a comprehensive solution to the complex challenges posed by real-world surveillance scenarios in the context of 5G and IoT. [ABSTRACT FROM AUTHOR]

Virtual reality (VR) locomotion should allow users to move freely in the virtual space while staying within the tracking area in the real space. The redirected walking (RDW) technique enables users to walk naturally in an unlimited virtual space within a limited tracking area by rotating the virtual scene view. However, conflicting visual and vestibular signals during RDW can lead to user discomfort and decreased immersion. To avoid user discomfort, an RDW gain should be within the detection threshold (DT) range. However, a large angle of walking redirection is required when physically avoiding obstacles or escaping from a narrow space, so DT expansion is necessary. In this study, to change the curvature DT range and enhance RDW performance, we proposed an optical flow (OF)-generating vection in a virtual environment. Further, we investigate methods to reduce user discomfort and increase RDW efficiency considering familiar and unfamiliar VR users. The findings showed that the introduction of OF led to a reduction in the DT range for all users, irrespective of the OF’s direction. However, conditions with OF resulted in an extended DT range for users familiar with VR while concurrently diminishing the DT range for those who were VR unfamiliar. To delve further, our analysis indicated that when both the OF and redirecting directions were identical, the RDW performance was robust to VR familiarity, whereas in opposing directions, the DT range increased for VR-familiar users. Our study findings suggested using OF for the RDW technique and extending its applicability in virtual environments. [ABSTRACT FROM AUTHOR]

An increasing body of evidence suggests that phonological processes exhibit both categorical and gradient effects. This study investigates categorical and gradient effects observed in /w/-deletion in Seoul Korean and how the patterns differ by context. Using ultrasound and a side-view camera, midsagittal images of the tongue and lip protrusion of four participants were recorded. The speakers participated in the task of explaining the semantic differences between 18 words of CV and CwV sequence (e.g. tweji ‘pig’ vs. teji ‘earth’). Optical Flow Analysis was used to quantify the magnitude of lip and tongue movement. Results show that different patterns of categorical and gradient effects of /w/-deletion are observed by context. When /w/ is preceded by a bilabial, /w/ tends to delete categorically, while both categorical deletion and gradient reduction appear to occur when /w/ is preceded by an alveolar or a velar. I argue that the categorical and gradient effects of a related phonological process can be present simultaneously, which can be accounted for by diachronic development of sound patterns. [ABSTRACT FROM AUTHOR]

[Significance] Vibration measurement is a fundamental means to ensure the safety and good condition of structures. Highly accurate vibration measurements provide precise structural dynamic properties for applications such as structural design, installation and commissioning, and health management. [Progress] Compared with traditional vibration measurement methods, video-based vibration measurement is an emerging noncontact and full-field method with high spatial density, high flexibility, easy operation, and cost-effectiveness. Through the video-based vibration measurement method, the vibration displacement information of the measured structure can be obtained from a long distance, which enables the direct observation of small vibration patterns of the structure and has received extensive attention in recent years. Therefore, summarizing the research on video-based vibration measurement technology is of considerable theoretical and applied significance. In the field of video-based vibration measurement technology, the currently commonly used measurement methods mainly include three types, namely, digital image correlation, target tracking, and optical flow methods. The digital image correlation method can obtain accurate three-dimensional vibration information using a dual-camera system. The target tracking method can conduct vibration displacement measurements of large dynamic structures with reliable markers. The optical flow method is a type of target-less video-based vibration measurement method, which can be classified into the intensity and phase optical flow methods according to the different objects to be processed on the surface of the object being measured. The camera arrangement, installation, and operation requirements are low when the object to be measured is small or difficult to access. The emergence of video motion amplification technology further expands the response range of video-based vibration measurement methods, enabling surveyors to more intuitively, rapidly, and dynamically obtain an understanding of the state of small vibrations. [Conclusions and Prospects] In recent years, with the continuous development of machine vision, artificial intelligence, and other technologies, future video-based vibration measurement methods are expected to achieve further breakthroughs in measurement accuracy, computing efficiency, and anti-interference capability. Video-based vibration measurement technology in the field of vibration measurement has unique advantages and practical value, has broad application prospects, and has attracted increasing attention from technicians and researchers. Video-based vibration measurement methods can not only provide new solutions to vibration measurement problems but will also become a powerful complement to traditional vibration measurement methods, particularly in the full-field measurement and dynamic visualization of the vibration. Video-based vibration measurement methods can undertake more important measurement tasks. With the further development of machine vision and optical technology, video-based vibration measurement methods will have broader application prospects as well as higher application value and research significance. [ABSTRACT FROM AUTHOR]

Optical flow is the apparent motion of the brightness patterns in an image. The pyramidal form of the Lucas-Kanade (LK) method is frequently used for its computation but experiments have shown that the method has deficiencies. Problems arise because of numerical issues in the least squares (LS) problem min A x − b 2 2 , A ∈ R m × 2 and m ≫ 2 , which must be solved many times. Numerical properties of the solution x 0 = A † b = (A T A) − 1 A T b of the LS problem are considered and it is shown that the property m ≫ 2 has implications for the error and stability of x 0 . In particular, it can be assumed that b has components that lie in the column space (range) R (A) of A, and the space that is orthogonal to R (A) , from which it follows that the upper bound of the condition number of x 0 is inversely proportional to cos θ , where θ is the angle between b and its component that lies in R (A) . It is shown that the maximum values of this condition number, other condition numbers and the errors in the solutions of the LS problems increase as the pyramid is descended from the top level (coarsest image) to the base (finest image), such that the optical flow computed at the base of the pyramid may be computationally unreliable. The extension of these results to the problem of total least squares is addressed by considering the stability of the optical flow vectors when there are errors in A and b. Examples of the computation of the optical flow demonstrate the theoretical results, and the implications of these results for extended forms of the LK method are discussed. [ABSTRACT FROM AUTHOR]

The realization and modulation of high-order topological states in three-dimensional (3D) photonic crystals (PCs) hold great significance for practical applications in optical communication, optical information processing, and optical computing. However, they have encountered technological challenges due to the difficulty in achieving a complete bandgap in 3D PCs. To address this issue, we introduce a design for a 3D PC based on the Su–Schrieffer–Heeger (SSH) model that exhibits distinctive topological surface states, topological hinge states, and topological corner states. Furthermore, these three types of topological states can be interconverted through the modulation of magnetic field because the topological states are closed and extremely sensitive to the structure, which are modulated by the magnetic field. This intriguing capability holds potential applications in the manipulation of optical flow, optical signal storage, and magnetic field sensors. [ABSTRACT FROM AUTHOR]

Data hiding along with the security plays a vital role in the wireless network. In recent years, object detection, Feature Extraction (FE) and correlation based technique are developed to predict the pixels that is suitable for embedding. However, traditional data hiding method still suffered in the selection of less distortion pixels. To overcome these shortcomings, this method proposes (i) Moving Object (MO) detection, (ii) FE, (iii) data hiding, and (iv) data encryption. Initially, the MO in the video are detected using optical flow and tracking the weight of MO. Next, data hiding features are extracted using Optimization Fruit fly Algorithm (OFA) with Differential Evolution (DE) and Opposition Based Learning (OBL) algorithm. Then, a modified data hiding techniques based on correlation, regression and pixel segmentation with Prediction Error (PE) techniques are used for improving the performance. Finally, the hidden data is encrypted using Binary Tree Structure (BTS) to provide additional layer of security. Experimental result shows the proposed method is superior to other state-of-the-art methods in terms of Peak-Signal-to-Noise Ratio (PSNR), embedding capacity and security analysis. [ABSTRACT FROM AUTHOR]

Deaf people communicate naturally through sign languages and often face barriers to communicating with hearing people and accessing information in written languages. These communication difficulties are aggravated in the health domain, especially in a hospital emergency, when human sign language interpreters are unavailable. This paper proposes a solution for automatically recognizing signs in Brazilian Sign Language (Libras) in the health context to reduce this problem. The idea is that the system could assist in the communication between a Deaf patient and his doctor in the future. Our solution involves a multiple-stream architecture that combines convolutional and recurrent neural networks, dealing with sign languages' visual phonemes individual and specialized ways. The first stream uses the optical flow as input for capturing information about the "movement" of the sign; the second stream extracts kinematic and postural features, including "handshapes" and "facial expressions"; and the third stream process the raw RGB images to address additional attributes about the sign not captured in the previous streams. Thus, we can process more spatiotemporal features that discriminate the classes during the training stage. The computational results show that the solution can recognize signs in Libras in the health context, with an average accuracy, precision, recall, and f1-score of 99.80%, 99.81%, 99.80%, and 99.80%, respectively. Our system also performed better than other works in the literature, obtaining an average accuracy of 100% in an Argentine Sign Language (LSA) dataset, which is usually used for comparison purposes. [ABSTRACT FROM AUTHOR]

Optical identification is often done with spatial or temporal visual pattern recognition and localization. Temporal pattern recognition, depending on the technology, involves a trade-off between communication frequency, range, and accurate tracking. We propose a solution with light-emitting beacons that improves this trade-off by exploiting fast event-based cameras and, for tracking, sparse neuromorphic optical flow computed with spiking neurons. The system is embedded in a simulated drone and evaluated in an asset monitoring use case. It is robust to relative movements and enables simultaneous communication with, and tracking of, multiple moving beacons. Finally, in a hardware lab prototype, we demonstrate for the first time beacon tracking performed simultaneously with state-of-the-art frequency communication in the kHz range. [ABSTRACT FROM AUTHOR]

In this paper, we address the challenge of detecting small moving targets in dynamic environments characterized by the concurrent movement of both platform and sensor. In such cases, simple image-based frame registration and optical flow analysis cannot be used to detect moving targets. To tackle this, it is necessary to use sensor and platform meta-data in addition to image analysis for temporal and spatial anomaly detection. To this end, we investigate techniques that utilize inertial data to enhance frame-to-frame registration, consistently yielding improved detection outcomes when compared against purely feature-based techniques. For cases where image registration is not possible even with metadata, we propose single-frame spatial anomaly detection and then estimate the range to the target using the platform velocity. The behavior of the estimated range over time helps us to discern targets from clutter. Finally, we show that a KNN classifier can be used to further reduce the false alarm rate without a significant reduction in detection performance. The proposed strategies offer a robust solution for the detection of moving targets in dynamically challenging settings. [ABSTRACT FROM AUTHOR]

The spatter generated by the interaction between laser and powder during Powder Bed Fusion-Laser Melting (PBF-LM) can significantly affect the quality of printed parts. A high-speed camera is used to observe the dynamic process of spatter's behavior under different layer thickness and laser powers during the printing process, and to analyze the printed samples' surface roughness, microstructure, and mechanical properties. In terms of spatter image processing, employing an optical flow approach to track and quantify the number of spatters efficiently eliminates statistical redundancy and improves statistical correctness. It is found that under the same laser power, the number of spatters produced by the laser scan direction with the gas flow (LSD-W) is more than that by the laser scan direction against the gas flow (LSD-A), and the number of spatters produced increases with the increase of laser power. Analyzing the mechanical properties and surface roughness of the printed samples under different process parameters quantitatively reveals that differences in the spatter amount generated under different process parameters in the PBF-LM process is not the determining factor affecting the difference in tensile strength of printed parts. During LSD-W, the number of spatters generated at laser power of 170 W and layer thickness of 0.03 mm is 87, and the tensile strength of the printed sample is 618 MPa. During LSD-W, the number of spatters generated at laser power of 320 W and layer thickness of 0.05 mm is 211, and the tensile strength of the printed sample is 680 MPa. Instead, spatter generation has a more direct impact on the surface roughness of printed parts. The layer thickness is 0.03 mm, the laser power is 170 W, and (Ra = 2.372 μm) is the surface roughness of the sample. The layer thickness is 0.05 mm, the laser power is 320 W, and (Ra = 8.163 μm) is the surface roughness of the sample. [ABSTRACT FROM AUTHOR]

Robust motion estimation and segmentation are two important interrelated tasks with a variety of applications. Traditional motion estimation and segmentation methods focus on normal scenarios. However, they may fail to robustly estimate and segment the moving objects because of the ill-defined edges and boundaries in these scenarios. In this paper, a robust framework for motion estimation and segmentation in adverse outdoor conditions is presented. In the proposed framework, a non-local total variation motion estimation method (NLTV- L 1 ) with a support weight function is proposed. It is efficient to preserve the edges and handle the smoothness near the edges using a coarse-to-fine strategy for adverse outdoor scenarios. To deal with the segmentation problem, we take advantage of the motion pattern based background-foreground layer extraction to segment the moving objects. Such a way effectively groups the pixels that move in a similar direction and thus these pixels can be extracted from the noisy background. In addition, our framework does not require any post-processing step to remove noise. We demonstrate that the proposed framework outperforms other state-of-the-art methods on two commonly used datasets (i.e., Middlebury and MPI Sintel) for motion estimation and segmentation. Moreover, the experiments on real outdoor benchmarks (i.e., Foggy Zurich, Traffic and CDnet2014) show the robustness and efficiency of our framework in adverse outdoor conditions. [ABSTRACT FROM AUTHOR]

Optical fiber tweezers offer a simple, low-cost and portable solution for non-invasive trapping and manipulation of particles. However, single-fiber tweezers require fiber tip modification (tapering, lensing, etc.) and the dual-fiber approach demands strict alignment and positioning of fibers for robust trapping of particles. In addition, both tweezing techniques offer a limited range of particle manipulation and operate in low flow velocity regimes (a few 100 μm s−1) when integrated with microfluidic devices. In this paper, we report a novel opto-hydrodynamic fiber tweezers (OHT) platform that exploits the balance between the hydrodynamic drag force and optical scattering forces to trap and manipulate single or multiple particles of various shapes, sizes, and material compositions in a microfluidic channel. 3D hydrodynamic flow focusing offers an easy and dynamic alignment of the particle trajectories with the optical axis of the fiber, which enables robust trapping of particles with high efficiency of >70% and throughput of 14 particles per minute (operating flow velocity: 1000 μm s−1) without the need for precision stages or complex fabrication. By regulating the optical power and flow rates, we were able to trap single particles at desired positions in the channel with a precision of ±10 μm as well as manipulate them over a long range upstream or downstream with a maximum distance of 500 μm. Our opto-hydrodynamic tweezers offer an alternative to conventional optical fiber tweezers for several applications in physics, biology, medicine, etc. [ABSTRACT FROM AUTHOR]

The critical goal of gait recognition is to acquire the inter-frame walking habit representation from the gait sequences. The relations between frames, however, have not received adequate attention in comparison to the intra-frame features. In this paper, motivated by optical flow, the bilateral motion-oriented block is proposed to explore motion description at the feature level. It can allow the classic 2D convolutional structure to have the capability to directly portray gait movement patterns while preventing costly computations on the estimation of optical flow. Based on such features, we develop a set of multi-scale temporal representations that force the motion context to be richly described at various levels of temporal resolution. Furthermore, the dynamic information is sensitive to inaccurate segmentation on the edge, so a correction block is devised to eliminate the segmentation noise of silhouettes for getting more precise gait modality. Subsequently, the temporal feature set and the spatial features are combined to comprehensively characterize gait processes. Extensive experiments are conducted on CASIA-B and OU-MVLP datasets, and the results achieve an outstanding identification performance, which has demonstrated the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Currently, shrimp fry vitality detection mainly relies on manual observation of shrimp fry swimming, but the evaluation results are susceptible to subjective influence. To address this problem, this study proposes a shrimp fry vitality detection method that combines the dense optical flow method and statistics. Firstly, shrimp fry with different vitality were obtained by rearing them in different water temperatures, and shrimp fry videos were collected; then the inter-frame motion features of shrimp fry in the videos were extracted using the Gunner Farneback optical flow method, which represents the instantaneous swimming state of the shrimp fry, and contains two dimensions, namely, magnitude and angle, and statistical analyses were performed at the same time; finally, we extracted the motion features between all the frames of the fixed-length shrimp video and applied the improved information entropy and mutual information to detect the vitality of the shrimp fry. After testing, the precision and recall of the two detection methods exceeded 98%, which can accurately and objectively realize the shrimp fry vitality detection. This study provides ideas and methods for automated shrimp fry quality detection, which has application value in aquaculture industry. [ABSTRACT FROM AUTHOR]

To accurately describe the thermal comfort of indoor occupant used to regulate heating, ventilation and air conditioning (HVAC) system is a key goal to reduce energy consumption in intelligent buildings. In this paper, we propose a noncontact measurement of occupant thermal discomfort behavior as an index of thermal comfort. The method takes the existing monitoring image data in the building as the input to infer the occupant thermal discomfort directly, which saves the cost because there is no need to install new sensors in the building and the occupant does not need to wear additional equipment. The framework combined three channels of body posture, motion and performance information to infer occupant thermal discomfort behavior, it consists of a human detection and crop module, a posture analysis module, an optical flow extraction module and a 3D convolutional neural network module. The three channels describe the actions from different perspectives, and the proposed method makes full use of the complementarity of the three modalities to identify the occupant's thermal discomfort behaviors. Sixteen postures related to thermal discomfort were identified through a questionnaire, and 14,800 video clips containing these postures were collected for experimental evaluation. The results demonstrate the superior performances of our approach to the state-of-the-art techniques. The framework achieves noninvasive, cost-effective thermal comfort evaluation, and has potential value in improving energy efficiency of HVAC systems. [ABSTRACT FROM AUTHOR]

North East Monsoon (NEM) is the major source of rainfall for the south-eastern parts of peninsular India. Short time rainfall prediction data (i.e., nowcasting) are based on the observations from Doppler weather radars which has a high spatial and temporal resolution. This study focuses on the short-term ensemble prediction system using weather radar data to predict precipitation during the NEM and is the first of its kind in the Indian region to make an assessment of the operational performance of the prediction system. Six rainfall events have been studied for the assessment of short-term prediction system where the precipitation systems are different and include a tropical storm observed over different days during the 2022 NEM season. To assess the performance of the system, Fractional Skill Scores (FSS) at a 1 km window have been computed for a lead time of 0–2 h for all the rainfall events with more than 750 samples using different optical flow methods and ensemble sizes. The best average skill score and maximum skill score obtained at a 2 h lead time is 0.65 and 0.78 for tropical storms, 0.5 and 0.78 for stratiform and 0.15 and 0.38 for convective precipitation. It has found that the performance of the model is best for precipitation systems that are widespread and have a longer life period. [ABSTRACT FROM AUTHOR]

The activity recognition gained immense popularity due to increasing number of surveillance cameras. The purpose of activity recognition is to detect the actions from the series of examination by varying the environmental condition. In this paper, Chaotic Whale Atom Search Optimisation (CWASO)-based Deep stacked autoencoder (CWASO-Deep SAE) is proposed for crowd behaviour recognition. The key frames are subjected to the descriptor of feature to extort the features, which bring out the classifier input vector. In this model, the statistical features, optical flow features and visual features are conducted to extract important features. Furthermore, the significant features are shown in the deep stacked auto-encoder (Deep SAE) for activity recognition, as the guidance of deep SAE is performed byCWASO, that is planned is designed by adjoining Atom search optimisation (ASO) algorithm and Chaotic Whale optimisation algorithm (CWOA). The proposed systems' performance is analysed using two datasets. By considering the training data, the projected method attains performance that is high for dataset-1 with maximum precision, sensitivity, and with specific value of 96.826%, 96.790%, and 99.395%, respectively. Similarly, by considering the K-Fold, this method attains the maximum precision of 96.897%, sensitivity of 96.885%, and with specific values of 97.245% for the dataset-1. [ABSTRACT FROM AUTHOR]

The 3D geometry understanding of dynamic scenes captured by moving cameras is one of the cornerstones of 3D scene understanding. Optical flow estimation, visual odometry, and depth estimation are the three most basic tasks in 3D geometry understanding. In this work, we present a unified framework for joint self-supervised learning of optical flow estimation, visual odometry, and depth estimation with two- and three-view geometric constraints. As we all know, visual odometry and depth estimation are more sensitive to dynamic objects, while optical flow estimation is more difficult to estimate the boundary area moved out of the image. To this end, we use estimated optical flow to help visual odometry and depth estimation process dynamic objects and use a rigid flow synthesized by the estimated pose and depth to help learn the optical flow of the area that moves out of the boundary due to camera motion. In order to further improve the consistency of cross-tasks, we introduce three-view geometric constraints and propose a three-view consistency loss. Finally, experiments on the KITTI data set show that our method can effectively improve the performance of the occluded boundary area and the dynamic object area. Moreover, our method achieves comparable or better performance than other monocular self-supervised state-of-the-art methods in these three subtasks. [ABSTRACT FROM AUTHOR]

Accuracy and temporal resolution of flow meters are often unacceptable below the microliter per minute scale, limiting their ability to evaluate the real-time performance of many microfluidic devices. For conventional flow meters, this problem arises from uncertainties that depend on physical effects, such as evaporation, whose relative impacts scale inversely with flow rate. More advanced techniques that can measure nanoliter per minute flows are often not dynamic and require specialized equipment. Herein, we report on new experimental and theoretical results that overcome both limitations using an optofluidic flow meter. Previously, we showed that this device can measure flow rates as low as 1 nl/min with roughly 5% relative uncertainty by leveraging the photobleaching rate of a fluorescent dye. We now extend that work by determining the flow meter's relaxation time over a wide range of flow rates and incident irradiances. Using a simplified analytical model, we deduce that this time constant arises from the interplay between the photobleaching rate and transit time of the dye through the optical interrogation region. This motivates us to consider a more general model of the device, which, surprisingly, implies that all time constants are related by a simple scaling relationship depending only on the flow rate and optical irradiance. We experimentally validate this relationship to within 5% uncertainty down to 1 nl/min. Additionally, we measure a relaxation time of the flow meter on the order of 100 ms for 1 nl/min flows, demonstrating the ability to make dynamic measurements of small flows with unprecedented accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we focus on exploring effective methods for faster and accurate semantic segmentation. A common practice to improve the performance is to attain high-resolution feature maps with strong semantic representation. Two strategies are widely used: atrous convolutions and feature pyramid fusion, while both are either computationally intensive or ineffective. Inspired by the Optical Flow for motion alignment between adjacent video frames, we propose a Flow Alignment Module (FAM) to learn Semantic Flow between feature maps of adjacent levels and broadcast high-level features to high-resolution features effectively and efficiently. Furthermore, integrating our FAM to a standard feature pyramid structure exhibits superior performance over other real-time methods, even on lightweight backbone networks, such as ResNet-18 and DFNet. Then to further speed up the inference procedure, we also present a novel Gated Dual Flow Alignment Module to directly align high-resolution feature maps and low-resolution feature maps where we term the improved version network as SFNet-Lite. Extensive experiments are conducted on several challenging datasets, where results show the effectiveness of both SFNet and SFNet-Lite. In particular, when using Cityscapes test set, the SFNet-Lite series achieve 80.1 mIoU while running at 60 FPS using ResNet-18 backbone and 78.8 mIoU while running at 120 FPS using STDC backbone on RTX-3090. Moreover, we unify four challenging driving datasets (i.e., Cityscapes, Mapillary, IDD, and BDD) into one large dataset, which we named Unified Driving Segmentation (UDS) dataset. It contains diverse domain and style information. We benchmark several representative works on UDS. Both SFNet and SFNet-Lite still achieve the best speed and accuracy trade-off on UDS, which serves as a strong baseline in such a challenging setting. The code and models are publicly available at https://github.com/lxtGH/SFSegNets. [ABSTRACT FROM AUTHOR]

• A novel wavelet convolution magnification network (WCMN) for ME spotting in long videos is proposed. • A novel selective magnification attention module is proposed and constrained by L guide and α-ReLU. • Our method verifies that motion magnification can facilitate ME spotting in long videos, which is neglected by previous methods. Facial Micro-Expressions (MEs) are transient and spontaneous, reflecting a person's authentic internal emotions and have more significant value in many fields. Due to the presence of many background disturbances, including irrelevant motion (such as blinks and head movements) and noise in long videos, it is challenging to spot subtle MEs from these disturbances. To spot subtle MEs, a novel Wavelet Convolution Magnification Network (WCMN) with optical flow feature enhancement for spotting facial micro-expressions in long videos is proposed, which has a U-Net-like architecture and consists of discrete wavelet transform networks and an attention magnification mechanism. It can effectively suppress background disturbances and magnify the optical flow features of MEs, making them easy to detect. Experiments are conducted on two long video datasets (CAS(ME)2 and SAMM Long Videos). The results show that our WCMN outperforms the state-of-the-art ME spotting methods, achieving overall F1-scores of 0.3791 on CAS(ME)2, and 0.3272 on SAMM Long Videos, respectively. We also provide visualization and analysis to explain what the network enhances, and to demonstrate the effectiveness of our WCMN through ablation studies. [ABSTRACT FROM AUTHOR]

Motions in videos are typically a mixture of local dynamic object motions and global camera motion, which are inconsistent in some cases, and even interfere with each other, causing difficulties in various downstream applications, such as video stabilization that requires the global motion, and action recognition that consumes local motions. Therefore, it is crucial to estimate them separately. Existing methods separate two motions from the mixed motion fields, such as optical flow. However, the quality of mixed motion determines the higher bounds of the performance. In this work, we propose a framework, GLOCAL, to directly estimate global and local motions simultaneously from adjacent frames in a self-supervised manner. Our GLOCAL consists of a Global Motion Estimation (GME) module and a Local Motion Estimation (LME) module. The GME module involves a mixed motion estimation backbone, an implicit bottleneck structure for feature dimension reduction, and an explicit bottleneck for global motion recovery based on the global motion bases with foreground mask under the training guidance of proposed global reconstruction loss. An attention U-Net is adopted for LME which produces local motions while excluding motion of irrelevant regions under the guidance of proposed local reconstruction loss. Our method can achieve better performance than the existing methods on the homography estimation dataset DHE and the action recognition dataset NCAA and UCF-101. • A unified framework to estimate global and local motion simultaneously. • An implicit–explicit bottleneck for global motion without irrelevant information. • Two loss functions to learn global and local motion in a self-supervised manner. • Achieves SOTA in regular homography estimation scenes and action recognition task. [Display omitted] [ABSTRACT FROM AUTHOR]

In this research, we explore the latest advancements in the domains of vehicle detection, recognition, and tracking through vision-based methodologies. Spanning from traditional techniques to cutting-edge deep learning models, the paper provides a thorough analysis of their strengths, limitations, and real-world applications. We examine the evolution of detection methods, including edge-based approaches and motion segmentation techniques like frame differencing, background subtraction, and optical flow, showcasing their efficacy in identifying moving vehicles. Convolutional neural networks (CNNs), in particular, are highlighted for the integration of appearance-based approaches due to their transformative impact on accuracy, particularly in difficult settings. In addition, we explore vehicle recognition and classification methods, highlighting their applications in traffic analysis, security, and urban planning. These methods include color recognition, license plate recognition, logo recognition, and vehicle type classification. In addition to comparing tracking algorithms based on accuracy, computing economy, and adaptability, the evaluation includes methodologies for tracking vehicles that include model-based, shape-based, and feature-based approaches. In conclusion, this research provides a thorough examination of the development of vision-based vehicle analysis, making it a useful tool for academics, professionals, and decision-makers working to improve traffic surveillance systems and plan for the future of urban transportation. [ABSTRACT FROM AUTHOR]

Although supervised approaches for traffic sign classification have demonstrated excellent performance, they are limited to classifying several traffic signs defined in the training dataset. This prevents them from being applied to different domains, i.e., different countries. Herein, we propose a self-supervised approach for few-shot learning-based traffic sign classification. A center-awareness similarity network is designed for the traffic sign problem and trained using an optical flow dataset. Unlike existing supervised traffic sign classification methods, the proposed method does not depend on traffic sign categories defined by the training dataset. It applies to any traffic signs from different countries. We construct a Korean traffic sign classification (KTSC) dataset, including 6000 traffic sign samples and 59 categories. We evaluate the proposed method with baseline methods using the KTSC, German traffic sign, and Belgian traffic sign classification datasets. Experimental results show that the proposed method extends the ability of existing supervised methods and can classify any traffic sign, regardless of region/country dependence. Furthermore, the proposed approach significantly outperforms baseline methods for patch similarity. This approach provides a flexible and robust solution for classifying traffic signs, allowing for accurate categorization of every traffic sign, regardless of regional or national differences. [ABSTRACT FROM AUTHOR]

With the rapid development in technology in recent years, the use of cameras and the production of video and image data have similarly increased. Therefore, there is a great need to develop and improve video surveillance techniques to their maximum extent, particularly in terms of their speed, performance, and resource utilization. It is challenging to accurately detect anomalies and increase the performance by minimizing false positives, especially in crowded and dynamic areas. Therefore, this study proposes a hybrid video anomaly detection model combining multiple machine learning algorithms with pixel-based video anomaly detection (PBVAD) and frame-based video anomaly detection (FBVAD) models. In the PBVAD model, the motion influence map (MIM) algorithm based on spatio–temporal (ST) factors is used, while in the FBVAD model, the k-nearest neighbors (kNN) and support vector machine (SVM) machine learning algorithms are used in a hybrid manner. An important result of our study is the high-performance anomaly detection achieved using the proposed hybrid algorithms on the UCF-Crime data set, which contains 128 h of original real-world video data and has not been extensively studied before. The AUC performance metrics obtained using our FBVAD-kNN algorithm in experiments were averaged to 98.0%. Meanwhile, the success rates obtained using our PBVAD-MIM algorithm in the experiments were averaged to 80.7%. Our study contributes significantly to the prevention of possible harm by detecting anomalies in video data in a near real-time manner. [ABSTRACT FROM AUTHOR]

The Maritime Continent (MC) regularly experiences powerful convective storms that produce intense rainfall, flooding and landslides, which numerical weather prediction models struggle to forecast. Nowcasting uses observations to make more accurate predictions of convective activity over short timescales (∼ 0–6 h). Optical flow algorithms are effective nowcasting methods as they are able to accurately track clouds across observed image series and predict forward trajectories. Optical flow is generally applied to weather radar observations; however, the radar coverage network over the MC is not complete and the signal cannot penetrate the high mountainous regions. In this research, we apply optical flow algorithms from the pySTEPS nowcasting library to satellite imagery to generate both deterministic and probabilistic nowcasts over the MC. The deterministic algorithm shows skill up to 4 h on spatial scales of 10 km and coarser and outperforms a persistence nowcast for all lead times. Lowest skill is observed over the mountainous regions during the early afternoon, and highest skill is seen during the night over the sea. A key feature of the probabilistic algorithm is its attempt to reduce uncertainty in the lifetime of small-scale convection. Composite analysis of 3 h lead time nowcasts, initialised in the morning and afternoon, produces reliable ensembles but with an under-dispersive distribution and produces area under the curve scores (i.e. ratio of hit rate to false alarm rate across all probability thresholds) of 0.80 and 0.71 over the sea and land, respectively. When directly comparing the two approaches, the probabilistic nowcast shows greater skill at ≤ 60 km spatial scales, whereas the deterministic nowcast shows greater skill at larger spatial scales ∼ 200 km. Overall, the results show promise for the use of pySTEPS and satellite retrievals as an operational nowcasting tool over the MC. [ABSTRACT FROM AUTHOR]

Copyright of Transactions of Nanjing University of Aeronautics & Astronautics is the property of Editorial Department of Journal of Nanjing University of Aeronautics & Astronautics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Although the excitation-time imaging condition offers a lower memory consumption and higher computational efficiency compared to cross-correlation imaging condition, it has not been widely used in industrial applications because of the accuracy problem of traveltime calculation and the influence of low-wave-number noise. In this paper, we introduce the physics-informed neural network (PINN) algorithm to achieve a high-precision traveltime calculation of the source forward wavefield. Subsequently, we introduce a technique for high-precision wavefield decomposition of the reverse-time wavefield via the optical flow vector, enabling us to realize a correlation-weighted stacking imaging of each wavefield. Model experiments and real data processing show that the proposed traveltime calculation algorithm based on PINN offers high accuracy and good applicability in the excitation-time reverse-time migration imaging of complex models, and correlation-weighted stacking imaging based on optical flow vector-based wavefield separation can significantly suppress the noise with low wave-number and achieve high-precision imaging of complex models. [ABSTRACT FROM AUTHOR]

Despite its wide applications in criminal investigations and clinical communications with patients suffering from autism, automatic micro-expression recognition remains a challenging problem because of the lack of training data and imbalanced classes problems. In this study, we proposed a meta-learning-based multimodel fusion network (Meta-MMFNet) to solve the existing problems. The proposed method is based on the metric-based meta-learning pipeline, which is specifically designed for few-shot learning and is suitable for model-level fusion. The frame difference and optical flow features were fused, deep features were extracted from the fused feature, and finally in themeta-learning-based framework, weighted summodel fusion method was applied for micro-expression classification. Meta-MMFNet achieved better results than state-of-the-art methods on four datasets. [ABSTRACT FROM AUTHOR]

This study presents a new vision-based deep learning method to monitor and evaluate the structural health of in-service infrastructure. For this purpose, three different camera placements, including remote, structure-mounted, and drone-mounted cameras, are proposed to capture the vibrations or displacements of bridges. The vision-based deep learning method is verified by an optical flow approach. Various techniques, such as visual data denoising and camera motion removal, are utilized to process the test data for displacement measurements and extract the structural frequencies. Structural models of bridges are analyzed to validate the measurements and assess the structural health of several pedestrian, traffic, and railway bridges without interfering with traffic. Measurements in the field experiments and results from the structural analysis on tested bridges show that the proposed framework works successfully and can be potentially engineered to monitor the structural health of existing bridges. [ABSTRACT FROM AUTHOR]

Falls are the main cause of increased morbidity and mortality in the elderly. Human activity recognition is a difficult task due to factors such as light changes and complex background. To address these challenges, this paper proposes a method for recognizing falling activity based on the human skeleton and optical flow sequence. First, videos of fall activities and not-fall activities of different testers are captured using the Kinect v2 sensor; then, the captured videos are pre-processed to obtain the skeleton images and grayscale images of each activity; and the optical flow sequences of adjacent images are obtained using the dense optical flow algorithm. This paper proposes a lightweight convolutional neural network for extracting movement features from multimodal data and identifying activities. The decision fusion algorithm combines the fuzzy comprehensive evaluation method and the majority voting strategy to determine the weight values that influence the final decision outcomes. The experimental results show that the accuracy of the method proposed in this paper is up to 95.31%, which has a good effect on the recognition of falling activity. [ABSTRACT FROM AUTHOR]

In this article, we address the problem of estimating fluid flows between two adjacent images containing fluid and non-fluid objects. Typically, traditional optical flow estimation methods lack accuracy, because of the highly deformable nature of fluid, the lack of definitive features, and the motion differences between fluid and non-fluid objects. Our approach captures fluid motions using an affine motion model for each small patch of an image. To obtain robust patch matches, we propose a best-buddies similarity-based method to address the lack of definitive features but many similar features in fluid phenomena. A dense set of affine motion models was then obtained by performing nearest-neighbor interpolation. Finally, dense fluid flow was recovered by applying the affine transformation to each patch and was improved by minimizing a variational energy function. Our method was validated using different types of fluid images. Experimental results show that the proposed method achieves the best performance. [ABSTRACT FROM AUTHOR]

The hotspot emission of accreting millisecond pulsars (AMPs) undergoes scattering in the accretion flow between the disk inner radius and the neutron star surface. The scattering optical depth of the flow depends on the photon emission angle, which is a function of the pulse phase, and reaches its maximum when the hotspot is closest to the observer. At sufficiently large optical depths the observed pulse profile should develop a secondary minimum, the depth of which depends on the accretion rate and the emission geometry. Such a dip evolving with the accretion rate might explain the phase shift and pulse profile evolution observed in AMPs during outbursts. Accounting for scattering is important for accurate modeling of the AMP pulse profiles in order to improve the accuracy of determination of the neutron star parameters, such as their masses and radii. In this paper we present a simplified analytical model for the Thomson optical depth of the accretion funnel, and apply it to simulating the pulse profiles. We show that scattering in the accretion funnel has a significant effect on the pulse profiles at accretion rates of M ≳ 10−10M⊙yr−1. Our model predicts a gradual evolution of the pulse profile with the accretion rate that appears to be consistent with the observations. [ABSTRACT FROM AUTHOR]

We propose a deep multi-stream model for left ventricular ejection fraction (LVEF) prediction in 2D echocardiographic (2DE) examinations. We use four standard 2DE views as model input, which are automatically selected from the full 2DE examination. The LVEF prediction model processes eight streams of data (images + optical flow) and consists of convolutional neural networks terminated with transformer layers. The model is made robust to missing, misclassified and duplicate views via pre-training, sampling strategies and parameter sharing. The model is trained and evaluated on an existing clinical dataset (12,648 unique examinations) with varying properties in terms of quality, examining physician, and ultrasound system. We report R 2 = 0.84 and mean absolute error = 4.0% points for the test set. When evaluated on two public benchmarks, the model performs on par or better than all previous attempts on fully automatic LVEF prediction. Code and trained models are available on a public project repository. [ABSTRACT FROM AUTHOR]

The evolution of flight is a rare event in vertebrate history, and one that demands functional integration across multiple anatomical/physiological systems. The neuroanatomical basis for such integration and the role that brain evolution assumes in behavioural transformations remain poorly understood. We make progress by (i) generating a positron emission tomography (PET)-based map of brain activity for pigeons during rest and flight, (ii) using these maps in a functional analysis of the brain during flight, and (iii) interpreting these data within a macroevolutionary context shaped by non-avian dinosaurs. Although neural activity is generally conserved from rest to flight, we found significant increases in the cerebellum as a whole and optic flow pathways. Conserved activity suggests processing of self-movement and image stabilization are critical when a bird takes to the air, while increased visual and cerebellar activity reflects the importance of integrating multimodal sensory information for flight-related movements. A derived cerebellar capability likely arose at the base of maniraptoran dinosaurs, where volumetric expansion and possible folding directly preceded paravian flight. These data represent an important step toward establishing how the brain of modern birds supports their unique behavioural repertoire and provide novel insights into the neurobiology of the bird-like dinosaurs that first achieved powered flight. [ABSTRACT FROM AUTHOR]

Recently, video-based micro-expression recognition (MER) applications have attracted attention in various scenarios. However, current deep learning-based MER methods frequently struggle with several challenges, such as insufficient data, difficulty in capturing subtle facial motions, and keyframe recognition. In this paper, we propose a robust MER solution without prior annotation of keyframes. To prevent traditional data augmentation techniques from destroying the slight motion information in the sequence frames, stride sampling is designed to increase the number of samples while preserving the important motion features of the micro-expression (ME). Moreover, to capture facial rapid and subtle changes to enhance the accuracy of ME classification, we construct a local spatial difference stack (LSDS) block and incorporate it into the lightweight spatiotemporal network VGGFace-TCN. Experiments demonstrate that our proposed algorithm can effectively detect the local facial movement details of MEs from original frames without additional visual features, e.g., optical flow, and minimize the risk of overfitting. Compared with other state-of-the-art methods, the proposed method obtained the best performance under the holdout database evaluation (HDE) strategy with an accuracy and F1-score of 57.46% and 0.3734, respectively. Furthermore, it attained an accuracy of 61.27% and an F1-score of 0.5343 on the Spontaneous Actions and Micro-movements (SAMM) dataset, which is significantly higher than other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In omnidirectional images or videos, the viewer receives an interactive and immersive experience from the viewport by changing the viewing angle. Due to the wide application of omnidirectional videos, the visual quality assessment for omnidirectional videos is becoming an urgent issue. Due to the large resolution of an omnidirectional video, regions with object motions usually catch the viewers' attention, so the motion regions have great influences on the visual quality perception. Since the number of potential viewports is huge and the viewer spends varying amounts of time for different viewports, viewport selection is a critical yet not resolved problem for omnidirectional video quality assessment (VQA). In this paper, we propose a two-stream network with viewport selection for blind omnidirectional VQA to incorporate the influences of motion regions and viewport selection. Firstly, we propose a two-stream multi-task convolutional neural network (TSMT) for VQA at any viewport, which uses video frame sequences and motion sequences as inputs. The motion sequences are represented as horizontal and vertical optical flows. Based on the observation that the low latitude regions, the front view, and the moving objects have higher possibilities that appearing in the viewport, we propose a viewport selection method based on a fusion-based saliency map that considers those regions. Experimental results on two datasets demonstrated that the proposed model outperforms state-of-the-art omnidirectional VQA methods. [ABSTRACT FROM AUTHOR]

The purpose of video inpainting is to get a reasonable content from the video to fill in the missing region. Video is a continuous four-dimensional sequence in the temporal dimension. It's difficult to ensure the temporal continuity of video by inpaint video frames respectively along the time dimension. Video inpainting has gone from the traditional inpainting algorithm to the advanced learning based inpainting method. It has been able to inpainting for a variety of scenes. However, there are still unresolved questions in video inpainting, and video inpainting is still a challenging task. Existing works focused on fixing the problem of object removal in the video, and neglected the importance of inpainting the occlusion scene in the middle region. For the occlusion problem in the middle region, we propose a local and nonlocal optical flow video inpainting framework. First, according to the forward and backward directions of the reference frame and the sampling window, we divide the video into local and nonlocal frames, extract the local and nonlocal optical flow and feed them to the residual network for rough inpainting. Next, our approach extracts and completes the edges of the predicted flow. Finally, the composed optical flow field guides the propagation of pixels to inpaint the video content. Experimental results on DAVIS and YouTube-VOS datasets show that our method has significantly improved in terms of the image quality and optical flow quality compared with the state of the art. Codes are available at {https://github.com/lengfengio/LNFVI.git.} [ABSTRACT FROM AUTHOR]

Automated assessment of broiler chicken welfare poses particular problems due to the large numbers of birds involved and the variety of different welfare measures that have been proposed. Active (sustained, defect-free) walking is both a universally agreed measure of bird health and a behavior that can be recognized by existing technology. This makes active walking an ideal starting point for automated assessment of chicken welfare at both individual and flock level. [ABSTRACT FROM AUTHOR]

Detecting violent behavior in videos to ensure public safety and security poses a significant challenge. Precisely identifying and categorizing instances of violence in real-life closed-circuit television, which vary across specifications and locations, requires comprehensive understanding and processing of the sequential information embedded in these videos. This study aims to introduce a model that adeptly grasps the spatiotemporal context of videos within diverse settings and specifications of violent scenarios. We propose a method to accurately capture spatiotemporal features linked to violent behaviors using optical flow and RGB data. The approach leverages a Conv3D-based ResNet-3D model as the foundational network, capable of handling high-dimensional video data. The efficiency and accuracy of violence detection are enhanced by integrating an attention mechanism, which assigns greater weight to the most crucial frames within the RGB and optical-flow sequences during instances of violence. Our model was evaluated on the UBI-Fight, Hockey, Crowd, and Movie-Fights datasets; the proposed method outperformed existing state-of-the-art techniques, achieving area under the curve scores of 95.4, 98.1, 94.5, and 100.0 on the respective datasets. Moreover, this research not only has the potential to be applied in real-time surveillance systems but also promises to contribute to a broader spectrum of research in video analysis and understanding. [ABSTRACT FROM AUTHOR]

Satellite time-series data contain information in three dimensions—spatial, spectral, and temporal—and are widely used for monitoring, simulating, and evaluating Earth activities. However, some time-phase images in the satellite time series data are missing due to satellite sensor malfunction or adverse atmospheric conditions, which prevents the effective use of the data. Therefore, we need to complement the satellite time series data with sequence image interpolation. Linear interpolation methods and deep learning methods that have been applied to sequence image interpolation lead to large errors between the interpolation results and the real images due to the lack of accurate estimation of pixel positions and the capture of changes in objects. Inspired by video frame interpolation, we combine optical flow estimation and deep learning and propose a method named Multi-Scale Optical Flow-Intermediate Feature Joint Network. This method learns pixel occlusion and detailed compensation information for each channel and jointly refines optical flow and intermediate features at different scales through an end-to-end network together. In addition, we set a spectral loss function to optimize the network's learning of the spectral features of satellite images. We have created a time-series dataset using Landsat-8 satellite data and Sentinel-2 satellite data and then conducted experiments on this dataset. Through visual and quantitative evaluation of the experimental results, we discovered that the interpolation results of our method retain better spectral and spatial consistency with the real images, and that the results of our method on the test dataset have a 7.54% lower Root Mean Square Error than other approaches. [ABSTRACT FROM AUTHOR]

This work presents a novel RGB-D dynamic Simultaneous Localisation and Mapping (SLAM) method that improves the precision, stability, and efficiency of localisation while relying on lightweight deep learning in a dynamic environment compared to the traditional static feature-based visual SLAM algorithm. Based on ORB-SLAM3, the GCNv2-tiny network instead of the ORB method, improves the reliability of feature extraction and matching and the accuracy of position estimation; then, the semantic segmentation thread employs the lightweight YOLOv5s object detection algorithm based on the GSConv network combined with a depth image to determine potentially dynamic regions of the image. Finally, to guarantee that the static feature points are used for position estimation, dynamic probability is employed to determine the true dynamic feature points based on the optical flow, semantic labels, and the state in last frame. We have performed experiments on the TUM datasets to verify the feasibility of the algorithm. Compared with the classical dynamic visual SLAM algorithm, the experimental results demonstrate that the absolute trajectory error is greatly reduced in dynamic environments, and that the computing efficiency is improved by 31.54% compared with the real-time dynamic visual SLAM algorithm with close accuracy, demonstrating the superiority of DLD-SLAM in accuracy, stability, and efficiency. [ABSTRACT FROM AUTHOR]

Recently, video super-resolution has achieved an outstanding performance. However, many existing methods to solve video super-resolution usually make use of complex strategies, such as explicit optical flow, deformable convolution, which increase complexity and computation. In this paper, we propose a lightweight network for video super-resolution, namely Efficient Lightweight Network for Video Super-Resolution (ELNVSR). We design a Multi-group Block extracting long-distance spatial information to construct a lightweight Bidirection Alignment Module which is implicitly capable of fusing and propagating spatial-temporal information in a bidirectional way. Meanwhile, a Multi-scale Pyramid Block is built as a lightweight reconstruction module to extract different levels of information layer by layer. Comprehensive experiments are conducted on public benchmarks. The results demonstrate a promising performance with fewer parameters. [ABSTRACT FROM AUTHOR]

Classifying sports videos is complex due to their dynamic nature. Traditional methods, like optical flow and the Histogram of Oriented Gradient (HOG), are limited by their need for expertise and lack of universality. Deep learning, particularly Convolutional Neural Networks (CNNs), offers more effective feature recognition in sports videos, but standard CNNs struggle with fast-paced or low-resolution sports videos. Our novel neural network model addresses these challenges. It begins by selecting important frames from sports footage and applying a fuzzy noise reduction algorithm to enhance video quality. The model then uses a bifurcated neural network to extract detailed features, leading to a densely connected neural network with a specific activation function for categorizing videos. We tested our model on a High-Definition Sports Video Dataset covering over 20 sports and a low-resolution dataset. Our model outperformed established classifiers like DenseNet, VggNet, Inception v3, and ResNet-50. It achieved high precision (0.9718), accuracy (0.9804), F-score (0.9761), and recall (0.9723) on the high-resolution dataset, and significantly better precision (0.8725) on the low-resolution dataset. Correspondingly, the highest values on the matrix of four traditional models are: precision (0.9690), accuracy (0.9781), F-score (0.9670), recall (0.9681) on the high-resolution dataset, and precision (0.8627) on the low-resolution dataset. This demonstrates our model's superior performance in sports video classification under various conditions, including rapid motion and low resolution. It marks a significant step forward in sports data analytics and content categorization. [ABSTRACT FROM AUTHOR]

We investigated the assessment of blood flow within polypoidal lesions using optical coherence tomography angiography (OCTA) to determine intravitreal brolucizumab (IVBr) efficacy for treating polypoidal choroidal vasculopathy (PCV). We retrospectively studied 46 eyes with PCV that completed 1-year IVBr treatment. Blood flow signals within polypoidal lesions were evaluated using OCTA after loading-phase treatment, and 1-year outcomes were compared between eyes in which blood flow signals disappeared versus persisting. After loading-phase treatment, blood flow signals within polypoidal lesions disappeared in 31 eyes and persisted in 15. In the former group, visual acuity improved significantly throughout the year (P < 0.01), while in the latter there was no significant difference between baseline and after 1 year. The total number of injections was significantly lower with than without disappearance of blood flow signals (6.0 vs. 6.9, P < 0.01). The intended injection interval at the last visit was significantly longer in the former than in the latter group (15.7 weeks vs. 12.5 weeks, P < 0.01). These results indicate that PCV cases showing disappearance of blood flow signals within polypoidal lesions by OCTA after loading-phase treatment had favorable 1-year outcomes of IVBr. Therefore, evaluating blood flow within polypoidal lesions by OCTA may allow noninvasive prediction of PCV treatment outcomes. [ABSTRACT FROM AUTHOR]

The ability to predict the future based on past experience lies at the core of the brain's ability to adapt behavior. However, the neural mechanisms that participate in generating and updating predictions are not clearly understood. Further, the evolutionary antecedents and the prevalence of predictive processing among vertebrates are even less explored. Here, we show evidence of predictive processing via the involvement of cerebellar circuits in larval zebrafish. We presented stereotyped optic flow stimuli to larval zebrafish to evoke swims and discovered that lesioning the cerebellum abolished prediction-dependent modulation of swim latency. When expectations of optic flow direction did not match with reality, error signals arrive at Purkinje cells via the olivary climbing fibers, whereas granule cells and Purkinje cells encode signals of expectation. Strong neural representations of expectation correlate with faster swim responses and vice versa. In sum, our results show evidence for predictive processing in nonmammalian vertebrates with the involvement of cerebellum, an evolutionarily conserved brain structure. [ABSTRACT FROM AUTHOR]

An efficient model to detect and track the objects in adverse weather is proposed using Tanh Softmax (TSM) EfficientDet and Jaccard Similarity based Kuhn-Munkres (JS-KM) with Pearson-Retinex in this paper. The noises were initially removed using Differential Log Energy Entropy adapted Wiener Filter (DLE-WF). The Log Energy Entropy value was calculated between the pixels instead of calculating the local mean of a pixel in the normal Wiener filter. Also, the segmentation technique was carried out using Fringe Binarization adapted K-Means Algorithm (FBKMA). The movement of segmented objects was detected using the optical flow technique, in which the optical flow was computed using the Horn-Schunck algorithm. After motion estimation, the final step in the proposed system is object tracking. The motion-estimated objects were treated as the target that is initially in the first frame. The target was tracked by JS-KM algorithm in the subsequent frame. At last, the experiential evaluation is conducted to confirm the proposed model's efficacy. The outcomes of Detection in Adverse Weather Nature (DAWN) dataset proved that in comparison to the prevailing models, a better performance was achieved by the proposed methodology. [ABSTRACT FROM AUTHOR]