Search

In this research paper, we introduce an adaptive block-matching motion estimation algorithm to improve the accuracy and efficiency of motion estimation (ME). First, we present a block generation system that creates blocks of varying sizes based on the detected motion location. Second, we incorporate predictive tools such as early termination and variable window size to optimize our block-matching algorithm. Furthermore, we propose two distinct search patterns to achieve maximum quality and efficiency. We evaluated the proposed algorithms on 20 videos and compared the results with known algorithms, including the full search algorithm (FSA), which is a benchmark for ME accuracy. Our proposed quality-based algorithm shows an improvement of 0.27 dB in peak signal-to-noise ratio (PSNR) on average for reconstructed frames compared to FSA, along with a reduction of 71.66% in searched blocks. Similarly, our proposed efficiency-based method results in a 0.07 dB increase in PSNR and a 97.93% reduction in searched blocks compared to FSA. These findings suggest that our proposed method has the potential to improve the performance of ME in video coding. [ABSTRACT FROM AUTHOR]

Event cameras, characterized by their low power consumption, expansive dynamic range, and high temporal resolution, have attracted great attentions in various computer vision tasks. Compared to frame-based cameras, event cameras exemplify a marked paradigmatic transition in data formation and output. However, the quality of event streams is compromised by background activity and hot pixels, leading to increased computational overheads and sub-optimal outcomes in subsequent applications, notably in recognition, video reconstruction, and target detection tasks. In this paper, a two-step denoising algorithm (referred as GMCM) is proposed to counteract these challenges. The GMCM algorithm comprises two steps: Gaussian denoising preprocessing and motion denoising. The former incorporates Gaussian temporal distribution and adaptive thresholding mechanisms to discern the inclusion of motion-related information within the event streams. Experimental results demonstrate that Gaussian denoising preprocessing can not only adeptly discern whether the event data stream contains motion information but also enhance computational efficiency. Conclusively, the GMCM algorithm achieves state-of-the-art performance, yielding SNR scores of 37.22 and 26.79 on the DVSCLEAN dataset at the noise ratios of 50% and 100%, respectively. [ABSTRACT FROM AUTHOR]

A number of hardware‐based and software‐based strategies have been suggested to eliminate motion artifacts for improvement of 3D‐optical coherence tomography (OCT) image quality. However, the hardware‐based strategies have to employ additional hardware to record motion compensation information. Many software‐based strategies have to need additional scanning for motion correction at the expense of longer acquisition time. To address this issue, we propose a motion artifacts correction and motion estimation method for OCT volumetric imaging of anterior segment, without requirements of additional hardware and redundant scanning. The motion correction effect with subpixel accuracy for in vivo 3D‐OCT has been demonstrated in experiments. Moreover, the physiological information of imaging object, including respiratory curve and respiratory rate, has been experimentally extracted using the proposed method. The proposed method offers a powerful tool for scientific research and clinical diagnosis in ophthalmology and may be further extended for other biomedical volumetric imaging applications. [ABSTRACT FROM AUTHOR]

Aiming at the problem of high altitude turbulence environment to the time-delay stable image acquisition of UAV, an anti-shaking algorithm for aerial video was proposed for hovering shooting and moving shooting. Firstly from the time-delay photography video captured the UAV camera, some video frames were extracted globally to compare their histogram distributions. This comparison could identify whether the video contained active camera motion or not, and help categorize the video accordingly. For videos with active camera motion, FAST corner detection and optical flow methods were used to extract and match feature points. The RANSAC algorithm could remove all mismatched feature points, and estimate the camera's motion trajectory. The resulting motion estimation parameters were then smoothed using Gaussian filtering, producing a stable camera motion trajectory. For videos without active camera motion, the first frame was divided into grids and feature points were extracted based on Harris matrix. Optical flow tracking was carried out on these feature points in subsequent frames. Reverse optical flow and Harris matrix calculation were used to extract and match feature points, to increase the constraint of feature points. Finally, the retained feature points were used to estimate the stable transformation from subsequent frames to the first frame. Experimental results showed that the video classification module could correctly distinguish between the two types of videos. The algorithm was used to classify the video scene and stabilize the picture. Compared to other methods, this algorithm could improve the average peak signal-to-noise ratio of stabilized video images the most. For videos without active camera motion, the image could be absolutely stable, and the average peak signal-to-noise ratio of the image was increased by more than 39%, while the other two methods only by 10% to 12%. [ABSTRACT FROM AUTHOR]

Background Subtraction (BgS) is a widely researched technique to develop online Change Detection algorithms for static video cameras. Many BgS methods have employed the unsupervised, adaptive approach of Gaussian Mixture Model (GMM) to produce decent backgrounds, but they lack proper consideration of scene semantics to produce better foregrounds. On the other hand, with considerable computational expenses, BgS with Deep Neural Networks (DNN) is able to produce accurate background and foreground segments. In our research, we blend both approaches for the best. First, we formulated a network called Convolutional Density Approximation (CDA) for direct density estimation of background models. Then, we propose a self-supervised training strategy for CDA to adaptively capture high-frequency color distributions for the corresponding backgrounds. Finally, we show that background models can indeed assist foreground extraction by an efficient Neural Motion Subtraction (NeMos) network. Our experiments verify competitive results in the balance between effectiveness and efficiency.

High-Efficiency Video Coding (HEVC) is a video compression standard designed to improve video compression efficiency compared to its predecessor, Advanced Video Coding (AVC). HEVC provides approximately twice the compression efficiency compared to the AVC. In the HEVC system, the most computational complexity module is the Motion Estimation (ME). ME module helps reduce redundancy by compensating for motion during video compression. However, the computational complexity makes it a bottleneck in the design of high-resolution video encoders. This paper proposes an efficient architecture for the Integer ME (IME) module of the HEVC encoder. The proposed architecture introduces an efficient memory usage scheme to support the Full Search motion search algorithm. The full pipeline architecture includes 4096 Processing Elements (PE) and an adder tree to compute the Sum of Absolute Difference (SAD) for every Prediction Unit (PU) partition. The proposed architecture was designed and implemented on Xilinx Virtex-7 XC7VX550T FPGA. Our design achieved up to 275 FPS and approximately 10 FPS at 4 K video for the Search Regions of [Formula: see text] and [Formula: see text] pixels, respectively.

Medical image analysis presents a significant problem in the field of image registration. Recently, medical image registration has been recognized as a helpful tool for medical professionals. Current state-of-the-art approaches solely focus on source image registration and lack quantitative measurement for object deviation in terms of loosening, subsidence and anteversion related to surgery. In this article, we have provided motion vectors for recognizing the object deviation, in addition to detecting and selecting the feature points. Firstly, the feature points will be detected using Hessian matrix determinants and octave and level sampling. Then the strongest feature points are selected which will be utilized for identifying the object deviation with respect to the reference image through motion vectors. The objective of this work is to combine image registration and temporal differencing to achieve independent motion detection. In comparison to state-of-the-art approaches, the proposed methodology achieves higher Information Ratio (IR), Mutual Information Ratio (MIR) and their lower bounds for image registration.

Abstract Visual object tracking, crucial in aerial applications such as surveillance, cinematography, and chasing, faces challenges despite AI advancements. Current solutions lack full reliability, leading to common tracking failures in the presence of fast motions or long‐term occlusions of the subject. To tackle this issue, a 3D motion model is proposed that employs camera/vehicle states to locate a subject in the inertial coordinates. Next, a probability distribution is generated over future trajectories and they are sampled using a Monte Carlo technique to provide search regions that are fed into an online appearance learning process. This 3D motion model incorporates machine‐learning approaches for direct range estimation from monocular images. The model adapts computationally by adjusting search areas based on tracking confidence. It is integrated into DiMP, an online and deep learning‐based appearance model. The resulting tracker is evaluated on the VIOT dataset with sequences of both images and camera states, achieving a 68.9% tracking precision compared to DiMP's 49.7%. This approach demonstrates increased tracking duration, improved recovery after occlusions, and faster motions. Additionally, this strategy outperforms random searches by about 3.0%.

Gait recognition is a form of identity verification that can be performed over long distances without requiring the subject's cooperation, making it particularly valuable for applications such as access control, surveillance, and criminal investigation. The essence of gait lies in the motion dynamics of a walking individual. Accurate gait-motion estimation is crucial for high-performance gait recognition. In this paper, we introduce two main designs for gait motion estimation. Firstly, we propose a fully convolutional neural network named W-Net for silhouette segmentation from video sequences. Secondly, we present an adversarial learning-based algorithm for robust gait motion estimation. Together, these designs contribute to a high-performance system for gait recognition and user authentication. In the experiment, two datasets, i.e., OU-IRIS and our own dataset, are used for performance evaluation. Experimental results show that, the W-Net achieves an accuracy of 89.46% in silhouette segmentation, and the proposed user-authentication method achieves over 99.6% and 93.8% accuracy on the two datasets, respectively. • A novel GAN-based learning approach for gait motion extraction. • A W-Net for enhanced gait silhouette extraction. • A new dataset containing 40 subjects for gait recognition. [ABSTRACT FROM AUTHOR]

Medical image analysis presents a significant problem in the field of image registration. Recently, medical image registration has been recognized as a helpful tool for medical professionals. Current state-of-the-art approaches solely focus on source image registration and lack quantitative measurement for object deviation in terms of loosening, subsidence and anteversion related to surgery. In this article, we have provided motion vectors for recognizing the object deviation, in addition to detecting and selecting the feature points. Firstly, the feature points will be detected using Hessian matrix determinants and octave and level sampling. Then the strongest feature points are selected which will be utilized for identifying the object deviation with respect to the reference image through motion vectors. The objective of this work is to combine image registration and temporal differencing to achieve independent motion detection. In comparison to state-of-the-art approaches, the proposed methodology achieves higher Information Ratio (IR), Mutual Information Ratio (MIR) and their lower bounds for image registration. [ABSTRACT FROM AUTHOR]

Underwater target motion estimation is a challenge for ocean military and scientific research. In this work, we propose a method based on the combination of polarization imaging and optical flow for turbid underwater target detection. Polarization imaging can reduce the influence of backscattered light and obtain high-quality images underwater. The optical flow shows the motion and structural information of the target. We use polarized optical flow to obtain the optical flow field and estimate the target motion. The experimental results of different targets under varying water turbidity levels illustrate that our method is realizable and robust. The precision is verified by comparing the results with the precise displacement data and calculating two error measures. The proposed method based on polarized optical flow can obtain accurate displacement information and a good recognition effect. Moving target segmentation based on the Otsu method further proves the superiority of the polarized optical flow under turbid water. This study is valuable for target detection and motion estimation in scattering environments. [ABSTRACT FROM AUTHOR]

Background Subtraction (BgS) is a widely researched technique to develop online Change Detection algorithms for static video cameras. Many BgS methods have employed the unsupervised, adaptive approach of Gaussian Mixture Model (GMM) to produce decent backgrounds, but they lack proper consideration of scene semantics to produce better foregrounds. On the other hand, with considerable computational expenses, BgS with Deep Neural Networks (DNN) is able to produce accurate background and foreground segments. In our research, we blend both approaches for the best. First, we formulated a network called Convolutional Density Approximation (CDA) for direct density estimation of background models. Then, we propose a self-supervised training strategy for CDA to adaptively capture high-frequency color distributions for the corresponding backgrounds. Finally, we show that background models can indeed assist foreground extraction by an efficient Neural Motion Subtraction (NeMos) network. Our experiments verify competitive results in the balance between effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Visual motion estimation is a well-studied challenge in autonomous navigation. Recent work has focused on addressing multimotion estimation in highly dynamic environments. These environments not only comprise multiple, complex motions but also tend to exhibit significant occlusion. Estimating third-party motions simultaneously with the sensor egomotion is difficult because an object's observed motion consists of both its true motion and the sensor motion. Most previous works in multimotion estimation simplify this problem by relying on appearance-based object detection or application-specific motion constraints. These approaches are effective in specific applications and environments but do not generalize well to the full multimotion estimation problem (MEP). This paper presents Multimotion Visual Odometry (MVO), a multimotion estimation pipeline that estimates the full SE (3) trajectory of every motion in the scene, including the sensor egomotion, without relying on appearance-based information. MVO extends the traditional visual odometry (VO) pipeline with multimotion segmentation and tracking techniques. It uses physically founded motion priors to extrapolate motions through temporary occlusions and identify the reappearance of motions through motion closure. Evaluations on real-world data from the Oxford Multimotion Dataset (OMD) and the KITTI Vision Benchmark Suite demonstrate that MVO achieves good estimation accuracy compared to similar approaches and is applicable to a variety of multimotion estimation challenges. [ABSTRACT FROM AUTHOR]

Fast and simple methods for motion estimation with subpixel accuracy are of interest in a variety of applications. In this paper, we extend a recently proposed method for quantifying 1D displacements with subpixel accuracy, referred to as the subtraction method (SM) to 2D motion. Simulation and experimental results are presented. The results indicate that any general motion in 2D involving combinations of in-plane motions in x and y can be determined using SM after a 1D calibration. The errors between the actual motion and estimated are examined. [ABSTRACT FROM AUTHOR]

Subject motion is a long-standing problem of magnetic resonance imaging (MRI), which can seriously deteriorate the image quality. Various prospective and retrospective methods have been proposed for MRI motion correction, among which deep learning approaches have achieved state-of-the-art motion correction performance. This survey paper aims to provide a comprehensive review of deep learning-based MRI motion correction methods. Neural networks used for motion artifacts reduction and motion estimation in the image domain or frequency domain are detailed. Furthermore, besides motion-corrected MRI reconstruction, how estimated motion is applied in other downstream tasks is briefly introduced, aiming to strengthen the interaction between different research areas. Finally, we identify current limitations and point out future directions of deep learning-based MRI motion correction. [ABSTRACT FROM AUTHOR]

Upper-limb paralysis requires extensive rehabilitation to recover functionality for everyday living, and such assistance can be supported with robot technology. Against such a background, we have proposed an electromyography (EMG)-driven hybrid rehabilitation system based on motion estimation using a probabilistic neural network. The system controls a robot and functional electrical stimulation (FES) from movement estimation using EMG signals based on the user's intention, enabling intuitive learning of joint motion and muscle contraction capacity even for multiple motions. In this study, hybrid and visual-feedback training were conducted with pointing movements involving the non-dominant wrist, and the motor learning effect was examined via quantitative evaluation of accuracy, stability, and smoothness. The results show that hybrid instruction was as effective as visual feedback training in all aspects. Accordingly, passive hybrid instruction using the proposed system can be considered effective in promoting motor learning and rehabilitation for paralysis with inability to perform voluntary movements. [ABSTRACT FROM AUTHOR]

Visual object tracking, crucial in aerial applications such as surveillance, cinematography, and chasing, faces challenges despite AI advancements. Current solutions lack full reliability, leading to common tracking failures in the presence of fast motions or long‐term occlusions of the subject. To tackle this issue, a 3D motion model is proposed that employs camera/vehicle states to locate a subject in the inertial coordinates. Next, a probability distribution is generated over future trajectories and they are sampled using a Monte Carlo technique to provide search regions that are fed into an online appearance learning process. This 3D motion model incorporates machine‐learning approaches for direct range estimation from monocular images. The model adapts computationally by adjusting search areas based on tracking confidence. It is integrated into DiMP, an online and deep learning‐based appearance model. The resulting tracker is evaluated on the VIOT dataset with sequences of both images and camera states, achieving a 68.9% tracking precision compared to DiMP's 49.7%. This approach demonstrates increased tracking duration, improved recovery after occlusions, and faster motions. Additionally, this strategy outperforms random searches by about 3.0%. [ABSTRACT FROM AUTHOR]

Abstract Accurate and robust time delay estimation is crucial for synthetic aperture sonar (SAS) imaging. A two‐step time delay estimation method based on displaced phase centre antenna (DPCA) micronavigation has been widely applied in motion estimation and compensation of SASs. However, the existing methods for time delay estimation are not sufficiently robust, which reduces the performance of SAS motion estimation. Deep learning is currently one of the cutting‐edge techniques and has brought about a remarkable progress in the field of underwater acoustic signal processing. In this study, a deep learning‐based time delay estimation method is introduced to SAS motion estimation and compensation. The subband processing is first applied to obtain ambiguous time delays between adjacent pings from phases of SAS echoes. Then, a lightweight neural network is utilised to construct phase unwrapping. The model of the employed neural network is trained with simulation data and applied to real SAS data. The results of time delay estimation and motion compensation demonstrate that the proposed neural network‐based method has much better performance than the two‐step and joint‐subband methods.

Abstract Video stabilization plays a pivotal role in enhancing video quality by eliminating unwanted jitter in shaky videos. This paper introduces a novel video stabilization algorithm that leverages low‐rank constraint and trajectory optimization to effectively eliminate undesirable motion and generate stabilized videos. In the proposed algorithm, a low‐rank constraint regularization term is incorporated to enhance the smoothness of motion trajectories. Additionally, a predictive path smoothness term is integrated to ensure the consistency of motion across neighbouring frames. To address the problem of excessive cropping resulting from aggressive smoothing, a flexible local window strategy that emphasizes local motion relationships within the trajectories is introduced. The experimental results show that, compared to other excellent video stabilization algorithms, the proposed algorithm improves the stability metric by approximately 2.3%. Furthermore, in the stabilized videos generated by the algorithm, an approximate improvement of 2.18 dB in average image temporal fidelity and a 5.7% increase in average structural similarity between adjacent frames are achieved. The code that implements the proposed method is publicly accessible at https://github.com/CZS0319/VS_Low‐Rank_Constraint_Trajectory_Optimization.

Abstract The lateral motion control of an in‐wheel motor drive electric vehicle (IWMD‐EV) necessitates an accurate measurement of the vehicle states. However, these measured states are always affected by delays due to sensor measurements, communication latencies, and computation time, which results in the degradation of the controller performance. Motivated by this issue, a novel reaching law based predictive sliding mode control (NRL‐PSMC) is proposed to maintain the lateral motion control of the IWMD‐EV subjected to unknown time delay. Initially, a PSMC framework is built, in which a predictor integrating with the sliding mode control is designed to eliminate the effect of time delay and generate the virtual control signals. Further, to alleviate the chattering phenomenon, a novel‐reaching law is developed, enabling the vehicle to track the desired states effectively. Subsequently, a dynamic control allocation technique is presented to optimally allocate the virtual control input to the actual control input. The accurate estimation of the aforementioned unknown delay is realized through a delay estimator. Finally, simulation and hardware‐in‐the‐loop experiments are performed for three specific driving manoeuvres, and the results demonstrate the effectiveness of the proposed controller design.

Abstract Recently, online optimisation‐based scene flow estimation has attracted significant attention due to its strong domain adaptivity. Although online optimisation‐based methods have made significant advances, the performance is far from satisfactory as only flow priors are considered, neglecting scene priors that are crucial for the representations of dynamic scenes. To address this problem, the authors introduce a dual‐branch MLP‐based architecture to encode implicit scene representations from a source 3D point cloud, which can additionally synthesise a target 3D point cloud. Thus, the mapping function between the source and synthesised target 3D point clouds is established as an extra implicit regulariser to capture scene priors. Moreover, their model infers both flow and scene priors in a stronger bidirectional manner. It can effectively establish spatiotemporal constraints among the synthesised, source, and target 3D point clouds. Experiments on four challenging datasets, including KITTI scene flow, FlyingThings3D, Argoverse, and nuScenes, show that our method can achieve potential and comparable results, proving its effectiveness and generality.

Abstract Visual surveillance systems have been playing an important role in monitoring and managing at public areas. However, the computational complexity of video compression in these applications is still a great challenge. To meet practical requirements, the authors propose in this paper a low‐complexity surveillance video coding solution in which the most recent Versatile Video Coding (VVC) standard is improved with a novel learning adaptive motion search algorithm. The proposed algorithm is designed based on the temporal motion and spatial texture characteristics of surveillance videos. First, the authors study and define a list of spatial and temporal features which indicates the motion and texture characteristics of surveillance video. These features are used together with a machine learning algorithm to appropriately assign a search range for the VVC motion search. Second, to reduce search points, the authors propose an adaptive Test Zone (TZ) search in which TZ steps are early terminated following the variation of spatial–temporal features. Performance evaluation conducted for a rich set of surveillance videos and relevant benchmarks have shown the superiority of the proposed method, notably with around 33% of encoding time saving when compared with the state‐of‐the art VVC solution and relevant benchmarks while asking for negligible compression loss.

This paper presents Inter prediction Multiple Reference Frames Impact on H266-versitele video coding (VVC) encoder. Video compression plays a crucial role in storing and transmitting video content. Recent developments in video coding have led to the development of the H266-VVC encoder, which aims to achieve greater compression efficiency than its predecessors. One of the key features of the H266-VVC encoder is its inter prediction method, which uses previously encoded frames as reference frames. Multiple reference frames are used in this study to determine how they affect the performance of the H266-VVC encoder. The use of the multiple reference frames, increases compression efficiency by providing more information for inter-prediction. The encoder must compare more frames to find the best match when using multiple reference frames, so it also increases the encoding time. To achieve the best compression efficiency and encoding speed, a trade-off between reference frame number and encoding time must be taken into account when selecting the number of reference frames to use in the H266-VVC encoder. [ABSTRACT FROM AUTHOR]

Video stabilization plays a pivotal role in enhancing video quality by eliminating unwanted jitter in shaky videos. This paper introduces a novel video stabilization algorithm that leverages low‐rank constraint and trajectory optimization to effectively eliminate undesirable motion and generate stabilized videos. In the proposed algorithm, a low‐rank constraint regularization term is incorporated to enhance the smoothness of motion trajectories. Additionally, a predictive path smoothness term is integrated to ensure the consistency of motion across neighbouring frames. To address the problem of excessive cropping resulting from aggressive smoothing, a flexible local window strategy that emphasizes local motion relationships within the trajectories is introduced. The experimental results show that, compared to other excellent video stabilization algorithms, the proposed algorithm improves the stability metric by approximately 2.3%. Furthermore, in the stabilized videos generated by the algorithm, an approximate improvement of 2.18 dB in average image temporal fidelity and a 5.7% increase in average structural similarity between adjacent frames are achieved. The code that implements the proposed method is publicly accessible at https://github.com/CZS0319/VS_Low‐Rank_Constraint_Trajectory_Optimization. [ABSTRACT FROM AUTHOR]

Accurate and robust time delay estimation is crucial for synthetic aperture sonar (SAS) imaging. A two‐step time delay estimation method based on displaced phase centre antenna (DPCA) micronavigation has been widely applied in motion estimation and compensation of SASs. However, the existing methods for time delay estimation are not sufficiently robust, which reduces the performance of SAS motion estimation. Deep learning is currently one of the cutting‐edge techniques and has brought about a remarkable progress in the field of underwater acoustic signal processing. In this study, a deep learning‐based time delay estimation method is introduced to SAS motion estimation and compensation. The subband processing is first applied to obtain ambiguous time delays between adjacent pings from phases of SAS echoes. Then, a lightweight neural network is utilised to construct phase unwrapping. The model of the employed neural network is trained with simulation data and applied to real SAS data. The results of time delay estimation and motion compensation demonstrate that the proposed neural network‐based method has much better performance than the two‐step and joint‐subband methods. [ABSTRACT FROM AUTHOR]

Purpose: Navigator‐based correction of rigid‐body motion reconciling high precision with minimal acquisition, minimal calibration and simple, fast processing. Methods: A short orbital navigator (2.3 ms) is inserted in a three‐dimensional (3D) gradient echo sequence for human head imaging. Head rotation and translation are determined by linear regression based on a complex‐valued model built either from three reference navigators or in a reference‐less fashion, from the first actual navigator. Optionally, the model is expanded by global phase and field offset. Run‐time scan correction on this basis establishes servo control that maintains validity of the linear picture by keeping its expansion point stable in the head frame of reference. The technique is assessed in a phantom and demonstrated by motion‐corrected imaging in vivo. Results: The proposed approach is found to establish stable motion control both with and without reference acquisition. In a phantom, it is shown to accurately detect motion mimicked by rotation of scan geometry as well as change in global B0. It is demonstrated to converge to accurate motion estimates after perturbation well beyond the linear signal range. In vivo, servo navigation achieved motion detection with precision in the single‐digit range of micrometers and millidegrees. Involuntary and intentional motion in the range of several millimeters were successfully corrected, achieving excellent image quality. Conclusion: The combination of linear regression and feedback control enables prospective motion correction for head imaging with high precision and accuracy, short navigator readouts, fast run‐time computation, and minimal demand for reference data. [ABSTRACT FROM AUTHOR]

The lateral motion control of an in‐wheel motor drive electric vehicle (IWMD‐EV) necessitates an accurate measurement of the vehicle states. However, these measured states are always affected by delays due to sensor measurements, communication latencies, and computation time, which results in the degradation of the controller performance. Motivated by this issue, a novel reaching law based predictive sliding mode control (NRL‐PSMC) is proposed to maintain the lateral motion control of the IWMD‐EV subjected to unknown time delay. Initially, a PSMC framework is built, in which a predictor integrating with the sliding mode control is designed to eliminate the effect of time delay and generate the virtual control signals. Further, to alleviate the chattering phenomenon, a novel‐reaching law is developed, enabling the vehicle to track the desired states effectively. Subsequently, a dynamic control allocation technique is presented to optimally allocate the virtual control input to the actual control input. The accurate estimation of the aforementioned unknown delay is realized through a delay estimator. Finally, simulation and hardware‐in‐the‐loop experiments are performed for three specific driving manoeuvres, and the results demonstrate the effectiveness of the proposed controller design. [ABSTRACT FROM AUTHOR]

In the recent era, the utilization of the H.264 encoder has been increasing due to its outstanding performance in video compression. However, compressing video with reduced power is still a challenging issue faced by H.264 encoders. Thus, the proposed study intends to minimize the power consumption of H.264 encoders on FPGA by optimizing the basic components of H.264, thereby enhancing performance. For this purpose, the elements like Motion Estimation, intra-prediction, transform unit and entropy encoder are optimized through the effective schemes introduced in the proposed work. Initially, the Motion Estimation unit can be alternated by optimizing the fundamental components of Block Matching Algorithms. To design the Block Matching Algorithms, the proposed study introduces low-power arithmetic units like an add-one circuit-based Carry SeLect Adder and Sum of Absolute Difference. With the help of these methods, the Block Matching Algorithms has designed, and the Motion Estimation Unit can be effectively optimized. Then, by adopting a comparator-less reusing method, the intra-prediction unit is optimized. Next, the transform unit is optimized by proposing a Steerable Discrete Cosine Transform and finally, the entropy encoders are optimized by combining Golomb and Rice entropy encoders. The proposed study uses the schemes above to improve the efficiency of H.264 encoders on FPGA. The experimental analysis in the proposed study is done using Xilinx software. The simulation results show that the proposed work obtained higher power, LUTs, delay, PSNR, frequency and MSE than other competing methods. [ABSTRACT FROM AUTHOR]

Multi-object tracking (MOT) aims to locate and identify objects in videos. As deep learning brings excellent performances to object detection, the tracking-by-detection (TBD) has gradually become a mainstream tracking framework. However, some drawbacks still exist in the current TBD framework: (1) inaccurate prediction of the bounding boxes would occur in the detection part, which is caused by overlooking the actual pedestrian ratio in the surveillance scene. (2) The width of the bounding boxes in the next frame might be indirectly predicted by the aspect ratio, which increases the error of width prediction in the motion prediction part. (3) Association is only performed for high-confidence detection boxes, and the low-confidence boxes caused by occlusion are discarded in the data association part, resulting in fragmentation of trajectories. To address the above issues, we propose a multi-target tracking model incorporating motion estimation and multi-stage association (MEMA). First, the aspect ratio of the ground-true bounding box is introduced to improve the fit of the detection and the ground-true bounding box, and we design the elliptical Gaussian kernel to improve the positioning accuracy of the object center point. Then, the prediction state vector of the Kalman filter is modified to predict the width and its corresponding velocity directly. It can reduce the width error of the prediction box and eliminate the velocity error of the motion estimation, which leads to a more pedestrian-friendly prediction bounding box. Finally, we propose a multi-stage association strategy to correlate different confidence boxes. Without using the appearance feature, the strategy can reduce the impact of occlusion and improve the tracking performance. On the MOT17 test set, the method proposed in this paper achieves a MOTA of 74.3% and an IDF1 of 72.4%, outperforming the current SOTA. [ABSTRACT FROM AUTHOR]

Versatile Video Coding (VVC) is the latest video coding standard. It provides higher compression efficiency than the previous video coding standards at the cost of significant increase in computational complexity. Motion estimation (ME) is the most time consuming and memory intensive module in VVC encoder. Therefore, in this paper, we propose an efficient VVC ME hardware. It is the first VVC ME hardware in the literature. It has real time performance with small hardware area. This efficiency is achieved by using a 64 × 64 systolic processing element array to support maximum coding tree unit (CTU) size of 128 × 128 and by using a novel memory-based sum of absolute differences (SAD) adder tree to calculate SADs of 128 × 128 CTUs. The proposed VVC ME hardware reduces memory accesses significantly by using an efficient data reuse method. It can process up to 30 4 K (3840 × 2160) video frames per second. [ABSTRACT FROM AUTHOR]

Visual surveillance systems have been playing an important role in monitoring and managing at public areas. However, the computational complexity of video compression in these applications is still a great challenge. To meet practical requirements, the authors propose in this paper a low‐complexity surveillance video coding solution in which the most recent Versatile Video Coding (VVC) standard is improved with a novel learning adaptive motion search algorithm. The proposed algorithm is designed based on the temporal motion and spatial texture characteristics of surveillance videos. First, the authors study and define a list of spatial and temporal features which indicates the motion and texture characteristics of surveillance video. These features are used together with a machine learning algorithm to appropriately assign a search range for the VVC motion search. Second, to reduce search points, the authors propose an adaptive Test Zone (TZ) search in which TZ steps are early terminated following the variation of spatial–temporal features. Performance evaluation conducted for a rich set of surveillance videos and relevant benchmarks have shown the superiority of the proposed method, notably with around 33% of encoding time saving when compared with the state‐of‐the art VVC solution and relevant benchmarks while asking for negligible compression loss. [ABSTRACT FROM AUTHOR]

Recently, online optimisation‐based scene flow estimation has attracted significant attention due to its strong domain adaptivity. Although online optimisation‐based methods have made significant advances, the performance is far from satisfactory as only flow priors are considered, neglecting scene priors that are crucial for the representations of dynamic scenes. To address this problem, the authors introduce a dual‐branch MLP‐based architecture to encode implicit scene representations from a source 3D point cloud, which can additionally synthesise a target 3D point cloud. Thus, the mapping function between the source and synthesised target 3D point clouds is established as an extra implicit regulariser to capture scene priors. Moreover, their model infers both flow and scene priors in a stronger bidirectional manner. It can effectively establish spatiotemporal constraints among the synthesised, source, and target 3D point clouds. Experiments on four challenging datasets, including KITTI scene flow, FlyingThings3D, Argoverse, and nuScenes, show that our method can achieve potential and comparable results, proving its effectiveness and generality. [ABSTRACT FROM AUTHOR]

Motion information from image sequences (videos) plays a key role in solving a number of real-world problems such as surveillance, traffic management, fire identification, weather prediction, and COVID-19 detection, etc. Generally, this motion is estimated in terms of optical flow between two consecutive image frames. Optical flow is a 2D vector field that illustrates object motion behavior. This paper presents a nonlinear fractional order variational model for estimating optical flow. The objective of this work is to provide dense and discontinuity preserving optical flow for different spectra and make the model robust against outliers. In particular, the presented model generalizes the integer order variational models for fractional order (0, 1). For this purpose, the proposed model is formulated with the help of Charbonnier norm and Caputo fractional derivative. Charbonnier norm is nonlinear in nature, which makes the model robust against noise and outliers, whereas Caputo fractional derivatives are well-capable to deal with discontinuous functions such as images, and therefore, preserve motion discontinuities. The Caputo derivative also manifests long-term memory effect and allows to choose the optimal value of the fractional order that corresponds to a stable solution. The numerical implementation of the formulated variational functional is performed by discretizing the fractional derivative using the Grünwald–Letnikov derivative. The resulting system of equations is solved using multi-variable fixed point iteration scheme. The entire framework is embedded in coarse-to-fine warping strategy, which helps in finding the global extremal. The experimental results are carried out on 20 different application oriented datasets such as fire and smoke, fluid, CXR, etc. The performance of the model is tested using different error measures and demonstrated against several outliers. Error concentration is shown through 3D histograms. The edge preserving nature is also realized through intensity distribution in RGB color channels. A detailed convergence analysis is also provided for the presented model. The validity of the proposed model is also verified through comparisons with other existing models. [ABSTRACT FROM AUTHOR]

Purpose: To develop a self‐navigated motion compensation strategy for 3D radial MRI that can compensate for continuous head motion by measuring rigid body motion parameters with high temporal resolution from the central k‐space acquisition point (self‐encoded FID navigator) in each radial spoke. Methods: A forward model was created from low‐resolution calibration data to simulate the effect of relative motion between the coil sensitivity profiles and the underlying object on the self‐encoded FID navigator signal. Trajectory deviations were included in the model as low spatial‐order field variations. Three volunteers were imaged at 3 T using a modified 3D gradient‐echo sequence acquired with a Kooshball trajectory while performing abrupt and continuous head motion. Rigid body‐motion parameters were estimated from the central k‐space signal of each spoke using a least‐squares fitting algorithm. The accuracy of self‐navigated motion parameters was assessed relative to an established external tracking system. Quantitative image quality metrics were computed for images with and without retrospective correction using external and self‐navigated motion measurements. Results: Self‐encoded FID navigators achieved mean absolute errors of 0.69 ± 0.82 mm and 0.73 ± 0.87° relative to external tracking for maximum motion amplitudes of 12 mm and 10°. Retrospective correction of the 3D radial data resulted in substantially improved image quality for both abrupt and continuous motion paradigms, comparable to external tracking results. Conclusions: Accurate rigid body motion parameters can be rapidly obtained from self‐encoded FID navigator signals in 3D radial MRI to continuously correct for head movements. This approach is suitable for robust neuroanatomical imaging in subjects that exhibit patterns of large and frequent motion. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Signal Processing is the property of Journal of Signal Processing and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Block matching methods are one of the most widely used methods in motion estimation and compensation. In this work, we propose a new hybrid block matching motion estimation algorithm based on the Lucas and Kanade method as a distortion criterion to improve the accuracy of estimated motion. The proposed algorithm proceeds in three steps. In the first step, a small hexagonal pattern is used, in order to find the smaller motion vectors and thus fewer searching points. In the second step, the modified large hexagonal pattern is used to identify the direction of motion vectors. In the third step, the small hexagonal search pattern is used to refine the solution search. The proposed algorithm is tested on several both synthetic and real images sequences. The experimental results show that our proposal could achieve good performances in terms of amplitude and angular errors, prediction quality, and computational complexity, compared to some related works.

Abstract The goal of this paper is twofold: firstly, to provide a novel mathematical model that describes the kinematic chain of motion of the human fingers based on Lagrangian mechanics with four degrees of freedom and secondly, to estimate the model parameters using data from able‐bodied individuals. In the literature there are a variety of mathematical models that have been developed to describe the motion of the human finger. These models offer little to no information on the underlying mechanisms or corresponding equations of motion. Furthermore, these models do not provide information as to how they scale with different anthropometries. The data used here is generated using an experimental procedure that considers the free response motion of each finger segment with data captured via a motion capture system. The angular data collected are then filtered and fitted to a linear second‐order differential approximation of the equations of motion. The results of the study show that the free response motion of the segments is underdamped across flexion/extension and ad/abduction.

Abstract This study compared the accuracy of facial landmark measurements using deep learning‐based fiducial marker (FM) and arbitrary width reference (AWR) approaches. It quantitatively analysed mandibular hard and soft tissue lateral excursions and head tilting from consumer camera footage of 37 participants. A custom deep learning system recognised facial landmarks for measuring head tilt and mandibular lateral excursions. Circular fiducial markers (FM) and inter‐zygion measurements (AWR) were validated against physical measurements using electrognathography and electronic rulers. Results showed notable differences in lower and mid‐face estimations for both FM and AWR compared to physical measurements. The study also demonstrated the comparability of both approaches in assessing lateral movement, though fiducial markers exhibited variability in mid‐face and lower face parameter assessments. Regardless of the technique applied, hard tissue movement was typically seen to be 30% less than soft tissue among the participants. Additionally, a significant number of participants consistently displayed a 5 to 10° head tilt.

When applying continuous motion estimation (CME) model based on sEMG to human-robot system, it is inevitable to encounter scenarios in which the motions performed by the user are different from the motions in the training stage of the model. It has been demonstrated that the prediction accuracy of the currently effective approaches on untrained motions will be significantly reduced. Therefore, we proposed a novel CME method by introducing muscle synergy as feature to achieve better prediction accuracy on untrained motion tasks. Specifically, deep non-smooth NMF (Deep-nsNMF) was firstly introduced on synergy extraction to improve the efficiency of synergy decomposition. After obtaining the activation primitives from various training motions, we proposed a redundancy classification algorithm (RC) to identify shared and task-specific synergies, optimizing the original redundancy segmentation algorithm (RS). NARX neural network was set as the regression model for training. Finally, the model was tested on prediction tasks of eight untrained motions. The prediction accuracy of the proposed method was found to perform better than using time-domain feature as input of the network. Through Deep-nsNMF with RS, the highest accuracy reached 99.7%. Deep-nsNMF with RC performed similarly well and its stability was relatively higher among different motions and subjects. Limitation of the approach is that the problem of positive correlation between the prediction error and the absolute value of real angle remains to be further addressed. Generally, this research demonstrates the potential for CME models to perform well in complex scenarios, providing the feasibility of dedicating CME to real-world applications.

Dim and small target detection plays an important role in infrared target recognition systems. In this paper, we present a dim and small target detection algorithm based on improved bilateral filtering and Gaussian motion probability estimation, aiming to improve the detection efficiency of the detection system. First, a bilateral filtering algorithm based on image patch analysis is proposed to complete the background modeling, compare with single pixel, image patch contains more neighborhood information. Then, we use the Gaussian process combining the target position of consecutive $n$ frames to predict the target position of the $(n+1)\text{th}$ frame, and the target energy is accumulated along the trajectory direction at the same time. Finally, we construct the grayscale probability model to realize the multi-frame correlation detection, which combining the grayscale features and the motion characteristics of the target. Six scenes and eleven comparison algorithms are selected for experiments, experimental results show the effectiveness and robustness of the proposed algorithm.

Video super-resolution is a fundamental task in computer vision, aiming to enhance the resolution and visual quality of low-resolution videos. Plug-and-Play Priors is one of the most widely used frameworks for solving computational imaging problems by integrating physical and learned models. Traditional approaches often rely on handcrafted priors, which are computationally expensive and may not generalize well to diverse video content. In this paper, we propose a novel approach for video super-resolution using Plug-and-Play Priors with motion estimation. By leveraging the power of deep learning and the flexibility of the Plug-and-Play framework, our method achieves promising results while maintaining computational efficiency. Experimental results on benchmark datasets demonstrate the superiority of our approach in terms of both quantitative metrics and visual quality.

With the escalating requirements for maintenance of wind turbines, the deployment of Unmanned Aerial Vehicles (UAVs) for inspection tasks has become increasingly prevalent. However, wind turbine blades, which are thin and long, possess weak texture features that lead to target confusion when tracking specific parts of the dynamic blades. Additionally, wind turbine units, being large dynamic structures, often exceed the camera’s field of view (FOV) and exhibit unique motion characteristics. These factors make the visual tracking of specific components unstable due to the lack of global motion information. In order to address the aforementioned challenges and achieve consistent calibration of key components under the dynamic operating conditions of wind turbines, this study has adopted a strategy of integrating the Squeeze-and-Excitation Network (SEnet) into the backbone network of YOLOv5. Innovatively, two hyperparameters have been introduced into the existing loss function to adjust the weights of samples under conditions of data imbalance, thereby enhancing the performance of the detection model. In the application of the DeepSORT tracking algorithm, Long Short-Term Memory (LSTM) networks have been combined to predict the trajectory of the rotor blade’s central point, and an optimized Kalman filter has been employed to significantly improve the system’s adaptability and precision under various motion conditions. Empirical results from this study underscore the efficacy of the proposed method, demonstrating its capability to accurately differentiate individual blades as well as specific blade segments. Compared to the traditional YOLOv5, the enhanced YOLOv5-SE has demonstrated a 5.3% improvement in the Mean Average Precision (mAP_0.5) evaluation metric. Moreover, the improved DeepSORT algorithm has exhibited high efficiency in maintaining continuous and stable tracking of moving blades, adeptly handling scenarios where rotor blades frequently enter and exit the FOV. This advancement paves the way for the broader application of UAVs in wind turbine inspections, offering the potential for more efficient and accurate maintenance protocols.

In this article, we consider using time-of-arrival (TOA) measurements from a single moving receiver to locate a moving target at constant velocity that emits a periodic signal with unknown signal period. First, we give the TOA measurement model and deduce the Cram $\acute{\text{e}}$ r-Rao lower bounds (CRLB). Then, we formulate a nonlinear least squares (NLS) problem to estimate the unknown parameters. We use semidefinite programming (SDP) techniques to relax the nonconvex NLS problem. However, it is shown that the SDP localization algorithm cannot provide a high-quality solution. Subsequently, we develop a fixed point iteration (FPI) method to improve the performance of the SDP algorithm. In addition, we also consider the presence of receiver position errors and develop the corresponding localization algorithm. Numerical simulations are conducted to demonstrate the localization performance of the proposed algorithms by comparing them with the CRLB. Index Term-Fixed point iteration (FPI), semidefinite programming (SDP), single moving receiver, target localization, time-of-arrival (TOA).

Abstract Precipitation forecasting plays an important role in disaster warning, agricultural production, and other fields. To solve this issue, some deep learning methods are proposed to forecast future radar echo images and convert them into rainfall distributions. Prevailing spatiotemporal sequence prediction methods are usually based on a ConvRNN structure that combines a Convolutional Neural Network and Recurrent Neural Network. However, these existing methods ignore the image change prediction, which causes the coherence of the predicted image has deteriorated. Moreover, these approaches mainly focus on complicating model structure to exploit more historical spatiotemporal representations. Nevertheless, they ignore introducing other valuable information to improve predictions. To tackle these two issues, we propose GCMT‐ConvRNN, a multi‐ask framework of ConvRNN. Except for precipitation nowcasting as the main task, it combines the motion field estimation and sub‐regression as auxiliary tasks. In this framework, the motion field estimation task can provide motion information, and the sub‐regression task offers future information. Besides, to reduce the negative transfer between the auxiliary tasks and the main task, we propose a new loss function based on the correlation of gradients in different tasks. The experiments show that all models applied in our framework achieve stable and effective improvement.

Abstract Inspired by biological vision mechanism, event‐based cameras have been developed to capture continuous object motion and detect brightness changes independently and asynchronously, which overcome the limitations of traditional frame‐based cameras. Complementarily, spiking neural networks (SNNs) offer asynchronous computations and exploit the inherent sparseness of spatio‐temporal events. Notably, event‐based pixel‐wise optical flow estimations calculate the positions and relationships of objects in adjacent frames; however, as event camera outputs are sparse and uneven, dense scene information is difficult to generate and the local receptive fields of the neural network also lead to poor moving objects tracking. To address these issues, an improved event‐based self‐attention optical flow estimation network (SA‐FlowNet) that independently uses criss‐cross and temporal self‐attention mechanisms, directly capturing long‐range dependencies and efficiently extracting the temporal and spatial features from the event streams is proposed. In the former mechanism, a cross‐domain attention scheme dynamically fusing the temporal‐spatial features is introduced. The proposed network adopts a spiking‐analogue neural network architecture using an end‐to‐end learning method and gains significant computational energy benefits especially for SNNs. The state‐of‐the‐art results of the error rate for optical flow prediction on the Multi‐Vehicle Stereo Event Camera (MVSEC) dataset compared with the current SNN‐based approaches is demonstrated.