Search

Pedestrians represent acutely complex and agile targets whose accident prevention has the highest priority concerning highly automated vehicles in urban traffic. Consequently, detection and tracking methods are necessary to determine motion behavior quickly and with high precision. With advancing microelectronics, powerful radar sensors and novel antenna structures are available for pedestrian detection systems, which can utilize unique features such as micro-Doppler signatures in several dimensions and high resolution. This paper presents a micro-Doppler based leg tracking framework that enables behavioral indications in a few sensor cycles. An adaptive motion model representing the kinematic locomotion of a human is derived to propagate the foot motion. Joint probabilistic data association assigns the detections to the respective leg and allows its temporal filtering of the position, micro-Doppler velocity, and kinematic parameters using an extended Kalman filter yielding real-time implementation capability. The presented approach provides the entire signal chain from raw radar data processing to extended multi-object state estimation based on highly relevant safety-critical motion maneuvers measured with radar parameterization corresponding to current serial short-range sensors.

Radar and camera information fusion sensing methods are used to solve the inherent shortcomings of the single sensor in severe weather. Our fusion scheme uses radar as the main hardware and camera as the auxiliary hardware framework. At the same time, the Mahalanobis distance is used to match the observed values of the target sequence. Data fusion based on the joint probability function method. Moreover, the algorithm was tested using actual sensor data collected from a vehicle, performing real-time environment perception. The test results show that radar and camera fusion algorithms perform better than single sensor environmental perception in severe weather, which can effectively reduce the missed detection rate of autonomous vehicle environment perception in severe weather. The fusion algorithm improves the robustness of the environment perception system and provides accurate environment perception information for the decision-making system and control system of autonomous vehicles.

In order to improve the speed of passenger screening while preserving the effective capability to detect more sophisticated threats, airport security imaging systems must be able to accurately characterize concealed body-worn objects. In addition to improving the passenger experience, this system capability will enhance airport security for the traveling public. This paper presents a real-time, fully-automatic algorithm for the wideband millimeter-wave (mm-wave) radar reconstruction of the nominal human body contours, even in the presence of an affixed weak dielectric object or when a portion of the body cross-section is not captured by the imaging scanner. The algorithm extracts the main contours from a noisy collection of 3D reconstructed reflectivity and approximates the nominal human body cross-section via fitting a low order angular Fourier series. This important step is essential for precise characterization of concealed body-worn explosives. A ranking algorithm is developed as a metric for the nominal body reconstruction accuracy. We verify the developed algorithm by applying it to the actual images of the High Definition-Advanced Imaging Technology (HD-AIT) system, a laboratory prototype mm-wave scanning system developed recently by the US Department of Homeland Security (DHS). The reconstructed body contours may be used to estimate the electric permittivity of the concealed person-worn objects.

For autonomous driving, radar sensors provide superior reliability regardless of weather conditions as well as a significantly high detection range. State-of-the-art algorithms for environment perception based on radar scans build up on deep neural network architectures that can be costly in terms of memory and computation. By processing radar scans as point clouds, however, an increase in efficiency can be achieved in this respect. While Convolutional Neural Networks show superior performance on pattern recognition of regular data formats like images, the concept of convolutions is not yet fully established in the domain of radar detections represented as point clouds. The main challenge in convolving point clouds lies in their irregular and unordered data format and the associated permutation variance. Therefore, we apply a deep-learning based method introduced by PointCNN that weights and permutes grouped radar detections allowing the resulting permutation invariant cluster to be convolved. In addition, we further adapt this algorithm to radar-specific properties through distance-dependent clustering and pre-processing of input point clouds. Finally, we show that our network outperforms state-of-the-art approaches that are based on PointNet++ on the task of semantic segmentation of radar point clouds., Comment: 5th International Conference on Artificial Intelligence, Automation and Control Technologies (AIACT 2021), 26-28 March 2021, Shanghai, China

Millimeter wave (mmWave) sensing promises to enable contactless and high-precision “in air" gesture-based human-computer interaction (HCI). While previous works have demonstrated its feasibility, they require tedious gesture collecting for person-independent recognition and they operate in an off-line mode without considering practical issues like segmenting gesture and recognition latency. In this work, we propose , a person-independent real-time mmWave gesture recognition solution. We first build a compact gesture model with a custom-designed neural network to distill the unique features underlying each gesture, while suppressing personalized discrepancy across different users without extra collection and re-training. Furthermore, we design a system status transition to decide when a gesture begins and ends, which enables automatic gesture segmentation and hence real-time recognition. We prototype on a commodity mmWave sensor and demonstrate its advantages using two practical applications: a contactless music player and camera. Extensive experiments and user studies show that has an accuracy of 99% and a short response latency within 25ms. Moreover, we also collect and release a comprehensive mmWave gesture dataset consisting of 54,620 instances from 144 persons, which may have an independent value of facilitating future research.

Autonomous mobile systems such as vehicles or robots are equipped with multiple sensor modalities including Lidar, RGB, and Radar. The fusion of multi-modal information can enhance task accuracy but indiscriminate sensing and fusion in all modality increases demand on available system resources. This paper presents a task-driven approach to input fusion that minimizes utilization of resource-heavy sensors and demonstrates its application to Visual-Lidar fusion for object tracking and path planning. Proposed spatiotemporal sampling algorithm activates Lidar only at regions-of-interest identified by analyzing visual input and reduces the Lidar ‘base frame rate’ according to kinematic state of the system. This significantly reduces Lidar usage, in terms of data sensed/transferred and potentially power consumed, without severe reduction in performance compared to both a baseline decision-level fusion and state-of-the-art deep multi-modal fusion.

Recently, the long coherent accumulation of radar signals becomes relevant in the analysis of Doppler portraits of targets. This interest is related to the desire of researchers to obtain more information about radar purposes. Detailed spectral analysis provides additional information on the parameters of the target and its maneuvers. Some studies published in special articles are being discussed. They analyzed the methods of high resolution for the spectrums of refl ected signals. It is possible to obtain detailed Doppler portraits. The development of the elemental base of the construction of radar devices contributes to the improvement of the processing processes of refl ected signals. The possibilities of highlighting useful signals of small objects against the background of refl ections from the sea surface are considered. The article provides the results of processing signals recorded by radars made by the technology of digital antenna array. The experimental results of the Doppler portraits used a long-term coherent accumulation of signals in the L and X bands. Doppler portraits of ISS signals, Boeing-type passenger aircraft, MiG-29 fi ghter, helicopter are analyzed. The possibility of compensating for changes in the signal phase, which occurs during the radial acceleration of the ISS is shown. This allowed to increase the of signal-to-noise ratio to 6 dВ. Experimental results allow you to observe the maneuver of the target, to measure the acceleration of individual elements of the structure and their evolution. It is proposed to use the method of assessing Doppler portraits in the «radial speed - radial acceleration» coordinate system. The method is useful to use in radar surveillance systems of space objects, in the development of methods for recognizing classes of radar targets and in determining the number of objects in the target group. The development of this method of analysis of signals in prospective RADAR with the digital antenna array will help to solve problems of optimal signal processing, identify the number of targets in the group and maneuver the target, recognize classes of targets by the number of brilliant points and cross-Size targets.

Ship detection using polarimetric synthetic radar (PolSAR) imagery attracts a lot of attention in recent years. Most notably, the detector polarimetric notch filter (PNF) has been demonstrated to be effective for ship detection in PolSAR imagery, which gives excellent performances. In this work, a mathematical form of one new PNF (NPNF) based on physical mechanisms of targets and clutter is further developed for partial targets. The different mechanisms have been revealed based on the projection matrix. The experimental results including simulated and measured data demonstrate that the NPNF exhibits a better performance than the original PNF.

Synthetic aperture radar raw data simulation (SAR-RDS) is beneficial to the SAR system design, signal processing method verification, and radar parameter optimization. Most SAR-RDS methods are based on the flat terrain assumption. However, the fluctuant terrain in real scene will induce severe SAR beam occlusion effect and produce radar shadow, leading to incorrect RDS results. Thus, a dynamic elevation angle interpolation (DEAI) algorithm is proposed for SAR shadow judgment by considering the actual SAR working process. The key of the proposed DEAI algorithm is the 1-D EAI and shadow visualization update, which avoids the problem that the existing methods cannot judge the shadow of partial areas due to the insufficiently refined mesh grid or the mismatch of the judgment model. Moreover, an evaluation framework named as joint image and signal criteria (JISC) is proposed from the perspectives of SAR imaging and signal processing results to objectively evaluate the SAR-RDS results and solve the problem that the existing evaluation methods cannot be compatible with fluctuant terrain. Finally, the numerical experiment verified our theoretical analyses.

In this paper, an ultra-wideband (UWB) radar is firstly employed to probe through the opaque wall media to detect behind-the-wall human motions. By employing such a radar, a high-resolution time-range map with different body parts’ reflections highly discriminable in range direction can be obtained. Secondly, a high-pass filter is applied to remove the wall effects in the raw time-range map. Then, with the aim of exploiting the rich range information so as to enhance their corresponding micro-Doppler features, a novel range-max enhancement strategy is proposed to extract the most significant micro-Doppler feature of each time-frequency cell along range direction for a specific motion. Lastly, the effectiveness of the proposed motion feature enhancement strategy is investigated by means of onsite experiments. Comparative classifications using different convolutional neural network structures (CNNs) show that the proposed approach outperforms other state-of-art micro-Doppler feature extraction methods. The comparison with narrowband detection case also proves its superiority in feature enhancement in narrowband detection scene.

With the popularity of unmanned aerial vehicles (UAVs), how to conduct automatic and effective detection to prevent unauthorized flying has become an important issue. The conventional constant false alarm rate (CFAR) detector based on radar signal has shown advantages in moving target detection. However, the CFAR-based detectors are strongly dependent on some manual experience, such as the ambient noise distribution estimation and the detection windows' size selection, and usually suffered poor performance on small UAV detection due to the weak signal. Inspired by the success of deep learning (DL) on natural scene object detection, this article tries to explore a DL-based method for UAV detection in pulse-Doppler radar. Concretely, we propose a convolutional neural network (CNN) with two heads: one for the classification of the input range-Doppler map patch into target present or target absent and the other for the regression of offset between the target and the patch center. Then, based on the output of the network, a nonmaximum suppression (NMS) mechanism composed of probability-based initial recognition, distribution density-based recognition, and voting-based regression is developed to reduce false alarms as well as control the false alarms. Finally, experiments on both simulated data and real data are carried out, and it is shown that the proposed method can locate the target more accurately and achieve a much lower false alarm rate at a comparable detection rate than CFAR.

With the development of the radar sensing technology, hand gesture sensing and recognition has attracted much attention. This letter adopts a frequency-modulated continuous wave (FMCW) radar to achieve short-range hand gesture sensing and recognition. Specifically, the range, Doppler, and angle parameters of hand gestures are measured by fast Fourier transformation (FFT) and multiple signal classification (MUSIC) algorithm, respectively. The mixup (MP) algorithm combined with augmentation (AU) algorithm using a weight factor is applied to expand the hand gesture data. Then, a complementary multidimensional feature fusion network-based hand gesture recognition (CMFF-HGR) is designed to extract the features and achieve HGR. Finally, a series of experiments are carried out to verify the effectiveness of the proposed approach, and the results show that the recognition accuracy is higher than the existing alternatives with low computational complexity.

Microwave imaging with large rotating angle and sparse sampling is an attractive approach to obtain the high-resolution target image with reduced radar resource. However, the popular imaging methods, e.g., Range-Doppler (RD), back projection (BP), and sparse recovery (SR), are difficult to deal with large rotating angle and sparse sampling simultaneously. In recent years, deep learning (DL) has been widely studied and been successfully used to handle the problems in computer vision. However, since most existing DL networks are put forward for the real visual image and a large amount of data is essential for network training, DL cannot be directly used to process the complex and sparse target echo for microwave imaging. In this article, a new learning imaging framework is proposed and a model-driven learning imaging network (MDLI-Net) is built for high-resolution microwave imaging with large rotating angle and sparse sampling. In the proposed framework, the electromagnetic scattering model is used to generate the training data efficiently, and the sparse microwave imaging theory is applied to guide the design of the deep imaging network. By inputting the 2-D sparse complex-valued target echo, the trained MDLI-Net can output the high-resolution and focused target image efficiently. The effectiveness of the proposed learning imaging method is validated by experiment results with both simulated and real data.

Ultrawideband (UWB) multiple-input-multiple-output (MIMO) radar is widely used for through-wall imaging (TWI) due to its excellent penetrability and large aperture. Multichannels in the MIMO radar system are usually time-division multiplexing based on microwave switches to reduce the complexity of the system in engineering. The switching process of the channel will bring time delay, which cannot be ignored in the TWI of the moving target. The switching time delay will cause the defocus and position shift of the TWI of the moving target. This letter proposes a motion compensation method based on multiframe data fusion (MFDF) used for correcting the echo of the through-wall moving target. A geometric model is established in the proposed method through the echo of the current frame and the next frame, and the compensated signal is obtained through the geometric solution. The proposed method is compared with before compensation and the traditional single-channel motion compensation algorithm (SCMCA) through simulation and experimental data verification. The visual images and quantitative results show that the proposed motion compensation method can obtain a good focus image of the through-wall moving target and reduce the positioning error.

Millimetre-wave (mmWave) radar is increasing in popularity for human activity recognition, due to its advantages of high resolution, non-intrusive nature and suitability for various environments. In this paper, we present a novel human posture estimation system using mmWave radars. The system detects people with arbitrary postures in indoor environments at close distances (within two metres), and estimates the posture by localising the key joints. We use two mmWave radars to capture the scene and a neural network model to estimate the posture. The neural network model consists of a part detector that estimates the subject’s joint positions, and a spatial model that learns the correlation between the joints. A temporal correlation step is introduced to further refine the estimate when in real-time operation. The system can provide an accurate posture estimate of the person in real-time at 20 fps, with a mean localisation error of 12.2 cm and an average precision of 71.3%.

Nonline-of-sight (NLOS) radar imaging is a novel technique that can inverse the scattering characteristics of targets in the NLOS area, which has been one of the hot pots of radar imaging field. However, the existing NLOS radar mainly focuses on 1-D or 2-D imaging, which inevitably suffers from the geometric loss of real 3-D scenes, and its applications are restricted in the urban environment. In this article, we propose an NLOS radar 3-D imaging model and method for looking around corner (LAC) situation by multi-input-multioutput (MIMO) millimeter-wave (mmW) array antennas. In this scheme, first, the model of NLOS radar 3-D imaging with mmW MIMO antennas is established and the multipath scattering of targets with this model is analyzed. Then, the theoretical resolution of LAC 3-D imaging is derived and discussed. Second, exploiting the three bounces of LAC and extraction of linear structure, an effective imaging algorithm with mirror projection theory and Radon transform, dubbed as mirror symmetry backprojection (MSBP), is proposed for 3-D image focusing. Moreover, to suppress the uncertainties of phase caused by both LAC and system error, the minimum entropy principle is introduced to MSBP. Finally, an NLOS 3-D imaging system with 77-GHz mmW MIMO radio frequency module and 2-D rails is developed. Different types of targets, such as metal balls and ornaments, are tested in LAC. The results demonstrate that our NLOS technique can not only provide a high-quality 3-D focusing of the hidden targets but also extract positions of targets without prior knowledge of the NLOS area.

Vertical-looking radar (VLR) is a significant milestone in the development of insect radars with the capability of detecting the behavior of migratory insects and their biological parameters. In current VLRs, high-speed continuous sampling and long-time integration can barely be performed simultaneously, leading to a low detection probability for tiny insects (weight < 10 mg). Based on the large amount of data acquired by our developed high-range resolution insect radar, the insect echo signals and vertical motion characteristics are initially analyzed and demonstrate that the linear-motion mode is dominant in insect migration; also, the echo signal power of most insects follows the gamma distribution. Based on these characteristics, a long-time integration and detection method for detecting migratory insects, especially tiny targets from echo signals that often dip below the noise level, is proposed. The radial target velocity is also measured as one of the output parameters. The theoretical derivation and optimal choice of detection thresholds are also presented. Simulation and experimental results demonstrate that the proposed method exhibits better insect detection performance and effectively increases the detection range compared with conventional methods. In addition, the measured target velocity can be directly applied to current continuous-sampling VLRs for the ascent and descent rate analysis. Many typical insect migration phenomena have been detected effectively utilizing our developed VLR, and the measured ascent and descent rates of insects agree well with typical take-off, cruising, and landing behaviors. This is the first reported successful VLR application on take-off and landing behaviors of migratory tiny and dense insects.

Traditional micro-Doppler (m-D)-based human activity classification system using monostatic radar suffers from the drawback that classification performance is vulnerable to the variation of human motion aspect angle. This leads to a performance degradation if the human movements are not directly toward or away with respect to the radar line of sight. The multistatic radar system has been suggested as an effective solution to solve the problem, as it can observe the target from multiple views and achieve favorable aspect angles to the targets. In this article, a novel human activity classification method based on motion orientation determining using multistatic m-D signals is proposed. First, the aspect angles of target motion direction with respect to each radar nodes are inferred by using the proposed motion orientation estimation method. The multistatic m-D data are then divided into several intervals based on the measured angle, and the data in the same interval are fused at the data level. Finally, the classification results are obtained through the adaptive weighted decision-level fusion. Compared with the traditional multistatic classification method, due to the consideration of the time-varying human motion aspect angle, the proposed method is more reasonable in data fusion and has better classification performance.

Traditional range-instantaneous Doppler (RID) methods for maneuvering target imaging suffer from the problems of low resolution and poor noise suppression. We propose a new super-resolution inverse synthetic aperture radar (ISAR) imaging method based on deep-learning-assisted time-frequency analysis (TFA). Our deep neural network resembles the basic structure of a U-net with two additional convolutional-upsampling layers and l₁-norm loss function for super-resolution generation and noise suppression. The neural network is trained in advance to learn the mapping function between the low-resolution time-frequency spectrum inputs and their high-resolution references. Then, the linear TFA assisted by the pretrained network is integrated into the RID-based ISAR imaging system and is found to achieve sharply focused and denoised target image with super-resolution. Both the simulated and real radar data are used to evaluate the performance of the proposed method. Numerical experimental results demonstrate the superiority of the proposed ISAR imaging method over traditional ones.

Low imaging spatial resolution hinders through-the-wall radar imaging (TWRI) from reconstructing complete human postures. This letter mainly discusses a convolutional neural network (CNN)-based human posture reconstruction method for TWRI. The training process follows a supervision-prediction learning pipeline inspired by the cross-modal learning technique. Specifically, optical images and TWRI signals are collected simultaneously using a self-develop radar containing an optical camera. Then, the optical images are processed with a computer-vision-based supervision network to generate ground-truth human skeletons. Next, the same type of skeleton is predicted from corresponding TWRI signals using a prediction network. After training, the model shows complete predictions in wall-occlusive scenarios solely using TWRI signals. Experiments show comparable quantitative results with the state-of-the-art vision-based methods in nonwall-occlusive scenarios and accurate qualitative results with wall occlusion.

Person reidentification (ReID) systems play a key role in intelligent visual surveillance systems and have widespread applications, for example, in public security. Usually, person ReID systems can identify a person with cameras. In this article, we focus on the relatively unexplored area of using low-cost automotive radar for the person ReID problem. Unlike the radar-based person identification, person ReID has some characteristics, such as the uncooperative scenes and the long-term robustness. Therefore, we design a new deep learning network to extract spatiotemporal information from 4-D radar point clouds. We also build a data set of radar point clouds collected from the real-world person ReID scenarios. The evaluation result shows that our method achieves 91% CMC-1 accuracy on the ReID task. Besides, for the person identification task, our method also achieves accuracies of 98% and 91% for 15 and 40 individuals, respectively. In addition, we discuss the potential of using radar for person ReID problems and intuitively explain the new method's performance. Finally, we analyze the robustness and the influence of different parameters on the method and the contributions of different modules to the network model. The results of our experiment indicate that radar-based ReID not only preserves privacy but also outperforms camera-based ReID in some cases, such as in low-light environments or with substantial clothing changes.

Ultrawideband (UWB) radar can achieve ultrahigh-resolution inverse synthetic aperture radar (ISAR) imaging of noncooperative targets by transmitting UWB signals. However, the spatial-variant (SV) high-order migration through range cell (MTRC) and phase errors produced by the UWB radar system have seriously challenged the feasibility of conventional ISAR imaging algorithms. Moreover, maneuvering targets has exacerbated this problem compared with the steady ones. In this article, a UWB ISAR imaging algorithm of maneuvering targets with joint high-order motion compensation and azimuth scaling (JHOMCAS) is proposed. For the azimuth SV linear MTRC and 2-D SV high-order MTRC caused by the maneuvering rotational motion of the targets, the cascaded generalized keystone transform (GKT) is adopted for precise correction. It is worth noting that, when eliminating the SV MTRC by the cascaded GKT, the 2-D SV high-order phase errors induced by the maneuvering rotational motion must be accurately compensated, or MTRC correction will fail. The traditional autofocus methods usually only address the phase errors shared by the total target without due attention to the fine SV property. In response to this problem, this article first develops a joint 2-D SV autofocus and azimuth scaling algorithm (JSVAAS) to achieve the integration of SV high-order phase error compensation and azimuth scaling. A JHOMCAS algorithm is proposed to perform the joint processing of GKT and JSVAAS, ``GKT-JSVAAS-GKT.'' This approach helps accomplish the high-precision UWB ISAR imaging of maneuvering targets, and the well-focused and scaled UWB ISAR images obtained will build a sound foundation for target classification and recognition. Extensive experiments based on both scattering point simulation data and electromagnetic calculation data verify that the proposed algorithm outperforms conventional ISAR imaging approaches in UWB ISAR imaging of maneuvering targets.

This letter deals with an imaging strategy able to manage effectively data collected on multiple lines by means of a Mini-Unmanned Aerial Vehicle (M-UAV) radar system operating in Sounder mode. The strategy, named multilines imaging approach (MIA), allows an effective 3-D pseudo-representation of the investigated volume. At the first step, MIA faces the problem of reconstructing 2-D domains (slices) by exploiting data collected on one or more lines. Then, MIA interpolates the 2-D reconstructions to provide the 3-D representation. The imaging of each slice is formulated as a linear inverse scattering problem, which is solved by means of the truncated singular value decomposition (TSVD) regularization scheme. The MIA effectiveness is assessed by processing real data collected at Archeological Park of Paestum and Velia, Paestum, Italy.

Radio frequency interference (RFI) can significantly pollute synthetic aperture radar (SAR) data and images, which is also harmful to SAR interferometry (InSAR) for retrieving elevational information. To address this issue, in recent years, a class of advanced RFI suppression methods has been proposed based on narrowband properties of RFI and sparsity assumptions of radar echoes or target reflectivity. However, for SAR echoes and the associated scene reflectivity, these assumptions are usually not feasible when the imaged scene is spatially extended. In view of these problems, this study proposes an InSAR-based RFI suppression method for the case of extended scenes. For this task, we combine the RFI-polluted SAR data with RFI-free interferometric data to form an interferometric SAR data pair. We show that such an InSAR data pair embeds an interferogram having the image amplitude multiplying by a complex exponential interferometric phase. We treat the interferogram as a kind of natural image and use discrete Fourier cosine transform (DCT) for its sparse representation. Then combining the DCT-domain sparsity with low-rank modeling of RFI, we retrieve the interferogram and reconstruct the SAR image via joint low-rank and sparse optimization. Numerical simulations show that the proposed method can effectively recover SAR images and interferometric phases from RFI-polluted SAR data.

Simultaneous multitarget vital signs measurement has become a hot issue for noncontact vital signs perception. However, there is still challenge in the multitarget heartbeat measurement due to the weakness of heartbeat signal and interference from complex environment. In this article, a new multiple-input–multiple-output (MIMO) continuous-wave (CW) radar system equipped with 2-D digital beamforming (DBF) is presented to measure the respiration and heartbeat of multiple human subjects at unknown positions simultaneously. Through 2-D beam scanning of the whole scene, a 2-D radar image is generated. From the image, the chest motion of multiple targets is accurately located. Then, the vital signs of targets are obtained through forming individual beams focusing on the chests of targets. Moreover, the low intermediate frequency (low-IF) architecture is adopted to minimize the impact of flicker noise in low-frequency amplifier stages. The experimental results demonstrate that the proposed MIMO 2-D imaging radar system can locate chest areas of multiple targets, suppress the clutters, and make vital signs measurement, heartbeat measurement in particular, more robust compared with single-input–multiple-output (SIMO) radar system in complex environment.

In this paper, a dual-frequency continuous wave radar is proposed to achieve both localization and activity/ gesture recognition simultaneously. Specifically, features of different movements will be classified by the activity and gesture recognition network (AGRNet) which is a lightweight network based on MobileNet. The data that are recognized corresponding to walking will be used for moving target localization by comparing the phase difference in the Doppler domain between dual frequencies. In addition, a segmentation method is proposed to effectively segment continuous signals into individual time-periods corresponding to different motions by detecting the boundaries of signal changing. The experimental results show that the proposed method accomplishes the classification accuracy over 91% with 8 motion classes with a localization accuracy in the centimeter level.

In the actual scene of interferometric inverse synthetic aperture radar (InISAR) imaging, the noncooperative targets may make a nonuniform 3-D rotational motion (3-D-RM), which contributes not only to the time-variant Doppler modulation but also to the spatial-variant wave path difference (SVWPD). This, in turn, seriously degrades the 3-D geometry reconstruction accuracy of the targets. Furthermore, it is an enormous challenge to realize InISAR imaging from sparse frequency band and sparse aperture (SFB-SA) signals. This article seeks to address the problems of fine image registration and 2-D joint sparse reconstruction (2-D-JSR) for InISAR imaging with SFB-SA signals. With regard to the maneuvering targets with 3-D-RM, a novel SVWPD signal model is established. Moreover, a new algorithm, named joint wave path difference compensation (JWPDC) algorithm, is developed to perform fine image registration. It can not only combine multiple channels to achieve image registration but also jointly compensate for the non-SVWPD (NSVWPD) and SVWPD. A joint multichannel 2-D-JSR (JMC-2-D-JSR) ISAR imaging algorithm is also proposed according to the SFB-SA signal model to produce high-resolution ISAR images. Underpinned by the Bayesian compressive sensing (BCS) theory, the JMC-2-D-JSR ISAR imaging can be realized by solving a sparsity-driven optimization problem via a modified quasi-Newton solver. Through iterative processing of JMC-2-D-JSR and JWPDC, the high-quality 3-D InISAR images of maneuvering targets with 3-D-RM can be obtained. Extensive experimental results based on both simulated and real data corroborate the effectiveness of the proposed algorithm that outperforms other available InISAR imaging frameworks in 2-D imaging, 3-D imaging, and motion compensation.

This article proposes a method to detect vocal signals and the corresponding locations of multiple humans using a multiple-input and multiple-output (MIMO) frequency-modulated continuous-wave (FMCW) radar system. The signals of multiple humans are extracted with digital beamforming by the virtual array generated by MIMO radar processing. To remove body motion artifacts that distort the speech information detected by the radar, a body motion effect cancellation method is proposed. The technique allows recovering vocal signals from detected distorted signals by estimating phase variations due to body motions. A combination of the proposed vocal signal detection algorithm with a human localization one enables to detect speaking humans by means of the MIMO radar. Experiments verify that the proposed method can accurately locate and detect speaking humans based on their vocal vibration signals.

To make a better discrimination between target and false alarm, amplitude information (AI) of radar signal has been incorporated into many target tracking algorithms, such as probabilistic data association, multiple hypothesis tracking and probability density hypothesis (PHD). In this paper, we present the cubature integration based Gaussian mixture implementation of the PHD filter with AI, namely the GMPHD-AI filter. Since the utilization of multiple sensors is more effective than using only one sensor, we also extend the GMPHD-AI filter to the multi-sensor application where the iterated-corrector scheme is exploited for multi-sensor information fusion. The effectiveness of the presented method is demonstrated by a multiple targets tracking scenario.

Multichannel tracking radars with phased antenna arrays are widely used to track air targets. The use of a phased array in combination with digital computing technology allows to control the radar radiation pattern and track several targets in the time distribution mode. Air target tracking in a multichannel radar is provided by subsystems for measuring range, radial velocity and angular coordinates, in most cases, without adaptation to the external influence characteristics. When tracking supermaneuverable air targets, such as 5th and so-called 4++ generation fighters, there is a decrease in the accuracy and stability of tracking relative to the area without maneuver. If the tracking system algorithms are tuned to a low intensity of maneuvering or its absence, a significant increase in the error of tracking the aircraft in the maneuvering section will lead to disruption of auto tracking due to a significant dynamic component of the error. The stability of auto-tracking of maneuvering targets by subsystems of range, radial velocity, and angular coordinates with fixed parameters for the case when the setting of the parameters of the tracking system algorithms coincide with the characteristics of the external influence is analyzes in the paper. The influence of the observation model parameters, the stochastic model of the target movement with exponentially correlated values of the target acceleration, and the measurement period of the target coordinates on the potential tracking accuracy by radio technical tracking systems of the multichannel radar is investigated. To assess the stability of auto-tracking, it is proposed to use the equivalent aperture size of the discriminating characteristic. The influence of the parameters of the target movement stochastic model, the observation model, and the measurement period of the target coordinates on the stability of auto-tracking in terms of range, radial velocity, and angular coordinates is estimated. It is shown that the "weak link" is the radial velocity tracking system. As a result of the research carried out, it becomes possible to further assess the feasibility of adapting the auto-tracking systems to the target maneuvering characteristics and to develop recommendations for choosing the measurement period of the target coordinates.

The synthetic aperture radar (SAR) image of ground moving target is easily defocused given the unknown relative motions between radar and target. In addition, the limitation of pulse repetition frequency for SAR easily induces the azimuth ambiguity (including Doppler center blur and spectrum ambiguity) of target signals, which causes the difficulty of moving target imaging. In this paper, a new efficient method for imaging and motion parameter estimation of ground moving targets with azimuth ambiguity is presented. Firstly, the time reversal process-2D scaled Fourier transform (TRP-2DSCFT) is developed to eliminate the effects of cross-track velocity and estimate the along-track velocity of target, simultaneously. Secondly, an operation based on discrete polynomial-phase transform and phase compensation function is proposed to estimate the cross-track velocity of target. Finally, a matched filter function based on estimated parameters is constructed to focus the moving targets. The well-focused result can be obtained by the presented method without the residual compensation errors. Moreover, the proposed method is computationally efficient given that the exhaustive searching steps are avoided. Additionally, the azimuth ambiguity can be effectively removed without the ambiguity number searching and pre-processing based on the prior information. The cross-terms for multiple target processing are analyzed. The effectiveness of the method is verified by both spaceborne and airborne real data results.

Emotion recognition has a significant impact on people’s health and life quality. Recent studies have shown that emotion recognition can be achieved by analyzing audio-visual emotion channels and physiological signals. However, the main challenges are: 1. the audio-visual techniques can be easily fooled by fake facial expressions; 2. the contact physiological signal monitoring introduces additional stress and is usually unsuitable for long-term monitoring; 3. The non-contact physiological signal monitoring methods are easily affected by complex environmental conditions. In this paper, a non-contact dual-modality emotion recognition system is proposed. First, the respiratory and heartbeat signals are measured by radar and camera simultaneously. Then, a hybrid signal optimization approach is proposed to remove the influence of body motion and light conditions on the physiological signal. It includes a light-intensity-based scheme selecting proper heartbeat signal for different light conditions, and an optical-flow-based algorithm pruning signals with significant radial body motion. Finally, the features extracted from the optimized physiological signals are fused to train an emotion recognition system. As shown in the experimental results, the proposed system could achieve high classification accuracy of 89.6% for 10-fold cross-validation at sample level, and 71.0 % for cross-validation at subject level. The respiratory and heartbeat signals by non-contact approaches are demonstrated to be reliable in emotion recognition.

Achieving general 3D motion estimation for all-visibility has been a key challenge in robotics, especially in extreme environments. The widely adopted camera and LiDAR-based motion estimation critically deteriorate under fog or smoke. In this letter, we devised a unique sensor system for 3D velocity estimation by compositing two orthogonal radar sensors. The proposed configuration allowed securing returns from the static objects on the ground. This work aimed at a realistic sensor deployment in a harsh environment by casing the bare sensor rig into a plastic box. As will be shown, the proposed velocity-based ego-motion estimation presented reliable performance over existing point matching-based methods, which degraded when measurement attenuates due to the casing. Furthermore, we introduce a novel radar instant velocity factor for pose-graph simultaneous localization and mapping (SLAM) framework and solve for 3D ego-motion in the integration with IMU. The validation reveals that the proposed method can be applied to estimate general 3D motion in both indoor and outdoor, targetting various visibility and the structureness in the environment.

To perform radar-based simultaneous localization and mapping, information on a robot’s ego-motion such as its rotation angle and velocity should be considered along with radar sensor data. In this paper, we propose a method to estimate the robot’s ego-motion using only a radar sensor without any other devices. To estimate the rotation angle of a robot, we use the distribution of detected points on a two-dimensional (2D) plane. The distributions of the detected points at successive time instants are correlated with each other, and the rotation angle can be estimated based on this correlation. In addition, the moving velocity of the robot is estimated from the trend line formed by the detected points on the 2D plane. To evaluate the performance of the proposed estimation method, the information about the robot’s ego-motion received from the robot motor is compared with the information estimated from the radar sensor data. The comparison of the yaw rate obtained through each method shows that our proposed method can estimate the rotation angle of the robot within an error of 3°. In addition, the ego-velocity of the radar-equipped robot can be estimated within an error of 0.073 m/s.

In this article, we review and compare different methods for the localization of an illuminator of opportunity for passive radar systems employing cooperative targets. For assessment and comparison, we use numerical simulations and real-world radar measurements. To achieve reliable operation under real-world conditions, a correct and robust association of the bistatic radar target observations with ground truth information is necessary. We introduce a novel solution to this issue with an efficient preprocessing algorithm based on a planar approximation.

Ghost targets caused by inter-reflections are by design unavoidable in radar measurements, and it is challenging to distinguish these artifact detections from real ones. In this letter, we propose a novel approach to detect radar ghost targets by using LiDAR data as a reference. For this, we adopt a multimodal transformer network to learn interactions between points. We employ self-attention to exchange information between radar points, and local crossmodal attention to infuse information from surrounding LiDAR points. The key idea is that a ghost target should have higher semantic affinity with the reflected real target than the other ones. Extensive experiments on nuScenes [1] show that our method outperforms the baseline method on radar ghost target detection by a large margin.

Forward-looking correlated imaging plays an increasingly important role in modern radar imaging systems. It overcomes disadvantages of traditional side or squint synthetic aperture radar (SAR) which is dependent on specific relative motion between the radar and target scene. A new microwave forward-looking correlated 3-D imaging method based on random radiation field combined with sparse reconstruction is proposed in this article. Firstly, phased array radar (PAR) is adopted to form different and random antenna patterns. Then, combined with the compressed sensing (CS) theory, the target image can be recovered with very few samples which can break through Rayleigh resolution limitation. Furthermore, the proposed method can achieve resolution at least 5.5 times higher than real aperture imaging. To raise computation efficiency of sparse reconstruction, an improved quasi-Newton iteration method based on graphics processing unit (GPU) platform is developed. Meanwhile, a GPU-based (NVIDIA Tesla K40c) accelerated computing method can significantly reduce the processing time compared with the time given by a personal computer (PC). Both simulation and field experiment verify the validity of the proposed method.

Environmental perception is a key technology for autonomous driving. Owing to the limitations of a single sensor, multiple sensors are often used in practical applications. However, multi-sensor fusion faces some problems, such as the choice of sensors and fusion methods. To solve these issues, we proposed a machine learning-based fusion sensing system that uses a camera and radar, and that can be used in intelligent vehicles. First, the object detection algorithm is used to detect the image obtained by the camera; in sequence, the radar data is preprocessed, coordinate transformation is performed, and a multi-layer perceptron model for correlating the camera detection results with the radar data is proposed. The proposed fusion sensing system was verified by comparative experiments in a real-world environment. The experimental results show that the system can effectively integrate camera and radar data results, and obtain accurate and comprehensive object information in front of intelligent vehicles.

Purpose This paper aims to develop a robust person tracking method for human following robots. The tracking system adopts the multimodal fusion results of millimeter wave (MMW) radars and monocular cameras for perception. A prototype of human following robot is developed and evaluated by using the proposed tracking system. Design/methodology/approach Limited by angular resolution, point clouds from MMW radars are too sparse to form features for human detection. Monocular cameras can provide semantic information for objects in view, but cannot provide spatial locations. Considering the complementarity of the two sensors, a sensor fusion algorithm based on multimodal data combination is proposed to identify and localize the target person under challenging conditions. In addition, a closed-loop controller is designed for the robot to follow the target person with expected distance. Findings A series of experiments under different circumstances are carried out to validate the fusion-based tracking method. Experimental results show that the average tracking errors are around 0.1 m. It is also found that the robot can handle different situations and overcome short-term interference, continually track and follow the target person. Originality/value This paper proposed a robust tracking system with the fusion of MMW radars and cameras. Interference such as occlusion and overlapping are well handled with the help of the velocity information from the radars. Compared to other state-of-the-art plans, the sensor fusion method is cost-effective and requires no additional tags with people. Its stable performance shows good application prospects in human following robots.

We investigate indoor human multi-target tracking in cartesian coordinates based on range, Doppler and Angle-of-Arrival measurements obtained with a four-antenna passive bistatic radar capturing 802.11ax Wi-Fi signals. A reference antenna selection method is described to perform angle processing correctly when dealing with target detection diversity among antennas. The tracking is performed by an Unscented Kalman Filter (UKF) to handle the non-linear relation between the measurement space and the state space. A Joint Probabilistic Data Association Filter is coupled to the UKF to handle the data association between tracks and measurements when dealing with multiple targets. Simulations are performed to determine the tracking parameters under heavy constraints and identify key scenarios. An experimental setup is built using Universal Software Radio Peripherals, featuring an over-the-air phase calibration for angle processing with an anchor antenna. It is used to validate the proposed single and multi-target tracking scheme., info:eu-repo/semantics/published

Developments in sensing technology have shown that UWB radars are becoming increasingly valuable sensing devices that can be used for monitoring of humans in military/police and civilian areas. It is known that the applicability of particular methods of human localization depends on the character of persons’ motion. With respect to this finding, the researchers’ attention has been aimed at two fundamental directions. While the former is focused on the localization of moving persons (MP), the latter approach is intended to localize static persons (SP). Then, a proper fusion of the methods developed for MP and SP localization allows monitoring of persons moving with an unknown time-variable character of motion (MP-SP). The analyses of the currently known methods of MP and SP localization in terms of their use for MP-SP localization have shown that while MP localization methods are in principle well developed, SP localization methods are not sufficiently adapted for their use in MP-SP monitoring. Motivated by these findings, we would like to introduce a new radar signal processing procedure for SP localization (SPL) that could be an efficient component of algorithms to be applied for MP-SP monitoring. The novel features of the proposed SPL consist especially in a new approach to SP detection and inclusion of SP tracking in SPL. Moreover, SPL is characterized by relatively low computational complexity and is, therefore, suitable for real-time implementation. Experimental results have shown that SPL introduced in this paper provides very good performance for multiple person localization for line-of-sight and through-the-wall scenarios.

Radio frequency (RF) sensors such as radar are instrumental for continuous, contactless sensing of vital signs, especially heart rate (HR) and respiration rate (RR). However, decades of related research mainly focused on static subjects, because the motion artifacts from other body parts may easily overwhelm the weak reflections from vital signs. This paper marks a first step in enabling RF vital sign sensing under ambulant daily living conditions. Our solution is inspired by existing physiological research that revealed the correlation between vital signs and body movement. Specifically, we propose to combine direct RF sensing for static instances and indirect vital sign prediction based on movement power estimation. We design customized machine learning models to capture the sophisticated correlation between RF signal pattern, movement power, and vital signs. We further design an instant calibration and adaptive training scheme to enable cross-subjects generalization, without any explicit data labeling from unknown subjects. We prototype and evaluate the framework using a commodity radar sensor. Under a variety of moving conditions, our solution demonstrates an average estimation error of 5.57 bpm for HR and 3.32 bpm for RR across multiple subjects, which largely outperforms state-of-the-art systems.

Two-dimensional pulse-to-pulse canceller (TDPC) of ground clutter can effectively suppress the clutter along the clutter trace, and therefore the moving target detectability of the following space-time adaptive processing (STAP) algorithm can be improved after TDPC as the clutter prefilter. However, TDPC may greatly impair the energy of moving target when inaccurate knowledge is exploited, which is detrimental to target detection. Aiming at this problem, a robust two-dimensional pulse-to-pulse canceller (RTDPC) of ground clutter is proposed. In order to enhance the TDPC’s robustness with inaccurate radar system parameters, which are mainly the platform velocity and crab angle, the errors of estimated platform velocity and crab angle are taken as the prior knowledge and added into the design of the clutter filter coefficient matrix. By exploiting RTDPC as the clutter prefilter, the moving target detectability of the following nonadaptive detection algorithm or STAP algorithm can also be enhanced. The simulated and MCARM data are utilized to verify the clutter suppression performance of RTDPC with inaccurate platform velocity and crab angle.