Search

This paper presents a hierarchical three-layer trajectory planning framework to realize real-time collision avoidance under complex driving conditions. This is mainly ascribed to the generation of a collision-free trajectory cluster based on the speed and the path re-planning. The upper-layer controller is to generate a reference quintic polynomial trajectory based on the Sequential Quadratic Programming by assuming mild speed and acceleration variations of the surrounding vehicles. The waypoints and time stamps can be obtained via the reference trajectory. When the assumption is invalid under complex driving conditions, the middle-layer controller would generate a Quadratic Programming-based trajectory cluster to assign different time stamps to each waypoint through time-based sampling methods. The lower-layer controller would be triggered to create a new feasible trajectory based on the path sampling if the collision avoidance requirements are not satisfied. The host vehicle will return to its original lane if no feasible time window is available to perform a lane change maneuver under the vehicle kinematics and lane change time/displacement constraints. The effectiveness of the proposed scheme is verified under various scenarios through comprehensive simulations.

Automated vehicles require efficient and safe planning to maneuver in uncertain environments. Largely this uncertainty is caused by other traffic participants, e.g., surrounding vehicles. Future motion of surrounding vehicles is often difficult to predict. Whereas robust control approaches achieve safe, yet conservative motion planning for automated vehicles, Stochastic Model Predictive Control (SMPC) provides efficient planning in the presence of uncertainty. Probabilistic constraints are applied to ensure that the maximal risk remains below a predefined level. However, safety cannot be ensured as probabilistic constraints may be violated, which is not acceptable for automated vehicles. Here, we propose an efficient trajectory planning framework with safety guarantees for automated vehicles. SMPC is applied to obtain efficient vehicle trajectories for a finite horizon. Based on the first optimized SMPC input, a guaranteed safe backup trajectory is planned using reachable sets. This backup is used to overwrite the SMPC input if necessary for safety. Recursive feasibility of the safe SMPC algorithm is proved. Highway simulations show the effectiveness of the proposed method regarding performance and safety.

Trajectory or path planning is a fundamental issue in a wide variety of applications. In this article, we show that it is possible to solve path planning on a maze for multiple start point and endpoint highly efficiently with a novel configuration of multilayer networks that use only weighted pooling operations, for which no network training is needed. These networks create solutions, which are identical to those from classical algorithms such as breadth-first search (BFS), Dijkstra's algorithm, or TD(0). Different from competing approaches, very large mazes containing almost one billion nodes with dense obstacle configuration and several thousand importance-weighted path endpoints can this way be solved quickly in a single pass on parallel hardware.

When viewing omnidirectional images (ODIs), viewers can access different viewports via head movement (HM), which sequentially forms head trajectories in spatial-temporal domain. Thus, head trajectories play a key role in modeling human attention on ODIs. In this paper, we establish a large-scale dataset collecting 21,600 head trajectories on 1,080 ODIs. By mining our dataset, we find two important factors influencing head trajectories, i.e., temporal dependency and subject-specific variance. Accordingly, we propose a novel approach integrating hierarchical Bayesian inference into long short-term memory (LSTM) network for head trajectory prediction on ODIs, which is called HiBayes-LSTM. In HiBayes-LSTM, we develop a mechanism of Future Intention Estimation (FIE), which captures the temporal correlations from previous, current and estimated future information, for predicting viewport transition. Additionally, a training scheme called Hierarchical Bayesian inference (HBI) is developed for modeling inter-subject uncertainty in HiBayes-LSTM. For HBI, we introduce a joint Gaussian distribution in a hierarchy, to approximate the posterior distribution over network weights. By sampling subject-specific weights from the approximated posterior distribution, our HiBayes-LSTM approach can yield diverse viewport transition among different subjects and obtain multiple head trajectories. Extensive experiments validate that our HiBayes-LSTM approach significantly outperforms 9 state-of-the-art approaches for trajectory prediction on ODIs, and then it is successfully applied to predict saliency on ODIs.

Reinforcement learning (RL) is a promising technique for designing a model-free controller by interacting with the environment. Several researchers have applied RL to autonomous underwater vehicles (AUVs) for motion control, such as trajectory tracking. However, the existing RL-based controller usually assumes that the unknown AUV dynamics keep invariant during the operation period, limiting its further application in the complex underwater environment. In this article, a novel meta-RL-based control scheme is proposed for trajectory tracking control of AUV in the presence of unknown and time-varying dynamics. To this end, we divide the tracking task for AUV with time-varying dynamics into multiple specific tasks with fixed time-varying dynamics, to which we apply meta-RL for training to distill the general control policy. The obtained control policy can transfer to the testing phase with high adaptability. Inspired by the line-of-sight (LOS) tracking rule, we formulate each specific task as a Markov decision process (MDP) with a well-designed state and reward function. Furthermore, a novel policy network with an attention module is proposed to extract the hidden information of AUV dynamics. The simulation environment with time-varying dynamics is established, and the simulation results reveal the effectiveness of our proposed method.

The Global Positioning System (GPS) has become a foundation for most location-based services and navigation systems, such as autonomous vehicles, drones, ships, and wearable devices. However, it is a challenge to verify if the reported geographic locations are valid due to various GPS spoofing tools. Pervasive tools, such as Fake GPS, Lockito, and software-defined radio, enable ordinary users to hijack and report fake GPS coordinates and cheat the monitoring server without being detected. Furthermore, it is also a challenge to get accurate sensor readings on mobile devices because of the high noise level introduced by commercial motion sensors. To this end, we propose DeepPOSE, a deep learning model, to address the noise introduced in sensor readings and detect GPS spoofing attacks on mobile platforms. Our design uses a convolutional and recurrent neural network to reduce the noise, to recover a vehicle's real-time trajectory from multiple sensor inputs. We further propose a novel scheme to map the constructed trajectory from sensor readings onto the Google map, to smartly eliminate the accumulation of errors on the trajectory estimation. The reconstructed trajectory from sensors is then used to detect the GPS spoofing attack. Compared with the existing method, the proposed approach demonstrates a significantly higher degree of accuracy for detecting GPS spoofing attacks.

Mobile robots are expected to move safely and efficiently in socially-compliant ways in dense crowds. In this paper, a unified multiple-motion-mode framework is proposed to achieve socially-compliant navigation in dense crowds. The proposed framework consists of three successive phases, the identification of pedestrian groups and prediction of their trajectories, the generation of candidate local trajectories of the robot, and the determination of an optimal local trajectory. The pedestrians are grouped according to distances among pedestrians and their velocities and a predictor predicts the groups' trajectories. Three robot motion modes, moving-solo, pedestrian-following, and courteous-stopping, are proposed based on a systematic analysis and classification of dense crowds and they are utilized to generate candidate local trajectories. A composite metric is proposed to determine the best local trajectory by simultaneously considering the robot motion safety, efficiency, and stability of the trajectory. The proposed robot navigation framework is evaluated both in simulations and real-world experiments with various scenarios. The results show that the proposed framework surpasses the state-of-the-art methods in terms of efficiency and social compliance.

In this paper, an adaptive admittance control scheme is developed for robots to interact with time-varying environments. Admittance control is adopted to achieve a compliant physical robot-environment interaction, and the uncertain environment with time-varying dynamics is defined as a linear system. A critic learning method is used to obtain the desired admittance parameters based on the cost function composed of interaction force and trajectory tracking without the knowledge of the environmental dynamics. To deal with dynamic uncertainties in the control system, a neural-network (NN)-based adaptive controller with a dynamic learning framework is developed to guarantee the trajectory tracking performance. Experiments are conducted and the results have verified the effectiveness of the proposed method.

The ability of robots to write Chinese strokes, which is recognized as a sophisticated task, involves complicated kinematic control algorithms. The conventional approaches for robotic writing of Chinese strokes often suffer from limited font generation methods, which limits the ability of robots to perform high-quality writing. This paper instead proposes a developmental evolutionary learning framework that enables a robot to learn to write fundamental Chinese strokes. The framework first considers the learning process of robotic writing as an evolutionary easy-to-difficult procedure. Then, a developmental learning mechanism called “Lift-constraint, act and saturate” that stems from developmental robotics is used to determine how the robot learns tasks ranging from simple to difficult by building on the learning results from the easy tasks. The developmental constraints, which include altitude adjustments, number of mutation points, and stroke trajectory points, determine the learning complexity of robot writing. The developmental algorithm divides the evolutionary procedure into three developmental learning stages. In each stage, the stroke trajectory points gradually increase, while the number of mutation points and adjustment altitudes gradually decrease, allowing the learning difficulties involved in these three stages to be categorized as easy, medium, and difficult. Our robot starts with an easy learning task and then gradually progresses to the medium and difficult tasks. Under various developmental constraint setups in each stage, the robot applies an evolutionary algorithm to handle the basic shapes of the Chinese strokes and eventually acquires the ability to write with good quality. The experimental results demonstrate that the proposed framework allows a calligraphic robot to gradually learn to write five fundamental Chinese strokes and also reveal a developmental pattern similar to that of humans. Compared to an evolutionary algorithm without the developmental mechanism, the proposed framework achieves good writing quality more rapidly.

Learning to drive in urban areas is an open challenge for autonomous vehicles (AVs), as complex decision making requirements are needed in multi-task co-ordinations environments. In this paper, we propose a hybrid framework with a new perception model involving affordance learning to simplify the surrounding urban scenes for training an AV agent, along with a planned trajectory and the associated driving measurements. Our proposed solution encompasses two main aspects. Firstly, a supervised learning network is used to map the input sensory data into affordance predictions. The predicted affordances provide a low-dimensional representation of surrounding scenes of the AV in the form of key perception indicators, e.g., true or false with respect to a traffic light signal. Secondly, a deep deterministic policy gradient model that maps the perception information into a series of actions is devised. We evaluate the proposed solution using the CARLA driving simulator in an urban town and evaluate the performance in a new, unseen town under different weather conditions. The quantitative and qualitative results indicate that our proposed solution can generalize well to cope with different traffic solutions and environmental conditions. Our proposed solution also outperforms other baseline methods in a comparative study in handling various AV driving tasks with different levels of difficulty. In addition, the model trained with simulated scenes yields promising prediction results when testing on recorded video streams on real-world highway and suburb environments with varying traffic and weather conditions.

Visual surveillance has become indispensable in the evolution of Intelligent Transportation Systems (ITS). Video object trajectories are key to many of the visual surveillance applications. Classifying varying length time series data such as video object trajectories using conventional neural networks, can be challenging. In this paper, we propose trajectory classification and anomaly detection using a hybrid Convolutional Neural Network (CNN) and Variational Autoencoder (VAE) architecture. First, we introduce a high level features for varying length object trajectories using color gradient representation. In the next stage, a semi-supervised way to annotate moving object trajectories extracted using Temporally Incremental Gravitational Model (TIGM) is used for class labeling. For training, anomalous trajectories are identified using t-Distributed Stochastic Neighbor Embedding (t-SNE). Finally, a hybrid CNN-VAE architecture has been proposed for trajectory classification and anomaly detection. The results obtained using publicly available surveillance video datasets reveal that the proposed method can successfully identify traffic anomalies such as violations in lane driving, sudden speed variations, abrupt termination of vehicle during movement, and vehicles moving in wrong directions. The accuracy of trajectory classification improves by a margin of 1-6% against popular neural networks-based classifiers across various datasets using the proposed high-level features. The gradient representation also improves the anomaly detection accuracy significantly (30-35%). Code and dataset can be found at https://github.com/santhoshkelathodi/CNN-VAE.

Trajectory prediction for mobile phone users is a cornerstone component to support many higher-level applications in LBSs (Location-Based Services). Most existing methods are designed based on the assumption that the explicit location information of the trajectories is available (e.g., GPS trajectories). However, collecting such kind of trajectories lays a heavy burden on the mobile phones and incurs privacy concerns. In this paper, we study the problem of trajectory prediction based on cell-id trajectories without explicit location information and propose a deep learning framework (called DeepCTP) to solve this problem. Specifically, we use a multi-graph embedding method to learn the latent spatial correlations between cell towers by exploiting handoff patterns. Then, we design a novel spatial-aware loss function for the encoder-decoder network to generate cell-id trajectory predictions. We conducted extensive experiments on real datasets. The experiment results show that DeepCTP outperforms the state-of-the-art cell-id trajectory prediction methods in terms of prediction error.

Microscopic modeling of driving behavior is the basis for traffic design and traffic simulation studies and can be applied to automated driving systems to provide human-like decision making. Previous modeling methods can be mainly divided into scenario-based modeling methods and field theory-based modeling methods. Scenario-based models are based on behavior theories that can explain behavioral mechanisms and field theory-based models are convenient for application to different scenarios. Combining two behavior theories and field theory, this paper aims to present a novel method to uniformly model the driving behavior in different scenarios. Risk homeostasis theory and preview-follower theory are used as the theoretical foundation, and field theory is utilized to connect the two behavior theories. A new risk field model is constructed for better coupling these behavior theories. Integrating these theories, this study then develops a subjectively perceived risk quantification method and a trajectory and motion planning model, which are validated using naturalistic data in car-following scenarios. Results show the effectiveness of this method and this model with reference to an effective risk quantification index (safety margin) and in comparison with the classical models (desired safety margin model and intelligent driver model) using naturalistic data in car-following scenarios.

This paper addresses the fixed-time control problem for the constrained quarter active vehicle suspension systems (AVSSs) via an event-triggered-based adaptive fuzzy fixed-time control method. The benefit of the usage of the time-varying barrier Lyapunov function (BLF) is to avoid the violation of the time-varying displacement constraint so that the stability and safety of AVSSs can be guaranteed. The relative-thresholdbased event-triggered controller is devised so as to reduce the communication burden from the controller to the actuator. In the light of fixed time theory, it is proved that both the stability and tracking performance of the closed-loop system can be obtained in fixed time. The fixed-time based eventtriggered control strategy is independent of initial states of AVSSs in comparison with the existing finite-time results. Some simulation results and comparisons on a quarter-car AVSS indicate better performance in terms of feasible fixed-time control and exact trajectory tracking.

Accurate destination prediction over sub-trajectories is essential for a wide range of location-based services. Traditional trip matching methods fail to capture temporal dependence hidden in trajectories and may suffer from data sparsity problems. With the help of massive trajectory data, state-of-the-art approaches based on deep learning (DL) have achieved great success. However, existing DL approaches rarely consider the influence of individual travel preferences in destination prediction. When the trip is long but the known partial trajectory is short, DL models are unable to produce satisfactory results. Thus, we design a feature extraction mechanism to extract useful temporal features, spatial features, and static covariates for destination prediction, among which the spatial features characterize individual travel preferences by considering two main movement patterns in daily travel. Then, a hierarchical model including multiple modules is proposed to finely process heterogeneous features. Extensive experiments conducted on two public datasets demonstrate the superior performance of the proposed model compared to the state-of-the-art methods. Moreover, further experimental results show that the proposed model still performs well when trajectory prefix is short or travel duration is long, which confirms the effectiveness of integrating individual travel preferences.

In complex and dynamic urban traffic scenarios, the accurate prediction of trajectories of surrounding traffic participants (vehicles, pedestrians, etc) with interactive behaviours plays an important role in the navigation and the motion planning of the ego vehicle. In this paper, based on the graph neural network (GNN), we propose a hierarchical GNN framework to model interactions of heterogeneous traffic participants (vehicles, pedestrians and riders) combined with LSTM to predict their trajectories. The proposed framework consists of two modules with two GNNs for interactive events recognition (IER) and trajectory prediction (TP). The IER module is used to recognise interactive events between traffic participants and the ego vehicle. With the recognised results as the input, the TP module is built for interactive trajectory prediction. In addition, to realise the multi-step prediction, a long short-term memory network (LSTM) is combined with GNN in the TP module. The proposed hierarchical framework is verified by the naturalistic driving data collected from the urban traffic environment. Comparative results with state-of-the-art methods indicate that the hierarchical GNN framework obtains an outstanding performance in the recognition of interactive events and the prediction of interactive behaviours.

Researchers have proposed many novel methods to detect abnormal taxi trajectories. However, most of the existing methods usually adopt a counting-based strategy, which may cause high false positives due to imprecisely identifying diverse trajectories as anomalies and therefore, they need the support of large-scale historical trajectories to work properly. To improve detection precision and efficiency, in this article, we propose STR, an online abnormal taxi trajectory detection method based on spatio-temporal relations. The basic principle behind STR is that given the displacement from the source point to a testing point, if the driving time and driving distance are not within the normal ranges, the point is identified as anomalous. To learn the two normal ranges for driving time and driving distance, STR defines two spatio-temporal models which characterize the relationship between displacement and driving distance/driving time. To improve detection efficiency, STR reduces the number of models that need to be learned by making full use of the similarity of transportation modes in different time periods and neighboring areas. The effectiveness and performance of STR are evaluated on real-world taxi trajectories. The experiment results show that compared with counting-based methods, STR achieves greater precision by reducing false positives. Furthermore, STR is more efficient than its counterparts and is suitable for online detection.

Automatic Dynamic Facial Expression Recognition (DFER) is challenging since how to effectively capture facial temporal dynamics is still an open problem. As the variations of facial expressions is a dynamic system that satisfies underlying rules, it is essential to explore the fundamental temporal properties for recognizing dynamic expressions. Inspired by the phase space reconstruction method for time series analysis, we propose a Phase Space Reconstruction Network (PSRNet) for learning spatio-temporal facial features. First, 3D convolutional neural networks are used to extract spatial and short-term features, which indicate each frame's state termed as observations in the phase space. All the observations compose the trajectory of the dynamical system. Then, a data-driven across-correlation matrix is inferred to reveal the relationship of the observations. With this matrix, the phase space reconstruction module reconstructs the trajectory by aggregating the observations adaptively. Reconstructed observations represent the gradual process of dynamic facial expressions, which is beneficial to recognize these expressions. The experiment results on Oulu, MMI, and CK+ demonstrate that PSRNet can extract more informative and representative spatio-temporal features for DFER. Moreover, the visualization reveals that the reconstructed features have global consistency in facial regions and find the underlying evolutionary pattern of dynamic facial expression.

Since the past few decades, human trajectory forecasting has been a field of active research owing to its numerous real-world applications: evacuation situation analysis, deployment of intelligent transport systems, traffic operations, to name a few. Early works handcrafted this representation based on domain knowledge. However, social interactions in crowded environments are not only diverse but often subtle. Recently, deep learning methods have outperformed their handcrafted counterparts, as they learned about human-human interactions in a more generic data-driven fashion. In this work, we present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. To objectively compare the performance of these interaction-based forecasting models, we develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting. We propose novel performance metrics that evaluate the ability of a model to output socially acceptable trajectories. Experiments on TrajNet++ validate the need for our proposed metrics, and our method outperforms competitive baselines on both real-world and synthetic datasets., IEEE format, Layer-wise Relevance Propagation added

Multi-agent systems (MASs) can autonomously learn to solve previously unknown tasks by means of each agent's individual intelligence as well as by collaborating and exploiting collective intelligence. This article considers a group of autonomous agents learning to track the same given reference trajectory in a possibly small number of trials. We propose a novel collective learning control method that combines iterative learning control (ILC) with a collective update strategy. We derive conditions for desirable convergence properties of such systems. We show that the proposed method allows the collective to combine the advantages of the agents' individual learning strategies and thereby overcomes trade-offs and limitations of single-agent ILC. This benefit is achieved by designing a heterogeneous collective, i.e., a different learning law is assigned to each agent. All theoretical results are confirmed in simulations and experiments with two-wheeled-inverted-pendulum robots (TWIPRs) that jointly learn to perform the desired maneuver.

A unified approach is proposed to design sampled-data observers for a certain type of unknown nonlinear systems undergoing recurrent motions based on deterministic learning in this article. First, a discrete-time implementation of high-gain observer (HGO) is utilized to obtain state trajectory from sampled output measurements. By taking the recurrent estimated trajectory as inputs to a dynamical radial basis function network (RBFN), a partial persistent exciting (PE) condition is satisfied, and a locally accurate approximation of nonlinear dynamics can be realized along the estimated sampled-data trajectory. Second, an RBFN-based observer consisting of the obtained dynamics from the process of deterministic learning is designed. Without resorting to high gains, the RBFN-based observer is shown capable of achieving correct state observation. The novelty of this article lies in that, by incorporating deterministic learning with the discrete-time HGO, the nonlinear dynamics can be accurately approximated along the estimated trajectory, and such obtained knowledge can then be utilized to realize nonhigh-gain state estimation for the same or similar sampled-data systems. Simulation is performed to validate the effectiveness of the proposed approach.

The predictive cruise control (PCC) is a promising method to optimize energy consumption of vehicles, especially the heavy-duty vehicles (HDV). Due to the limited sensing range and computational capabilities available on-board, the conventional PCC system can only obtain a sub-optimal speed trajectory based on a shorter prediction horizon. The recently emerging information and communication technologies such as vehicular communication, cloud computing, and Internet of Things provide huge potentials to improve the traditional PCC system. In this paper, we propose a general framework for the enhanced cloud-based PCC system which integrates a data-driven traffic predictive model and the instantaneous control algorithms. Specifically, we introduce a novel multi-view CNN deep learning algorithm to predict traffic situation based on the historical and real-time traffic data collected from fields, and the time-varying adaptive model predictive control (MPC) to calculate the instantaneous optimal speed profile with the aim of minimizing energy consumption. We verified our approach via simulations in which the impact of various traffic condition on the PCC-enabled HDV has been fully evaluated.

We present a new framework for online dense 3D reconstruction of indoor scenes by using only depth sequences. This research is particularly useful in cases with a poor light condition or in a nearly featureless indoor environment. The lack of RGB information makes long-range camera pose estimation difficult in a large indoor environment. The key idea of our research is to take advantage of the geometric prior of Manhattan scenes in each stage of the reconstruction pipeline with the specific aim to reduce the cumulative registration error and overall odometry drift in a long sequence. This idea is further boosted by local Manhattan frame growing and the local-to-global strategy that leads to implicit loop closure handling for a large indoor scene. Our proposed pipeline, namely ManhattanFusion, starts with planar alignment and local pose optimization where the Manhattan constraints are imposed to create detailed local segments. These segments preserve intrinsic scene geometry by minimizing the odometry drift even under complex and long trajectories. The final model is generated by integrating all local segments into a global volumetric representation under the constraint of Manhattan frame-based registration across segments. Our algorithm outperforms others that use depth data only in terms of both the mean distance error and the absolute trajectory error, and it is also very competitive compared with RGB-D based reconstruction algorithms. Moreover, our algorithm outperforms the state-of-the-art in terms of the surface area coverage by 10-40 percent, largely due to the usefulness and effectiveness of the Manhattan assumption through the reconstruction pipeline.

Modeling interactions among vehicles is critical in improving the efficiency and safety of autonomous driving since complex interactions are ubiquitous in many traffic scenarios. To model interactions under different traffic scenarios, most existing works consider interaction information implicitly in their specific tasks with hand-crafted features and pre-defined maneuvers. Extracting interaction representation which can be commonly used among different downstream tasks is not explored. In this paper, we propose a general and novel Graph Self-Attention Network (GSAN) to learn the spatial-temporal interaction representation among vehicles by a framework consisting of pre-training and fine-tuning. Specifically, in the pre-training step, we construct the GSAN module based on a graph self-attention layer and a gated recurrent unit layer and use trajectory auto-regression to learn the interaction information among vehicles. In the fine-tuning step, we propose two different adaptation schemes to utilize the learned interaction information in various downstream tasks and fine-tune the entire model with only a few steps. To illustrate the effectiveness and generality of our spatial-temporal interaction model, we conduct extensive experiments on two typical interaction-related tasks, namely, lane-changing classification and trajectory prediction. The experiment results demonstrate that our approach significantly outperforms the state-of-the-art solutions of these two tasks. We also visualize the impact of surrounding vehicles on the ego vehicle in different interaction scenes. The visualization offers an intuitive explanation on how our model captures the dynamic changing interactions among vehicles and makes good predictions in various interaction-related tasks.

To shorten the scanning time by focusing on the local scanning for such specimens as cells, this paper proposes an advanced scanning strategy based on autonomous exploration, so as to detect the specimen in real time and achieve fast imaging for an atomic force microscopy (AFM). More specifically, fast raster scanning is first performed to locate the initial boundary point of the specimen. On this basis, a boundary tracking algorithm is proposed to construct the internal boundary of the specimen online through the autonomous exploration of the probe. Afterwards, the boundary is expanded according to the internal boundary tracking direction, based on which a convex hull of the specimen is further constructed for the local scanning. Furthermore, the local slow scanning is performed for the specimen according to the generated scanning trajectory. Subsequently, unscanned areas in the sample can also be scanned by this autonomous method. Experimental results verify that the proposed method can achieve fast scanning while ensuring high-quality imaging.

As an important component in transportation maps, three-dimensional (3D) structure information of grade-separated junctions is crucial for applications such as intelligent driving, route planning and traffic control. In order to acquire spatial layouts of road junctions, researchers have developed algorithms to extract planar structures from various data sources. However, it is less common to refine maps of grade-separated junctions with 3D structure information using tracking data. The objective of this study is to find an approach to extracting 3D structures of grade-separated junctions from vehicle trajectories. The proposed method is based on semantic segmentation and data fusion. Trajectories were divided into sections with different trends of elevation by detecting change points. The ranges and elevations of slopes and level sections were derived by seeking consensus among different trajectories using a data fusion technique. Based on semantic segmentation and aggregated elevations, we reconstructed detailed 3D junction structures. This method was validated on multiple crowdsourced trajectory datasets and compared to cluster center linking method. Experiments show that the proposed method had a higher overall accuracy of semantic segmentation than baseline method. The accuracy of vertical relationship at intersections is comparable to baseline. Despite large elevation discrepancy among trajectories, the performance of the proposed method was similar across crowdsourced trajectory datasets from open and commercial projects.

Pedestrian motion trajectory prediction is an important task in intelligent driving, and it can provide a valuable reference for the subsequent path decision of intelligent driving. However, so far, there are only a few models in the field of specific pedestrian motion track prediction in intelligent driving from far shot first-person perspective video. To accomplish this task, we proposed a deep learning model for pedestrian motion trajectory prediction from far shot first-person perspective video with four key innovations: a) A macroscopic pedestrian trajectory prediction module is established under the close correlation between neighboring frames to estimate the pedestrian motion track on the whole; b) A relative motion transformation module of vehicle-mounted camera is designed to consider the effect of vehicle-mounted camera's ego-motion on the pedestrian motion track; c) We set up a circular training module to maintain the number of parameters in our model to simplify and reduce the size of model; d) A new far shot first-person pedestrian motion dataset under intelligent driving is specifically established to train and test the proposed model. The above four modules are integrated into the proposed deep learning model, which achieves state-of-the-art results for predicting pedestrian motion trajectory from both far and close shot first-person perspective video.