Search

With the increasing accumulation of spatial-temporal trajectory data, location-based data mining has recently been extensively studied. A fundamental research topic in this field is learning the embedding vectors of locations through self-supervised pre-training. Pre-trained embedding vectors can utilize the highly available unlabeled trajectory data, and benefit downstream tasks in multiple aspects. However, most existing methods ignore the temporal information hidden in the visited time of locations in trajectories. Considering that human activities are highly regulated by specific periods of a day, temporal information can reflect some intrinsic characteristics of locations, so it is necessary to fuse them into location embedding vectors. In this paper, we propose a Time-Aware Location Embedding (TALE) pre-training method based on the CBOW framework, which is able to incorporate temporal information into the learned embedding vectors of locations. A novel temporal tree structure is designed to extract temporal information during the calculation of Hierarchical Softmax. In order to verify the effectiveness of TALE, we apply the learned embedding vectors into three downstream location-based prediction tasks, i.e., location classification, location visitor flow prediction and user next location prediction. Experiments are conducted on four real-world user trajectory datasets, and the experimental results demonstrate that our TALE model can obviously help downstream tasks gain better performance. [ABSTRACT FROM AUTHOR]

Distantly supervised relation extraction aims to obtain relational facts from unstructured texts. Although distant supervision can automatically generate labeled training instances, it inevitably suffers from the wrong-label problem. Most of the current work is based on the bag-level for solving the noise problem, where a bag is composed of multiple sentences containing mentions of the same entity pair. However, previous studies mostly represent sentences from a single perspective, wherein insufficient modeling of global information restricts the effectiveness of denoising. In this study, we propose a novel distantly supervised relation extraction approach that incorporates the global contextual information of sentences to guide the denoising process and generate an effective bag-level representation. Simultaneously, knowledge-aware word embeddings were generated to enrich sentence-level representations by introducing both structured knowledge from external knowledge graphs and semantic knowledge from the corpus. The experimental results demonstrate that our proposed approach outperforms state-of-the-art methods on both versions of the large-scale benchmark New York Times dataset. In addition, the differences between the two versions of the dataset were investigated through further comparative experiments. [ABSTRACT FROM AUTHOR]

Accurate traffic forecasting is critical in improving safety, stability, and efficiency of intelligent transportation systems. Despite years of studies, accurate traffic prediction still faces the following challenges, including modeling the dynamics of traffic data along both temporal and spatial dimensions, and capturing the periodicity and the spatial heterogeneity of traffic data, and the problem is more difficult for long-term forecast. In this paper, we propose an Attention based Spatial-Temporal Graph Neural Network (ASTGNN) for traffic forecasting. Specifically, in the temporal dimension, we design a novel self-attention mechanism that is capable of utilizing the local context, which is specialized for numerical sequence representation transformation. It enables our prediction model to capture the temporal dynamics of traffic data and to enjoy global receptive fields that is beneficial for long-term forecast. In the spatial dimension, we develop a dynamic graph convolution module, employing self-attention to capture the spatial correlations in a dynamic manner. Furthermore, we explicitly model the periodicity and capture the spatial heterogeneity through embedding modules. Experiments on five real-world traffic flow datasets demonstrate that ASTGNN outperforms the state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Correlated time series refer to multiple time series which are recorded simultaneously to monitor the changing of multiple observations in a whole system. Correlated time series prediction plays a significant role in many real-world applications to help people make reasonable decisions. Yet it is very challenging, because different from single time series, correlated time series show both intra-sequence temporal dependencies and inter-sequence spatial dependencies. In addition, correlated time series are also affected by external factors in actual scenarios. Although RNNs have been proved to be effective on sequential data modeling, existing related works only focus on sequential patterns in a single time series, failing to comprehensively consider the inter-dependencies among multiple time series, which is essential for correlated time series prediction. In this paper, we propose a novel variant of LSTM, named CTS-LSTM, to collectively forecast correlated time series. Specifically, spatial and temporal correlations are explicitly modeled and respectively maintained in cells to capture the complex non-linear patterns in correlated time series. A general interface for handling external factors is further designed to enhance forecasting performance of the model. Experiments are conducted on two types of real-world datasets, viz., civil aviation passenger demand data and air quality data. And our CTS-LSTM achieves at least 9.0%, 16.5% and 21.3% lower RMSE, MAE and MAPE compared to the state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Time series forecasting is ubiquitous in various scientific and industrial domains. Powered by recurrent and convolutional and self-attention mechanism, deep learning exhibits high efficacy in time series forecasting. However, the existing forecasting methods are suffering some limitations. For example, recurrent neural networks are limited by the gradient vanishing problem, convolutional neural networks cost more parameters, and self-attention has a defect in capturing local dependencies. What's more, they all rely on time invariant or stationary since they leverage parameter sharing by repeating a set of fixed architectures with fixed parameters over time or space. To address the above issues, in this paper we propose a novel time-variant framework named Self-Attention-based Time-Variant Neural Networks (SATVNN), generally capable of capturing dynamic changes of time series on different scales more accurately with its time-variant structure and consisting of self-attention blocks that seek to better capture the dynamic changes of recent data, with the help of Gaussian distribution, Laplace distribution and a novel Cauchy distribution, respectively. SATVNN obviously outperforms the classical time series prediction methods and the state-of-the-art deep learning models on lots of widely used real-world datasets. [ABSTRACT FROM AUTHOR]

Providing both point estimation and uncertainty quantification for traffic forecasting is crucial for supporting accurate and reliable services in intelligent transportation systems. However, the majority of existing traffic forecasting works mainly focus on point estimation without quantifying the uncertainty of predictions. Meanwhile, existing uncertainty quantification (UQ) methods fail to capture the inherent static characteristics of traffic uncertainty along both the spatial and temporal dimensions. Directly equipping the traffic forecasting works with uncertainty quantification techniques may even damage the prediction accuracy. In this paper, we propose a novel traffic forecasting model aiming at providing point estimation and uncertainty quantification simultaneously, called STUP. Compared to the traditional graph convolution networks (GCNs), our framework is able to incorporate uncertainty quantification into traffic forecasting to further improve forecasting performance. Specifically, we first develop an adaptive strategy to initialize uncertainty distribution. Then a kind of spatial–temporal uncertainty layer is carefully designed to model the evolution process of both the traffic state and its corresponding uncertainty, along with a gated adjusting unit to avoid error information propagation. Finally, we propose a novel constraint loss to further help improve the forecasting accuracy and to alleviate the training difficulty caused by the lack of uncertainty labels. Experiments on five real-world traffic datasets demonstrate that STUP outperforms the state-of-the-art baselines on both the traffic prediction task and uncertainty quantification task. [Display omitted] • A spatial–temporal traffic forecasting method considering uncertainty is proposed. • Discovering the relationship between traffic data and factors of uncertainty. • Balancing the accuracy and reliability on traffic forecasting. [ABSTRACT FROM AUTHOR]

In intelligent transportation systems (ITS), deploying fine-grained sensors to continuously collect spatial–temporal traffic data is important but impractical due to the expensive cost. Fortunately, spatial–temporal kriging methods bring advanced solutions for interpolating traffic data for locations without sensors, but they still have the following two drawbacks: (1) The widely adopted predefined and adaptive graphs are either inflexible or limited to transductive learning. (2) The sampling strategies to support inductive learning on new graphs result in losing important partial information. To overcome the above issues, in this paper, we propose an Inductive and Adaptive Graph Convolution Networks (IAGCN) for spatial–temporal traffic data kriging in an inductive manner. Specifically, we propose an adaptive graph constructor to model the hidden spatial relation of nodes and learn the spatial dependency that the predefined graph structure cannot capture. It can work inductively and does not require retraining when introducing new nodes. Additionally, we design a framework that integrates inductive graph convolution and temporal convolution to simultaneously capture the complex spatial–temporal dependencies of traffic data. Finally, to address the information loss issue caused by random sampling, we design a predicted-based constraint task that perceives and utilizes the history information of all observed sensors to predict the current data, as well as approach the result of the interpolation and prediction. Experiments on four real-world datasets show that IAGCN outperforms the state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Accurate forecasting of time series mitigates the uncertainty of future outlooks and is a great help in reducing errors in decisions. Despite years of researches, there are still some challenges to accurate forecasting of time series, including the difficulty of dynamic modeling, the problem of capturing short-term correlations, and the conundrum of long-term forecasting. This paper offers an A dversarial Truncated Cauchy S elf- A ttentive T ime- V ariant Neural N etwork (ASATVN) for multi-step ahead time series forecasting. Specifically, the proposed model builds on Generative Adversarial Networks, in which the generator is composed of a novel time-variant model. The time-variant model contributes to learning dynamic time-series changes with its time-variant architecture and employs a newly proposed Truncated Cauchy Self-Attention block to capture the local sequential dependencies better. For the discriminator, two self-attentive discriminators are presented to regularize predictions with fidelity and continuity, which is beneficial to predicting sequence over longer time horizons. Our proposed ASATVN model outperforms the state-of-the-art predictive models on eleven real-world benchmark datasets, demonstrating its effectiveness. • A time-variant network learns various dynamics across multiple time steps. • A new self-attention are more sensitive to the local context of time series. • Two discriminators regularize predictor to offer realistic and continuous forecasts. [ABSTRACT FROM AUTHOR]