Search

Multivariate time series (MTS) forecasting is a research field that is gaining more and more importance as time series data generators proliferate in the growing era of Internet of Things. Deep learning architecture for MTS data has been and still a very active research area as there is no comprehensive comparative study of the different architectures, let alone a perfect architecture that can solve all types of time series modeling problem. In this paper, we start by highlighting time series analysis requirements, to propose a new model for conditional multivariate time series forecasting based on the lately introduced WaveNet architecture. Our model is composed of stacked residual causal dilated convolutions that provide large scope in time series history and foster learning of long-term dependencies. Parameterized skip connections allow catching early trends and properties while conditioning enables modeling the associations that exist between multivariate data. We used group normalization for its stability and independence from the batch size. We present a deep comparison of the structure, flexibility, flow control, memory consumption, robustness and stability capacities of our new model against those of recurrent neural networks state-of-the-art approaches—long-short-term-memory (LSTM) and gated recurrent unit (GRU). To assess the performances of our model, we conduct extensive experimental testing on the task of urban air quality prediction in Marrakesh city using six real-world multi-sensor and multivariate time series datasets. We compare the results of our model against the results of LSTM and GRU. Our experiments revealed that our proposed WaveNet-temporal-CNN outperform recurrent models ability to learn long-term dependencies in a time-efficient way. [ABSTRACT FROM AUTHOR]

One of the most groundbreaking structures in deep learning is ResNet, which utilizes skip connections to make neural networks (NNs) significantly deeper. While there are many successful applications of skip connections in computer vision (CV) and natural language processing (NLP), the applications of skip connections in electric load forecasting are still quite limited. Moreover, as compared to the deep NNs used in CV and NLP, most NNs used in load forecasting are relatively shallow NNs, whose performance is partially restricted by the relatively shallow structures. To improve probabilistic forecasting by deepening NN, we investigate the applicability of the skip connections in probabilistic load forecasting NN by proposing new structures and studying the philosophy behind the forecasting improvements brought by skip connections. Case studies show that by deepening the NNs, the proposed structures can generate more accurate probabilistic load forecasts than state-of-the-art methods, which implies that the ideas of skip connections have significant value in probabilistic load forecasting. It is also validated in observations and mathematical proofs that the proposed structures improve the probabilistic forecasting by alleviating the gradient vanishing and exploding problems in deep NNs. [ABSTRACT FROM AUTHOR]

In the early diagnosis of diabetic retinopathy, the morphological attributes of blood vessels play an essential role to construct a retinal computer-aided diagnosis system. However, due to the challenges including limited densely annotated data, inter-vessel differences and structured prediction problem, it remains challenging to segment accurately the retinal vessels, particularly the capillaries on color fundus images. To address these issues, in this paper, we propose a novel deep learning-based model called NFN＋ to effectively extract multi-scale information and make full use of deep feature maps. In NFN＋, the front network converts an image patch into a probabilistic retinal vessel map, and the followed network further refines the map to achieve a better post-processing module, which helps represent the vessel structures implicitly. We employ the inter-network skip connections to unite two identical multi-scale backbones, which enables the useful multi-scale features to be directly transferred from shallow layers to deeper layers. The refined probabilistic retinal vessel maps produced from the augmented images are then averaged to construct the segmentation results. We evaluated this model on the digital retinal images for vessel extraction (DRIVE), structured analysis of the retina (STARE), and the child heart and health study (CHASE) databases. Our results indicate that the elaborated cascaded designs can produce performance gain and the proposed NFN＋ model, to our best knowledge, achieved the state-of-the-art retinal vessel segmentation accuracy on color fundus images (AUC: 98.30%, 98.75% and 98.94%, respectively). • Proposed a novel network followed network (NFN＋) model for retinal vessel segmentation. • Design an elaborate cascaded architecture to enable to learn the segmentation again. • Use intra- and inter-network skip connections to transfer multi-scale information. • Achieve the state-of-the-art performances on three benchmark retinal vessel databases. [ABSTRACT FROM AUTHOR]

Time series forecasting is the process of predicting future values of a time series from knowledge of its past data. Although there are several models for making short-term predictions, the problem of long temporal sequences can still receive new contributions. Recent studies have applied Transformers-based solutions to the long-time series forecasting task and achieved good results. When it comes to time series modeling, the goal is to capture the temporal relationships within a sequence of ordered and continuous points. While using positional encoding and embedding sub-series as tokens in Transformers can help to preserve some ordering information, the permutation-invariant self-attention mechanism used in these models can lead to loss of temporal information. In this paper, we used convolutional networks in a binary tree structure with skip connections between the levels of the tree that allowed greater precision and efficiency in the training. The proposed model was evaluated on five real-life datasets. Experimental results show that our model significantly improves forecast accuracy relative to existing solutions. • We propose a new method for predicting Long sequence time-series. • Using skip connections to improve the convergence of the model. • Results using the proposed method proved superior to other approaches. [ABSTRACT FROM AUTHOR]