Search

In this article, a special type of fault isolation problem is reported where temporal information is required for the estimation of parameters from susceptible fault sub-space. Any physical system always obeys certain mathematical constraints under the normal operating condition, and that is incorporated in a behavioral model, which is essentially flowing and force or moment balance. In a fault detection and identification (FDI) model, only concurrent states are known. In FDI, the constraint relationships need to be derived only in terms of known variables, i.e., the measurements and nominal parameters, which are termed analytical redundancy relations (ARRs). The numerical evaluation of ARRs is residuals, and those oscillate within a definite bound of error when computed using test data. In this work, a novel function is formulated using the ARRs at different time instants and that function is minimized to estimate the suspected parameter values belonging to the non-isolable sub-space. Single-fault hypothesis is considered, and a genetic algorithm (GA) is used for the optimization. This FDI analysis may be beneficial for power hydraulic-driven heavy machinery such as hydraulic excavators, dumpers, and front-end loaders, which are mostly used in mines and construction sites. [ABSTRACT FROM AUTHOR]

Multivariate synchronization index (MSI), as an effective recognition algorithm for steady-state visual evoked potential (SSVEP) brain-computer interface (BCI), can accurately decode target frequencies without training. To further consider temporal features or extract harmonic components, extended MSI (EMSI), temporally local MSI (TMSI), and filter bank MSI (FBMSI) have been proposed. However, the promotion effects of the above three strategies on MSI have not been compared in detail. In this paper, the performance of EMSI, TMSI, and FBMSI under different time windows was analyzed with the same dataset. The results indicated that the improvement effect of the temporally local method on MSI was better than that of the other two methods under the short time window, and the effect of the filter bank method was better when the time window was greater than 0.8 s. Based on the idea of simultaneously extracting time–frequency features, FBEMSI and FBTMSI were proposed by integrating time delay embedding and temporally local method into FBMSI respectively. The two improved methods, which has no significant difference, can improve the recognition effect of FBMSI. But the computing time of FBEMSI was shorter, which can be a potential method for SSVEP-BCI. [ABSTRACT FROM AUTHOR]

A recent development in deep learning techniques has attracted attention to the decoding and classification of electroencephalogram (EEG) signals. Despite several efforts to utilize different features in EEG signals, a significant research challenge is using time-dependent features in combination with local and global features. Several attempts have been made to remodel the deep learning convolution neural networks (CNNs) to capture time-dependency information. These features are usually either handcrafted features, such as power ratios, or splitting data into smaller-sized windows related to specific properties, such as a peak at 300 ms. However, these approaches partially solve the problem but simultaneously hinder CNNs' capability to learn from unknown information that might be present in the data. Other approaches, like recurrent neural networks, are very suitable for learning time-dependent information from EEG signals in the presence of unrelated sequential data. To solve this, we have proposed an encoding kernel (EnK), a novel time-encoding approach, which uniquely introduces time decomposition information during the vertical convolution operation in CNNs. The encoded information lets CNNs learn time-dependent features in addition to local and global features. We performed extensive experiments on several EEG data sets--physical human-robot collaborations, P300 visual-evoked potentials, motor imagery, movement-related cortical potentials, and the Dataset for Emotion Analysis Using Physiological Signals. The EnK outperforms the state of the art with an up to 6.5% reduction in mean squared error (MSE) and a 9.5% improvement in F1-scores compared to the average for all data sets together compared to base models. These results support our approach and show a high potential to improve the performance of physiological and non-physiological data. Moreover, the EnK can be applied to virtually any deep learning architecture with minimal effort. [ABSTRACT FROM AUTHOR]

The fire heat release rate (HRR) is a crucial parameter for describing the combustion process and its thermal effects. In recent years, some studies have employed fire scene images and deep learning algorithms to predict real-time fire HRR, which has led to the advancement of HRR prediction in terms of both lightweightness and real-time monitoring. Nevertheless, the development of an early-stage monitoring system for fires and the ability to predict future HRR based on current moment data represents a crucial foundation for evaluating the scale of indoor fires and enhancing the capacity to prevent and control such incidents. This paper proposes a deep learning model based on continuous fire scene images (containing both flame and smoke features) and their time-series information to predict the future transient fire HRR. The model (Att-BiLSTM) comprises three bi-directional long- and short-term memory (Bi-LSTM) layers and one attention layer. The model employs a bidirectional feature extraction approach, followed by the introduction of an attention mechanism to highlight the image features that have a critical impact on the prediction results. In this paper, a large-scale dataset is constructed by collecting 27,231 fire scene images with instantaneous HRR annotations from 40 different fire trials from the NIST database. The experimental results demonstrate that Att-BiLSTM is capable of effectively utilizing fire scene image features and temporal information to accurately predict future transient HRR, including those in high-brightness fire environments and complex fire source situations. The research presented in this paper offers novel insights and methodologies for fire monitoring and emergency response. [ABSTRACT FROM AUTHOR]

Bearing is one of the most prone to failure components, and the most commonly used supporting parts in rotating machinery. There are important economic and safety significance for studying effective bearing fault diagnosis method. Deep learning has been widely applied to intelligent fault diagnosis of rotating machines. In particular, convolutional neural network (CNN) improves diagnostic accuracy due to automatic feature extraction ability. However, CNN cannot fully use the temporal information of the input signals and extract global features. This article presents a new depth model called two-stream multi-head self-attention convolutional neural network (TSMSCNN) for bearing fault diagnosis to solve the above problems. First, we obtain temporal information through the difference of data points in the input signal. Next, the temporal information and original signal are used as input for two CNN branches. Then, the multi-head self-attention (MS) mechanism adaptively aggregates the features and assigns weights to the features extracted by CNN. Finally, the learned high-level representations are fed into the full connect (FC) layer for fault diagnosis. Experimental results prove that the proposed TSMSCNN has higher accuracy than state-of-the-art methods and has excellent noise immunity and transfer performance. [ABSTRACT FROM AUTHOR]

Precise prediction of irrigation volumes is crucial in modern agriculture. This study proposes an optimized long short-term memory (LSTM) model-based irrigation prediction method that combines bidirectional LSTM networks. The model provides farmers with more precise irrigation management decisions, facilitating optimal utilization of water resources and effective crop production management. This proposed model aims to fully exploit spatio-temporal features and sequence dependencies to enhance prediction accuracy and reliability. We aim to fully leverage crop irrigation volumes’ spatio-temporal features and sequence dependencies to improve prediction accuracy and reliability. First, this study adopts a bidirectional LSTM (BiLSTM) model to simulate the temporal features of irrigation volumes and learn the sequential dependencies of crop growth data from historical records. Then, this study passes the irrigation volume data through a convolutional neural network (CNN) model to extract spatial features and capture correlations among various features such as temperature, precipitation, and wind speed. Our prediction performance significantly improved after incorporating an attention mechanism that involves weighting features and enhancing focus on crucial aspects. The proposed BiLSTM-CNN-Attention approach is used to predict irrigation volume for spring corn in significant irrigation areas in Jilin Province, China. The results demonstrate that the proposed method surpasses recurrent neural network (RNN), CNN, LSTM, BiLSTM, and BiLSTM-CNN methods in terms of mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE) (0.000004, 0.005968, 0.004599), and R² (0.9749), making a superior solution for predicting the volume of crop irrigation. [ABSTRACT FROM AUTHOR]

The precise reconstruction of accelerated magnetic resonance imaging (MRI) brings about notable advantages, such as enhanced diagnostic precision and decreased examination costs. In contrast, traditional cardiac MRI necessitates repetitive acquisitions across multiple heartbeats, resulting in prolonged acquisition times. Significant strides have been made in accelerating MRI through deep learning-based reconstruction methods. However, these existing methods encounter certain limitations: (1) The intricate nature of heart reconstruction involving multiple complex time-series data poses a challenge in exploring nonlinear dependencies between temporal contexts. (2) Existing research often overlooks weight sharing in iterative frameworks, impeding the effective capturing of non-local information and, consequently, limiting improvements in model performance. In order to improve cardiac MRI reconstruction, we propose a novel temporal-spatial transformer with a strategy in this study. Based on the multi-level encoder and decoder transformer architecture, we conduct multi-level spatiotemporal information feature aggregation over several adjacent views, that create nonlinear dependencies among features and efficiently learn important information among adjacent cardiac temporal frames. Additionally, in order to improve contextual awareness between neighboring views, we add cross-view attention for temporal information fusion. Furthermore, we introduce an iterative strategy for training weights during the reconstruction process, which improves feature fusion in critical locations and reduces the number of computations required to calculate global feature dependencies. Extensive experiments have demonstrated the substantial superiority of this procedure over the most advanced techniques, suggesting that it has broad potential for clinical use. [ABSTRACT FROM AUTHOR]

Several computer vision techniques start with the recognition of moving objects in video streams. The most popular strategy for recognizing moving items is background modelling and subtraction, yet it is still difficult to accurately detect moving things. For the first N frames, some approaches initialize the backdrop model at each pixel. However, because the backdrop model primarily consists of temporal information, it cannot function well in scenes with dynamic backgrounds. This research offers an unique motion detection approach, based on Grey relational analysis that merges a multi-quality background creation module and a moving object recognition module to accomplish motion detection, which is both dependable and accurate in streaming content of variable bit-rate. Here, a brand-new, parametric moving object recognition approach, which incorporates both spatial and spectral features is proposed. For this Mask Recurrent Neural Network (Mask RCNN) is proposed. The changing background can be precisely detected using the suggested method. It is also suggested to use a number of new ways to preserve and update the background model. The experimental findings, based on the image sequences in open datasets, demonstrate that the suggested method, when compared with existing methods, offers greater stability and efficiency in dynamic backdrop scenarios. [ABSTRACT FROM AUTHOR]

Drones are widespread and actively employed in a variety of applications due to their low cost and quick mobility and enabling new forms of action surveillance. However, owing to various challenges- limited no. of aerial view samples, aerial footage suffers with camera motion, illumination changes, small actor size, occlusion, complex backgrounds, and varying view angles, human action recognition in aerial videos even more challenging. Maneuvering the same, we propose Aerial Polarized-Transformer Network (AP-TransNet) to recognize human actions in aerial view using both spatial and temporal details of the video feed. In this paper, we present the Polarized Encoding Block that performs ( i) Selection with Rejection to select the significant features and reject least informative features similar to Light photometry phenomena and ( ii) boosting operation increases the dynamic range of encodings using non-linear softmax normalization at the bottleneck tensors in both channel and spatial sequential branches. The performance of the proposed AP-TransNet is evaluated by conducting extensive experiments on three publicly available benchmark datasets: drone action dataset, UCF-ARG Dataset and Multi-View Outdoor Dataset (MOD20) supporting with ablation study. The proposed work outperformed the state-of-the-arts. [ABSTRACT FROM AUTHOR]

With the escalation of global warming and human activities, large-scale wildfires have become increasingly frequent, posing significant threats to both ecological environments and human societal safety. Satellite remote sensing technology plays a pivotal role in wildfire monitoring and risk assessment, providing extensive geographical coverage and continuous monitoring capabilities. Traditional methods for predicating wildfire risk, however, face limitations in processing large-scale remote sensing data, especially in cloud detection and temporal information analysis. In response to this challenge, a novel set of image processing algorithms has been developed to enhance the efficiency and accuracy of wildfire risk prediction. Initially, a cloud detection and removal algorithm based on deep learning is introduced. This algorithm effectively identifies and eliminates cloud interference in remote sensing images, thereby significantly improving the quality and usability of image data. Subsequently, a temporal information capturing technique is proposed, capable of processing vast amounts of remote sensing data and extracting time series features. This technique provides robust data support for wildfire risk prediction. The application of these technologies not only improves the efficiency of the data processing workflow but also enhances the timeliness and accuracy of the prediction model, holding significant practical importance for guiding actual wildfire prevention and response measures. [ABSTRACT FROM AUTHOR]

Graph Convolutional Networks (GCN) are prevalent techniques in collaborative filtering recommendations. However, current GCN-based approaches for collaborative filtering recommendation have limitations in effectively embedding neighboring nodes during node and neighbor information aggregation. Furthermore, weight allocation for the user (or item) representations after convolution of each layer is too uniform. To resolve these limitations, we propose a new Graph Convolutional Collaborative Filtering recommendation method based on temporal information during the node aggregation process (TA-GCCF). The method aggregates and propagates information using Gated Recurrent Units, while dynamically updating features based on the timing and sequence of interactions between nodes and their neighbors. Concurrently, we have developed a convolution attention coefficient to ascertain the significance of embedding at distinct layers. Experiments on three benchmark datasets show that our method significantly outperforms the comparison methods in the accuracy of prediction. [ABSTRACT FROM AUTHOR]

The article introduces RESTC, a framework for improving session-based recommendation systems. It addresses the limitations of existing Graph Neural Network (GNN)-based models by incorporating spatio-temporal contrastive learning. It is reported that RESTC aligns spatial and temporal representations to capture user preferences over time.

A recent development in deep learning techniques has attracted attention to the decoding and classification of electroencephalogram (EEG) signals. Despite several efforts to utilize different features in EEG signals, a significant research challenge is using time-dependent features in combination with local and global features. Several attempts have been made to remodel the deep learning convolution neural networks (CNNs) to capture time-dependency information. These features are usually either handcrafted features, such as power ratios, or splitting data into smaller-sized windows related to specific properties, such as a peak at 300 ms. However, these approaches partially solve the problem but simultaneously hinder CNNs' capability to learn from unknown information that might be present in the data. Other approaches, like recurrent neural networks, are very suitable for learning time-dependent information from EEG signals in the presence of unrelated sequential data. To solve this, we have proposed an encoding kernel (EnK), a novel time-encoding approach, which uniquely introduces time decomposition information during the vertical convolution operation in CNNs. The encoded information lets CNNs learn time-dependent features in addition to local and global features. We performed extensive experiments on several EEG data sets—physical human-robot collaborations, P300 visual-evoked potentials, motor imagery, movement-related cortical potentials, and the Dataset for Emotion Analysis Using Physiological Signals. The EnK outperforms the state of the art with an up to 6.5% reduction in mean squared error (MSE) and a 9.5% improvement in F1-scores compared to the average for all data sets together compared to base models. These results support our approach and show a high potential to improve the performance of physiological and non-physiological data. Moreover, the EnK can be applied to virtually any deep learning architecture with minimal effort.

Accurate depth information is crucial for various computer vision applications such as augmented reality, 3D modeling, and autonomous vehicles. Recent advancements have significantly improved both supervised and self-supervised methods for depth estimation. However, most current approaches primarily focus on monocular depth estimation and face challenges in overcoming quality limitations due to the inherent constraints of supervised learning in deep neural networks. Incorporating temporal information from sequential frames can enhance the quality of these methods. This paper explores innovative methods for integrating recurrent blocks into existing pipelines for supervised depth estimation using convolutional long short-term memory (convLSTM). By utilizing convLSTM, we can capture a wealth of valuable information. To accurately measure object distances, we employed a geospatial approach. After extensive analysis of training methods, new deep neural network architectures have been designed specifically for monocular video depth estimation. Our research emphasizes using an attention mechanism to extract information from previous frames. The proposed approach effectively measures both long and short-range object distances by combining geospatial and temporal mechanisms, demonstrating superior performance in monocular depth estimation tasks.

Change detection (CD) in satellite time series is one of the hot research topics in the field of remote sensing (RS). Recently, graph convolutional network (GCN)-based CD methods have garnered significant attention from numerous researchers, as GCN can perform convolutional operations on irregular graphs using superpixels. However, these methods still face two challenges, i.e., inaccurate superpixel segmentation may lead to detection errors, and temporal information is ignored in the GCN-based CD methods. To this end, we propose a temporal information enhanced GCN with superpixel-pixel gated knowledge dynamic selection for CD (TSCD) in satellite time series. This method fully leverages the strengths of both GCN and convolutional neural network (CNN) frameworks to accurately identify changes in satellite time series. It is composed of three parts: First, a superpixel-level temporal information enhanced GCN (STIG) subnetwork, which is designed to capture the large-scale discriminative features of land surfaces, in which temporal information enhanced GCN (TIGCN) not only extracts spatial-spectral features of images, but also captures temporal correlations within satellite time series, thereby reducing the pseudochanges caused by temporal dynamics; second, a pixel-level 3D-CNN (P3DC) subnetwork, which is proposed to focus on subtle features by extracting temporal-spatial-spectral information to correct detection errors caused by superpixel segmentation; third, a gated knowledge dynamic selection module, which is designed to dynamically select the optimal features in the spectral dimension from the superpixel-level, pixel-level, and their combined features, thereby reducing the impact of redundant information on the detection performance. TSCD not only extracts large-scale and subtle features simultaneously, but also captures the temporal correlations among images. Comparative experimental results with eight state-of-the-art CD methods on nine publicly available satellite time series scenes demonstrate the effectiveness of the TSCD method.

The 3D object detection task is a crucial subtask of the environment perception module in Automated Driving System (ADS). The accuracy of object detection directly impacts the effectiveness of downstream autonomous driving tasks such as tracking, prediction, and planning. Existing 3D object detection networks in ADS that rely on multi-sensor fusion lack the utilization of temporal information and fail to fully consider the dynamic nature of the surrounding traffic environment. Therefore, we proposed BEVTemporal for ADS, which fuses LIDAR point clouds and surrounding multi-channel images. Its unique temporal module establishes the correlation between historical data and current data, effectively leveraging the temporal information of surrounding objects. We train and validate BEVTemporal on the nuScenes datasets. After incorporating temporal module, the Average Precision (AP) metrics of the network improved by 0.3%~1.7%, and mean Average precision (mAP) achieves 0.87% higher, nuScenes Detection Score (NDS) increased by 0.46%. The validation results on subsets of occluded objects show that the model effectively alleviates the problem of missed detection caused by sample occlusion, with significant improvements observed for heavily occluded samples. In different scenarios (sunny, rainy, daytime, night), mAP improvement ranges from 0.75% to 1.18%. Notably, in challenging scenarios such as rainy and night, AP can be improved by up to 3.6%. The experimental results show that BEVTemporal not only improves the accuracy of the 3D object detection network, but also significantly enhances the robustness of model in various scenarios and recall of objects under low visibility conditions.

Tourists spend a lot of effort in planning itineraries when organizing a trip. This is a complex activity that involves selecting the places to visit and dealing with a number of temporal issues to generate a schedule for the visit. In the paper we propose an architecture for a recommender that suggests personalized tourist itineraries and a personalized time schedule. The approach takes into account (i) user preferences for the places to be included in the itinerary and (ii) several temporal dimensions concerning both temporal information and constraints (e.g., opening hours, time for visiting each place, time to move among places) and time-related user preferences (e.g., number of days of the visit, preferring a dense schedule vs having a lot of free time, the variety of the types of attractions during a day of visit). The approach is based on a combination of genetic algorithms and temporal reasoning. It focuses on generating temporally annotated itineraries starting from a ranked list of places to be included. Thus, our solution is designed as a module that can be coupled with any system recommending places to visit. The design started from a user study we carried out to analyze the temporal dimensions to take into account, and to find relationships between such dimensions and users’ personality traits, and led to the development and evaluation of a prototypical implementation that generates personalized itineraries for the city of Turin.

Multimodal sentiment analysis (MSA) is an emerging field focused on interpreting complex human emotions and expressions by integrating various data types, including text, audio, and visuals. Addressing the challenges in this area, we introduce SentDep, a groundbreaking framework that merges cutting-edge fusion methods with modern deep learning structures. Designed to effectively blend the unique features of textual, acoustic, and visual data, SentDep offers a unified and potent representation of multimodal data. Our extensive tests on renowned datasets like CMU-MOSI and CMU-MOSEI demonstrate that SentDep surpasses current leading models, setting a new standard in MSA performance. We conducted thorough ablation studies and supplementary experiments to identify what drives SentDep’s success. These studies highlight the importance of the size of pre-training data, the effectiveness of various fusion techniques, and the critical role of temporal information in enhancing the model’s capabilities.

Addressing inherent limitations in distinguishing metrics relying solely on Euclidean distance, especially within the context of geo-indistinguishability (Geo-I) as a protection mechanism for location-based service (LBS) privacy, this paper introduces an innovative and comprehensive metric. Our proposed metric not only incorporates geographical information but also integrates semantic, temporal, and query data, serving as a powerful tool to foster semantic diversity, ensure high servifice similarity, and promote spatial dispersion. We extensively evaluate our technique by constructing a comprehensive metric for Dongcheng District, Beijing, using road network data obtained through the OSMNX package and semantic and temporal information acquired through Gaode Map. This holistic approach proves highly effective in mitigating adversarial attacks based on background knowledge. Compared with existing methods, our proposed protection mechanism showcases a minimum 50% reduction in service quality and an increase of at least 0.3 times in adversarial attack error using a real-world dataset from Geolife. The simulation results underscore the efficacy of our protection mechanism in significantly enhancing user privacy compared to existing methodologies in the LBS location privacy-protection framework. This adjustment more fully reflects the authors' preference while maintaining clarity about the role of Geo-I as a protection mechanism within the broader framework of LBS location privacy protection. [ABSTRACT FROM AUTHOR]

The target recognition algorithm based on canonical correlation analysis (CCA) has been widely used in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces. To reduce visual fatigue and improve the information transfer rate (ITR), how to improve the accuracy of algorithms within a short time window has become one of the main problems at present. There were filter bank CCA (FBCCA), individual template CCA (ITCCA), and temporally local CCA (TCCA), which improve the CCA algorithm from different aspects.This paper proposed to consider individual, frequency, and time information at the same time, so as to extract features more effectively. A comparison of the various methods was performed using benchmark dataset. Classification accuracy and ITR were used for performance evaluation. In the different extensions of CCA, the method incorporating the above three kinds of information simultaneously achieved the best performance within a short time window. This study explores the effect of using a variety of information to improve the CCA algorithm. [ABSTRACT FROM AUTHOR]

Studies in human–human interaction have introduced the concept of F-formation to describe the spatial arrangement of participants during social interactions. This paper has two objectives. It aims at detecting F-formations in video sequences and at predicting the next speaker in a group conversation. The proposed approach exploits time information and multimodal signals of humans in video sequences. In particular, we rely on measuring the engagement level of people as a feature of group belonging. Our approach makes use of a recursive neural network, the Long Short Term Memory (LSTM), to predict who will take the speaker's turn in a conversation group. Experiments on the MatchNMingle dataset led to 85% true positives in group detection and 98% accuracy in predicting the next speaker. • New method for analyzing videos of groups of persons in interaction. • New method for detecting groups (F-formations) using temporal information and multimodal signals. • Consider engagement level between persons as a feature of group belonging. • New method for predicting the next speaker based on LSTM. • Experiments on the MatchNMingle dataset: 85% true positives in group detection, 98% accuracy in predicting next speaker. [ABSTRACT FROM AUTHOR]

Micro- and macro-expression spotting in an untrimmed video is a challenging task, due to the mass generation of false positive samples. Most existing methods localize higher response areas by extracting hand-crafted features or cropping specific regions from all or some key raw images. However, these methods either neglect the continuous temporal information or model the inherent human motion paradigms (background) as foreground. Consequently, we propose a novel two-stream network, named Local suppression and Global enhancement Spotting Network (LGSNet), which takes segment-level features from optical flow and videos as input. LGSNet adopts anchors to encode expression intervals and selects the encoded deviations as the object of optimization. Furthermore, we introduce a Temporal Multi-Receptive Field Feature Fusion Module (TMRF $^{3}$ 3 M) and a Local Suppression and Global Enhancement Module (LSGEM), which help spot short intervals more precisely and suppress background information. To further highlight the differences between positive and negative samples, we set up a large number of random pseudo ground truth intervals (background clips) on some discarded sliding windows to accomplish background clips modeling to counteract the effect of non-expressive face and head movements. Experimental results show that our proposed network achieves state-of-the-art performance on the CAS(ME) $^{2}$ 2 , CAS(ME) $^{3}$ 3 and SAMM-LV datasets. [ABSTRACT FROM AUTHOR]

Skeleton‐based human action recognition is gaining significant attention and finding widespread application in various fields, such as virtual reality and human‐computer interaction systems. Recent studies have highlighted the effectiveness of graph convolutional network (GCN) based methods in this task, leading to a remarkable improvement in prediction accuracy. However, most GCN‐based methods overlook the varying contributions of self, centripetal and centrifugal subsets. Besides, only a single‐scale temporal feature is adopted, and the multi‐temporal scale information is ignored. To this end, firstly, in order to differentiate the importance of different skeleton subsets, we develop a refinement graph convolution, which can adaptively learn a weight for each subset feature. Secondly, a multi‐temporal scale aggregation module is proposed to extract more discriminative temporal dynamic information. Furthermore, a multi‐temporal scale aggregation refinement graph convolutional network (MTSA‐RGCN) is proposed, and four‐stream structure is also adopted in this paper, which can comprehensively model complementary features and eventually achieves a significant performance boost. In the empirical experiments, the performance of our approach has been greatly improved on both NTU‐RGB+D 60 and NTU‐RGB+D 120 datasets, compared to other state‐of‐the‐art methods. [ABSTRACT FROM AUTHOR]

Copyright of Psychological Science is the property of Psychological Science Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Accurate climate data at fine spatial resolution are essential for scientific research and the development and planning of crucial social systems, such as energy and agriculture. Among them, sea surface temperature plays a critical role as the associated El Niño–Southern Oscillation (ENSO) is considered a significant signal of the global interannual climate system. In this paper, we propose an implicit neural representation-based interpolation method with temporal information ( T _ I N R I ) to reconstruct climate data of high spatial resolution, with sea surface temperature as the research object. Traditional deep learning models for generating high-resolution climate data are only applicable to fixed-resolution enhancement scales. In contrast, the proposed T _ I N R I method is not limited to the enhancement scale provided during the training process and its results indicate that it can enhance low-resolution input by arbitrary scale. Additionally, we discuss the impact of temporal information on the generation of high-resolution climate data, specifically, the influence of the month from which the low-resolution sea surface temperature data are obtained. Our experimental results indicate that T _ I N R I is advantageous over traditional interpolation methods under different enhancement scales, and the temporal information can improve T _ I N R I performance for a different calendar month. We also examined the potential capability of T _ I N R I in recovering missing grid value. These results demonstrate that the proposed T _ I N R I is a promising method for generating high-resolution climate data and has significant implications for climate research and related applications. [ABSTRACT FROM AUTHOR]

Summary: The messages in microblogs, like Twitter, present a wealth of information around the hot topics and public opinions. Topic detection and tracking enables a better understanding of the incoming information from social networks. Many of the studies performed in this field consider a set of predefined number of topics (clusters) which remain constant during the detection process. Such approaches are not efficient when data is dynamic and cumulative. In addition, nonparametric evolutionary topic models do not have an accurate performance on short texts due to the data deficiency. In this article, we propose an evolutionary clustering model based on the Bayesian nonparametric Dirichlet process mixture model, which automatically determines the number of clusters. To improve the algorithm performance when dealing with short social media texts, temporal information of tweets and other social network parameters are used in a weighted combination to augment the text similarity in detecting topics. Model evaluation is performed on the data gathered from the Twitter during a 2.5‐month period. Results reveal that the social network information can promote the topic detection performance. The proposed model outperforms previous study in terms of topic coherence and clustering performance, leading to a better clustering on short texts. [ABSTRACT FROM AUTHOR]

Textual documents serve as representations of discussions on a variety of subjects. These discussions can vary in length and may encompass a range of events or factual information. Present trends in constructing knowledge bases primarily emphasize fact-based common sense reasoning, often overlooking the temporal dimension of events. Given the widespread presence of time-related information, addressing this temporal aspect could potentially enhance the quality of common-sense reasoning within existing knowledge graphs. In this comprehensive survey, we aim to identify and evaluate the key tasks involved in constructing temporal knowledge graphs centered around events. These tasks can be categorized into three main components: (a) event extraction, (b) the extraction of temporal relationships and attributes, and (c) the creation of event-based knowledge graphs and timelines. Our systematic review focuses on the examination of available datasets and language technologies for addressing these tasks. An in-depth comparison of various approaches reveals that the most promising results are achieved by employing state-of-the-art models leveraging large pre-trained language models. Despite the existence of multiple datasets, a noticeable gap exists in the availability of annotated data that could facilitate the development of comprehensive end-to-end models. Drawing insights from our findings, we engage in a discussion and propose four future directions for research in this domain. These directions encompass (a) the integration of pre-existing knowledge, (b) the development of end-to-end systems for constructing event-centric knowledge graphs, (c) the enhancement of knowledge graphs with event-centric information, and (d) the prediction of absolute temporal attributes. [ABSTRACT FROM AUTHOR]

Autonomous decision-making in unmanned aerial vehicle (UVA) confrontations presents challenges in making optimal strategy. Therefore, deep reinforcement learning (DRL) has been adopted to address these issues. However, existing DRL decision-making models suffer from poor situational awareness and inability to distinguish between different intentions. Therefore, a multi-intent autonomous decision-making is proposed in this paper. First, three typical intentions are designed comprising head-on attacking, pursuing and fleeing to derive decision models representing different intentions. Reinforcement learning based air combat game model is constructed with different intentions, which contains designing reward functions for intentions to deal with the problem of sparse rewards. Then, we propose the Temporal Proximal Policy Optimization (T-PPO) algorithm, which optimizes the Proximal Policy Optimization algorithm by integrating the long short-term memory network and feedforward neural network. This algorithm extracts the historical temporal information to enhance situational awareness. In addition, a basic-confrontation progressive training method is proposed to provide intention guidance and increase training diversity, which can improve learning efficiency and intelligent decision-making capability. Finally, experiments in our constructed UAV confrontation environment demonstrate that the proposed intentional decision models exhibit good performance in stability and learning efficiency, achieving high rewards, win rates, and low steps. Specifically, our autonomous decision-making increases win rate by 26% when head-on attacking and learning efficiency by 50% when pursuing. It is further proof of the potential and value of our multi-intent autonomous decision-making applications. [ABSTRACT FROM AUTHOR]

The fire heat release rate (HRR) is a crucial parameter for describing the combustion process and its thermal effects. In recent years, some studies have employed fire scene images and deep learning algorithms to predict real-time fire HRR, which has led to the advancement of HRR prediction in terms of both lightweightness and real-time monitoring. Nevertheless, the development of an early-stage monitoring system for fires and the ability to predict future HRR based on current moment data represents a crucial foundation for evaluating the scale of indoor fires and enhancing the capacity to prevent and control such incidents. This paper proposes a deep learning model based on continuous fire scene images (containing both flame and smoke features) and their time-series information to predict the future transient fire HRR. The model (Att-BiLSTM) comprises three bi-directional long- and short-term memory (Bi-LSTM) layers and one attention layer. The model employs a bidirectional feature extraction approach, followed by the introduction of an attention mechanism to highlight the image features that have a critical impact on the prediction results. In this paper, a large-scale dataset is constructed by collecting 27,231 fire scene images with instantaneous HRR annotations from 40 different fire trials from the NIST database. The experimental results demonstrate that Att-BiLSTM is capable of effectively utilizing fire scene image features and temporal information to accurately predict future transient HRR, including those in high-brightness fire environments and complex fire source situations. The research presented in this paper offers novel insights and methodologies for fire monitoring and emergency response.

A knowledge hypergraph is a form of heterogeneous graph representing the real world through n-ary relations, but existing knowledge hypergraphs are usually incomplete in both general and vertical domains. Therefore, it is challenging to infer the missing links from the existing links in knowledge hypergraphs. Most of the current studies employ knowledge representation learning methods based on n-ary relations to accomplish link prediction in knowledge hypergraphs, but they only learn the embedding vectors of entities and relations from time-unknown hyperedges without considering the influence of temporal factors on the dynamic evolution of facts, which results in poor prediction performance in dynamic environments. Firstly, based on the definition of temporal knowledge hypergraphs proposed by this paper for the first time, this paper puts forward a link prediction model for temporal knowledge hypergraphs and learns static and dynamic representations of entities from their roles, positions, and timestamps of temporal hyperedges. Then these representations are merged in a certain proportion and utilized as final entity embedding vectors for link prediction tasks to realize the full exploitation of hyperedge temporal information. Meanwhile, it is theoretically proven that the proposed model is fully expressive with linear space complexity. Additionally, a temporal knowledge hypergraph dataset CB67 is constructed from the public business data of listed companies, and a large number of experimental evaluations are conducted on this dataset. The experimental results show that the proposed model can effectively perform link prediction tasks on the temporal knowledge hypergraph dataset. [ABSTRACT FROM AUTHOR]

The increasing complexity of industrial processes brings new challenges to fault diagnosis tasks, and different types of faults have higher and higher requirements on the performance of fault classification models. This paper proposes a novel multivariate nonlinear temporal‐related fault diagnosis method based on gated recurrent units (GRUs). First, to improve the performance of the model in local information extraction and global integration, high dimensional variables are divided into multiple sub‐blocks according to the structure of chemical process units, and a new block normalization method is proposed to improve the performance of local feature extraction. Second, aiming at the slow drifting faults, the GRU network is adopted inside the sub‐block to extract local sparse and nonlinear temporal features. By combining the variance features of variables after block normalization, the performance of the model on multiplicative faults will improve. Finally, aiming at the complex correlation between variables, a new recurrent matrix method is proposed to extract the time transform information inside each variable to improve the comprehensive performance of the model. Through a multi‐level feature integration strategy, the model can be trained in parallel to improve the training speed. The proposed method shows good performance in the Tennessee Eastman process, and the extracted multi‐class features allow the model to be trained end‐to‐end and simultaneously diagnose multiple types of faults. [ABSTRACT FROM AUTHOR]

Past research indicates that humans can infer hidden causesfrom covariational evidence, and readily use temporal informa-tion to infer relationships among events. Here we explore a set-ting in which people can attribute events to a common hiddencause or causal relationships among observed events, includingcausal cycles, purely on the basis of timing information. Wepresent data from three behavioral experiments and extend pre-viously proposed Bayesian models that makes use of order anddelay information for causal structure learning. Our findingssupport the idea that people rely on the delays between eventsrather than order information alone. Meanwhile, deviationsfrom our model predictions suggest that people have an induc-tive bias against common hidden causes and rely on heuristicsto distinguish between causal structures, such as event over-laps, at least with the cover story considered in these experi-ments. Further, our data suggest that people have particularlyflexible representations of cyclic relationships.

As a structured form of human knowledge,knowledge graphs have played a great supportive role in supporting the semantic intercommunication of massive,multi-source,heterogeneous data,and effectively support tasks such as data analysis,attracting the attention of academia and industry.At present,most knowledge graphs are constructed based on non-real-time static data,without considering the temporal characteristics of entities and relationships.However,data in application scenarios such as social network communication,financial trade,and epidemic spreading network are highly dynamic and exhibit complex temporal properties.How to use time series data to build knowledge graphs and effectively model them is a challenging problem.Recently,numerous studies use temporal information in time series data to enrich the characteristics of knowledge graphs,endowing know-ledge graphs with dynamic features,expanding fact triples into quadruple representation(head entity,relationship,tail entity,time).The knowledge graph which utilizes time-related quadruples to represent knowledge are collectively referred to as temporal knowledge graph.This paper summarizes the research work of temporal knowledge graph representation learning by sorting out and analyzing the corresponding literature.Specifically,it first briefly introduce the background and definition of temporal know-ledge graph.Next,it summarizes the advantages of the temporal knowledge graph representation learning method compared with the traditional knowledge graph representation learning method.Then it elaborates on the recent method of temporal knowledge graph representation learning from the perspective of the method modeling facts,introduces the dataset used by the above method and summarizes the main challenges of this technology.Finally,the future research direction is prospected.

Visual media, in today’s world, has swept across most forms of day to day communication. In the paradigm of generative modelling, restricted Boltzmann machines (RBMs) are used to solve complex tasks such as feature extraction, neuroimaging, collaborative filtering, radar target cognition, etc. In this paper, we implement an unsupervised Boltzmann machine-based time-aware recommendation system (UBMTR) which detects underlying hidden features in user-movie ratings data in connection with the time at which each rating was made (temporal information). The model takes ratings and time as a dual-input and outputs binary values via the contrastive divergence algorithm which samples from a Monte Carlo Markov Chain. Arguably, there exists a correlation between the content requested and the temporal conditions, which is exactly what our model tries to exploit. There is seldom any work in the field of recommender systems built using Boltzmann machines that incorporate temporal information, which necessitates research in this domain. RBMs are adept at pattern completion to tackle missing values, and can deal with imbalanced datasets and unstructured data by encoding raw data into latent variables. Using RBM, the UBMTR outperforms many earlier made attempts made at recommendation systems done through CF and deep learning or their hybridized models.

While the evolution of mobile computing is experiencing considerable growth, it is at the same time seriously threatened by the limitations of battery technology, which does not keep pace with the evergrowing increase in energy requirements of mobile applications. Yet, with the limits of human perception and the diversity of requirements that individuals may have, a question arises of whether the effort should be made to always deliver the highest quality result to a mobile user? In this work we investigate how a user's physical activity, the spatial/temporal properties of the video, and the user's personality traits interact and jointly influence the minimal acceptable playback resolution. We conduct two studies with 45 participants in total and find out that the minimal acceptable resolution indeed varies across different contextual factors. Our predictive models inferring the lowest acceptable playback resolution, together with the reduced power consumption we measure at lower resolutions, open an opportunity for saving a mobile's energy through context-adaptable approximate computing. [ABSTRACT FROM AUTHOR]

Copyright of Journal Of Sichuan University (Natural Sciences Division) / Sichuan Daxue Xuebao-Ziran Kexueban is the property of Editorial Department of Journal of Sichuan University Natural Science Edition and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

As one of the essential modules in intelligent surveillance systems, loitering detection plays an important role in reducing theft incidents by analyzing human behavior. This paper introduces a novel strategy for detecting the loitering activities of humans in the monitoring area for an intelligent surveillance system based on a vision sensor. The proposed approach combines spatial and temporal information in the feature extraction stage to decide whether the human movement can be regarded as loitering. This movement has been previously tracked using human detectors and particle filter tracking. The proposed method has been evaluated using our dataset consisting of 20 videos. The experimental results show that the proposed method could achieve a relatively good accuracy of 85% when utilizing the random forest classifier in the decision stage. Thus, it could be integrated as one of the modules in an intelligent surveillance system. [ABSTRACT FROM AUTHOR]

Addressing inherent limitations in distinguishing metrics relying solely on Euclidean distance, especially within the context of geo-indistinguishability (Geo-I) as a protection mechanism for location-based service (LBS) privacy, this paper introduces an innovative and comprehensive metric. Our proposed metric not only incorporates geographical information but also integrates semantic, temporal, and query data, serving as a powerful tool to foster semantic diversity, ensure high servifice similarity, and promote spatial dispersion. We extensively evaluate our technique by constructing a comprehensive metric for Dongcheng District, Beijing, using road network data obtained through the OSMNX package and semantic and temporal information acquired through Gaode Map. This holistic approach proves highly effective in mitigating adversarial attacks based on background knowledge. Compared with existing methods, our proposed protection mechanism showcases a minimum 50% reduction in service quality and an increase of at least 0.3 times in adversarial attack error using a real-world dataset from Geolife. The simulation results underscore the efficacy of our protection mechanism in significantly enhancing user privacy compared to existing methodologies in the LBS location privacy-protection framework. This adjustment more fully reflects the authors’ preference while maintaining clarity about the role of Geo-I as a protection mechanism within the broader framework of LBS location privacy protection.

Accurate climate data at fine spatial resolution are essential for scientific research and the development and planning of crucial social systems, such as energy and agriculture. Among them, sea surface temperature plays a critical role as the associated El Niño–Southern Oscillation (ENSO) is considered a significant signal of the global interannual climate system. In this paper, we propose an implicit neural representation-based interpolation method with temporal information (T_INRI) to reconstruct climate data of high spatial resolution, with sea surface temperature as the research object. Traditional deep learning models for generating high-resolution climate data are only applicable to fixed-resolution enhancement scales. In contrast, the proposed T_INRI method is not limited to the enhancement scale provided during the training process and its results indicate that it can enhance low-resolution input by arbitrary scale. Additionally, we discuss the impact of temporal information on the generation of high-resolution climate data, specifically, the influence of the month from which the low-resolution sea surface temperature data are obtained. Our experimental results indicate that T_INRI is advantageous over traditional interpolation methods under different enhancement scales, and the temporal information can improve T_INRI performance for a different calendar month. We also examined the potential capability of T_INRI in recovering missing grid value. These results demonstrate that the proposed T_INRI is a promising method for generating high-resolution climate data and has significant implications for climate research and related applications.