Search

We focus on unsupervised representation learning for skeleton based action recognition. Existing unsupervised approaches usually learn action representations by motion prediction but they lack the ability to fully learn inherent semantic similarity. In this paper, we propose a novel framework named Prototypical Contrast and Reverse Prediction (PCRP) to address this challenge. Different from plain motion prediction, PCRP performs reverse motion prediction based on encoder-decoder structure to extract more discriminative temporal pattern, and derives action prototypes by clustering to explore the inherent action similarity within the action encoding. Specifically, we regard action prototypes as latent variables and formulate PCRP as an expectation-maximization (EM) task. PCRP iteratively runs (1) E-step as to determine the distribution of action prototypes by clustering action encoding from the encoder while estimating concentration around prototypes, and (2) M-step as optimizing the model by minimizing the proposed ProtoMAE loss, which helps simultaneously pull the action encoding closer to its assigned prototype by contrastive learning and perform reverse motion prediction task. Besides, the sorting can also serve as a temporal task similar as reverse prediction in the proposed framework. Extensive experiments on N-UCLA, NTU 60, and NTU 120 dataset present that PCRP outperforms main stream unsupervised methods and even achieves superior performance over many supervised methods.

Action recognition via 3D skeleton data is an emerging important topic. Most existing methods rely on hand-crafted descriptors to recognize actions, or perform supervised action representation learning with massive labels. In this paper, we for the first time propose a contrastive action learning paradigm named AS-CAL that exploits different augmentations of unlabeled skeleton sequences to learn action representations in an unsupervised manner. Specifically, we first propose to contrast similarity between augmented instances of the input skeleton sequence, which are transformed with multiple novel augmentation strategies, to learn inherent action patterns (“pattern-invariance ”) in different skeleton transformations. Second, to encourage learning the pattern-invariance with more consistent action representations, we propose a momentum LSTM , which is implemented as the momentum-based moving average of LSTM based query encoder, to encode long-term action dynamics of the key sequence. Third, we introduce a queue to store the encoded keys, which allows flexibly reusing proceeding keys to build a consistent dictionary to facilitate contrastive learning. Last, we propose a novel representation named Contrastive Action Encoding (CAE) to represent human’s action effectively. Empirical evaluations show that our approach significantly outperforms hand-crafted methods by 10–50% Top-1 accuracy, and it can even achieve superior performance to many supervised learning methods (Our codes are available at https://github.com/Mikexu007/AS-CAL ).

Pervasive edge computing (PEC) is an emerging paradigm for the industrial Internet of Things (IIoT), and software-defined networks (SDN) offer lower latency services, and massive intelligent devices connectivity for the IIoT. However, the PEC has some issues with data security, and privacy while PEC devices sharing data among edges. What's more, the centralized SDN suffers from single point of attacks such as distributed denial of service (DDoS) from IIoT devices, and has the challenge of data leakage. In this article, we use blockchain, and proxy reencryption (PRE) technologies to tackle these challenges. The blockchain authorizes all devices in the network to improve their credibility, and authenticity. In addition, a blockchain-based data sharing framework that combines a PRE scheme is introduced for secure device-to-device communication in PEC environments. A series of smart contracts are designed for flexible operations of searching, and updating records on the blockchain. The experiments reveal that our design is highly efficient, and has high performance.

The explosive growth of mobile devices promotes the prosperity of novel mobile applications, which can be realized by service offloading with the assistance of edge computing servers. However, due to limited computation and storage capabilities of a single server, long service latency hinders the continuous development of service offloading in mobile networks. By supporting multi-server cooperation, Pervasive Edge Computing (PEC) is promising to enable service migration in highly dynamic mobile networks. With the objective of maximizing the system utility, we formulate the optimization problem by jointly considering the constraints of server storage capability and service execution latency. To enable dynamic service placement, we first utilize Lyapunov optimization method to decompose the long-term optimization problem into a series of instant optimization problems. Then, a sample average approximation-based stochastic algorithm is proposed to approximate the future expected system utility. Afterwards, a distributed Markov approximation algorithm is utilized to determine the service placement configurations. Through theoretical analysis, the time complexity of our proposed algorithm is linear to the number of users, and the backlog queue of PEC servers is stable. Performance evaluations are conducted based on both synthetic and real trace-driven scenarios, with numerical results demonstrating the effectiveness of our proposed algorithm from various aspects.

The Internet of Medical Things (IoMT) is an extension and specialization of that original Internet of Things (IoT) concept, and applies to the interconnectedness of devices, software applications, and data which are specific to the medical industry. IoMT can add smart technologies to medical devices to monitor the progression of a disease away from the doctor’s office and learn things that could impact future care guidelines and patients. It can also provide a better way to care for our elderly by tracking vitals and heart performance, glucose and other body systems, and activity and sleeping levels. During the outbreak of pandemic (e.g., COVID-19), IoMT can even be used to detect main symptoms ubiquitously using intelligent sensors and trace the origin of the outbreak based on aggregated IoT data (e.g., geographic mobile data and purchase history). Although most of the contemporary IoMT systems can measure risks, make decisions, and take actions automatically, the lack of emotion-aware abilities will be an obstacle to more harmonious human–machine interaction and more efficient medical process. Besides, mental disorders, such as depression, schizophrenia, and anxiety, have become a more noticeable cause of suffering. The integration of emotion-aware abilities into IoMT can also contribute to monitor emotional dysregulation continuously in subjects with mental disorders or undergoing serious pandemic such as COVID-19, and give these patients personalized therapy recommendations. Research on affective computing has defined a framework to recognize, interpret, and process human affects, but more research is needed to investigate its application to biomedical applications, especially “in the wild” and over extended periods of time, and how to integrate emotion-aware abilities into IoMT organically is still an open question. This special issue aims to create a platform for researchers, developers, and practitioners from both academia and industry to disseminate the state-of-the-art results and to advance the Emotion-Aware ubiquitous computing in IoMT.

Soybean is a source of edible oil for humans and provides a third of the vegetable oil consumed worldwide. Increasing seed oil content in seeds is thus a key objective in soybean breeding. In the present study, a four-way recombinant inbred line (FW-RIL) population comprising 144 lines, planted in 10 environments, and a germplasm panel of 455 accessions, planted in two environments, were used to collect oil-content phenotypes. First, 59 quantitative trait loci (QTL) were detected in the FW-RIL population by inclusive complete interval mapping on a linkage map consisting of 2232 single-nucleotide polymorphism (SNP) markers. Also in the FW-RILs, 44 quantitative trait nucleotides (QTNs) were detected by association analysis using 109,676 SNP markers and five methods of multi-locus genome-wide association study. Second, 77 QTN were detected by association analysis in the germplasm panel using 63,306 markers. Comparison of the QTL and QTN suggested four QTN controlling oil content. Pathway analysis was performed on genes in attenuation regions of these four QTN, and two candidate genes involved in the synthesis or metabolism of soybean oil were identified. These findings provide useful information about the genetics of oil content and may contribute to its genetic improvement by marker-assisted selection.

Automation in smart manufacturing depends on many parties in the supply chain and distribution networks. A smart manufacturer needs to track the location and state of materials from its suppliers and products to its customers/distributors to manage the corresponding production schedule. The need for such information can be fulfilled by the use of Industrial Internet of Things (IIoT) devices installed at each respective party. However, even though all parties may trust one another, they have confidential data produced by the IIoT devices that are not allowed to be shared with other parties, or only allowed to be exchanged with certain parties. Even though the data are not confidential, they may still be required to share partial data based on the needs of each party. On the other hand, all parties have their independent systems that differ from each other, which leads to complex integration. Regarding those situations, a private-blockchain-based IIoT is promising to bridge the need for product and material tracking information exchange between parties while maintaining the respective confidentiality.

The rapid development of IoMT facilitates pervasive healthcare networks. However, numerous MUs lead to deficient wireless channel and computation resources. Edge computing based healthcare system is appreciated as a satisfactory paradigm to solve this obstacle. Based on the physical bound of WBANs, the IoMT is separated into two sub-networks, that is, intra- WBANs and beyond-WBANs. Considering the characteristics of healthcare systems, the cost of MUs relies on medical urgency, AoI and energy dissipation. In intra-WBANs, the wireless channel resource allocation problem is modeled by a cooperative game. The Nash bargaining solution is utilized to obtain the unique Pareto optimal point. In beyond-WBANs, MUs are considered to be rational and potentially selfish. Thus, a non-cooperative game is formulated to minimize the system-wide cost. To demonstrate the effectiveness of our proposed solution, performance evaluations are conducted with respect to the system-wide cost and the number of MUs benefiting from edge computing. Finally, several research challenges and open issues are discussed for further work.

This study aimed to construct a novel multimodal model by fusing different electroencephalogram (EEG) data sources, which were under neutral, negative and positive audio stimulation, to discriminate between depressed patients and normal controls. The EEG data of different modalities were fused using a feature-level fusion technique to construct a depression recognition model. The EEG signals of 86 depressed patients and 92 normal controls were recorded simultaneously while receiving different audio stimuli. Then, from the EEG signals of each modality, linear and nonlinear features were extracted and selected to obtain features of each modality. In addition, a linear combination technique was used to fuse the EEG features of different modalities to build a global feature vector and find several powerful features. Furthermore, genetic algorithms were used to perform feature weighting to improve the overall performance of the recognition framework. The classification accuracy of each classifier, namely the k-nearest neighbor (KNN), decision tree (DT), and support vector machine (SVM), was compared, and the results were encouraging. The highest classification accuracy of 86.98% was obtained by the KNN classifier in the fusion of positive and negative audio stimuli, demonstrating that the fusion modality could achieve higher depression recognition accuracy rate compared with the individual modality schemes. This study may provide an additional tool for identifying depression patients.

Mild cognitive impairment (MCI) is a pre-existing state of Alzheimer's disease (AD). An accurate prediction on the conversion from MCI to AD is of vital clinical significance for potential prevention and treatment of AD. Longitudinal studies received widespread attention for investigating the disease progression, though most studies did not sufficiently utilize the evolution information. In this paper, we proposed a cerebral similarity network with more progression information to predict the conversion from MCI to AD efficiently. First, we defined the new dynamic morphological feature to mine longitudinal information sufficiently. Second, based on the multiple dynamic morphological features the cerebral similarity network was constructed by sparse regression algorithm with optimized parameters to obtain better prediction performance. Then, leave-one-out cross-validation and support vector machine (SVM) were employed for the training and evaluation of the classifiers. The proposed methodology obtained a high accuracy of 92.31% (Sensitivity = 100%, Specificity = 82.86%) in a three-year ahead prediction of MCI to AD conversion. Experiment results suggest the effectiveness of the dynamic morphological feature, serving as a more sensitive biomarker in the prediction of MCI conversion.

With the emergence of the ever-increasing vehicular applications and booming Internet services, the requirements of low-latency and high efficient transmission among vehicles become urgent to meet, and their corresponding solutions need to be well investigated. To resolve the above challenges, we propose a fog computing-based content transmission scheme with collective filtering in edge of vehicles. We first provide a system model based on fog-based rode side units by considering location-awareness, content-caching and decentralized computing. Then, a content-caching strategy in RSUs is designed to minimize the downloading latency. Specifically, we model the moving vehicles with the two-dimensional Markov chains, and calculate the probabilities of file caching in RSUs to minimize the latency in file downloading. Each vehicle can also maintain a neighbor list to record the encounters with high similarities, and update it based on the historic and real-time contacts. Finally, we carry on the experiments based on the real-world taxi trajectories in Beijing and Shanghai, China. Simulation results demonstrate the effectiveness of our proposed method.

Visual relationship detection (VRD) is one newly developed computer vision task, aiming to recognize relations or interactions between objects in an image. It is a further learning task after object recognition, and is important for fully understanding images even the visual world. It has numerous applications, such as image retrieval, machine vision in robotics, visual question answer (VQA), and visual reasoning. However, this problem is difficult since relationships are not definite, and the number of possible relations is much larger than objects. So the complete annotation for visual relationships is much more difficult, making this task hard to learn. Many approaches have been proposed to tackle this problem especially with the development of deep neural networks in recent years. In this survey, we first introduce the background of visual relations. Then, we present categorization and frameworks of deep learning models for visual relationship detection. The high-level applications, benchmark datasets, as well as empirical analysis are also introduced for comprehensive understanding of this task.

Soybean is an important global crop for edible protein and oil, and plant height is a main breeding goal which is closely related to its plant shape and yield. In this research, a high-density genetic linkage map was constructed by 1996 SNP-bin markers on the basis of a recombinant inbred line population derived from Dongnong L13 × Henong 60. A total of 33 QTL related to plant height were identified, of which five were repeatedly detected in multiple environments. In addition, a 455-germplasm population with 63,306 SNP markers was used for multi-locus association analysis. A total of 62 plant height QTN were detected, of which 26 were detected repeatedly under multiple methods. Two candidate genes, Glyma.02G133000 and Glyma.05G240600, involving in plant height were predicted by pathway analysis in the regions identified by multiple environments and backgrounds, and validated by qRT-PCR. These results enriched the soybean plant height regulatory network and contributed to molecular selection-assisted breeding.

Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications. Existing methods typically learn body and motion features from the body-joint trajectory, whereas they lack a systematic way to model body structure and underlying relations of body components beyond the scale of body joints. In this paper, we for the first time propose a Self-supervised Multi-scale Skeleton Graph Encoding (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics from unlabeled skeleton graphs of various scales to learn an effective skeleton representation for person Re-ID. Specifically, we first devise multi-scale skeleton graphs with coarse-to-fine human body partitions, which enables us to model body structure and skeleton dynamics at multiple levels. Second, to mine inherent correlations between body components in skeletal motion, we propose a multi-scale graph relation network to learn structural relations between adjacent body-component nodes and collaborative relations among nodes of different scales, so as to capture more discriminative skeleton graph features. Last, we propose a novel multi-scale skeleton reconstruction mechanism to enable our framework to encode skeleton dynamics and high-level semantics from unlabeled skeleton graphs, which encourages learning a discriminative skeleton representation for person Re-ID. Extensive experiments show that SM-SGE outperforms most state-of-the-art skeleton-based methods. We further demonstrate its effectiveness on 3D skeleton data estimated from large-scale RGB videos. Our codes are open at https://github.com/Kali-Hac/SM-SGE., Accepted at ACMMM 2021 Main Track. Sole copyright holder is ACMMM. Codes are available at https://github.com/Kali-Hac/SM-SGE

Skeleton-based person re-identification (Re-ID) is an emerging open topic providing great value for safety-critical applications. Existing methods typically extract hand-crafted features or model skeleton dynamics from the trajectory of body joints, while they rarely explore valuable relation information contained in body structure or motion. To fully explore body relations, we construct graphs to model human skeletons from different levels, and for the first time propose a Multi-level Graph encoding approach with Structural-Collaborative Relation learning (MG-SCR) to encode discriminative graph features for person Re-ID. Specifically, considering that structurally-connected body components are highly correlated in a skeleton, we first propose a multi-head structural relation layer to learn different relations of neighbor body-component nodes in graphs, which helps aggregate key correlative features for effective node representations. Second, inspired by the fact that body-component collaboration in walking usually carries recognizable patterns, we propose a cross-level collaborative relation layer to infer collaboration between different level components, so as to capture more discriminative skeleton graph features. Finally, to enhance graph dynamics encoding, we propose a novel self-supervised sparse sequential prediction task for model pre-training, which facilitates encoding high-level graph semantics for person Re-ID. MG-SCR outperforms state-of-the-art skeleton-based methods, and it achieves superior performance to many multi-modal methods that utilize extra RGB or depth features. Our codes are available at https://github.com/Kali-Hac/MG-SCR.

Background: Frogeye leaf spot (FLS) is a destructive fungal disease in soybean production. The most economical and effective strategy to control FLS is to use resistant cultivars. However, the use of a limited number of resistant loci in FLS management will be countered by the emergence of new, and more virulent, Cercospora sojina races. Thus, we identified quantitative trait loci (QTL) that control resistance to FLS and identified novel resistant genes using a genome-wide association study (GWAS) on 234 Chinese soybean cultivars.Results: A total of 30,890 single nucleotide polymorphism (SNP) markers were used to estimate linkage disequilibrium (LD) and population structure. The GWAS results identified six loci (pGlyma20g31630 encoding pyruvate dehydrogenase (PDH), Glyma05g28980 encoding mitogen-activated protein kinase 7 (MPK7), and Glyma20g31510, Glyma20g31520 encoding the calcium-dependent protein kinase 4(CDPK4) in the haplotype blocks were found to be associated with FLS resistance.Conclusions: The proteins encoded by these four genes directly, or indirectly participate in the biological pathway of salicylic acid (SA) and jasmonic acid (JA). These two plant hormones can induce the expression of disease resistance related genes and are essential for plant systemic acquired resistance (SAR). Therefore, the novel loci and candidate genes provide genomic location information for further advances in FLS resistance gene cloning and molecular breeding.

Efficient detection of outliers from massive data with a high outlier ratio is challenging but not explicitly discussed yet. In such a case, existing methods either suffer from poor robustness or require expensive computations. This paper proposes a Low-rank based Efficient Outlier Detection (LEOD) framework to achieve favorable robustness against high outlier ratios with much cheaper computations. Specifically, it is worth highlighting the following aspects of LEOD: (1) Our framework exploits the low-rank structure embedded in the similarity matrix and considers inliers/outliers equally based on this low-rank structure, which facilitates us to encourage satisfying robustness with low computational cost later; (2) A novel re-weighting algorithm is derived as a new general solution to the constrained eigenvalue problem, which is a major bottleneck for the optimization process. Instead of the high space and time complexity (O((2n)2)/O((2n)3)) required by the classic solution, our algorithm enjoys O(n) space complexity and a faster optimization speed in the experiments; (3) A new alternative formulation is proposed for further acceleration of the solution process, where a cheap closed-form solution can be obtained. Experiments show that LEOD achieves strong robustness under an outlier ratio from 20% to 60%, while it is at most 100 times more memory efficient and 1000 times faster than its previous counterpart that attains comparable performance. The codes of LEOD are publicly available at https://github.com/demonzyj56/LEOD.

As an application of “smart transport” for Internet of Things, Internet of Vehicle (IoV) has emerged as a new research field based on vehicular ad hoc networks (VANETs). With the development of smart vehicles and the integration of sensors, applications of traffic management and road safety in large-scale IoV systems have drawn great attentions. By sensing events occurred on roads, vehicles can broadcast messages to inform others about traffic jams or accidents. However, the store-carry-and-forward transmission pattern may cause a large transmission delay, making the implementation of large-scale traffic management difficult. In this paper, we put forward a feasible solution to minimize the response time for traffic management service, by enabling real-time content dissemination based on heterogeneous network access in IoV systems. We first design a crowdsensing-based system model for large-scale IoV systems. Then, a cluster-based optimization framework is investigated to provide timely responses for traffic management. Specifically, we estimate the message transmission delay by stochastic theory, which can provide a guideline for the next-hop relay selection in our delay-sensitive routing scheme. Furthermore, network performances are evaluated based on two city-road maps, and performance metrics, containing average delivery delay, average delivery ratio, average communication cost, and access ratio, demonstrate the superiority of our system. Finally, we conclude our work and discuss the further work.

In this paper, we used electroencephalography (EEG)-eye movement (EM) synchronization acquisition network to simultaneously record both EEG and EM physiological signals of the mild depression and normal controls during free viewing. Then, we consider a multimodal feature fusion method that can best discriminate between mild depression and normal control subjects as a step toward achieving our long-term aim of developing an objective and effective multimodal system that assists doctors during diagnosis and monitoring of mild depression. Based on the multimodal denoising autoencoder, we use two feature fusion strategies (feature fusion and hidden layer fusion) for fusion of the EEG and EM signals to improve the recognition performance of classifiers for mild depression. Our experimental results indicate that the EEG-EM synchronization acquisition network ensures that the recorded EM and EEG data require that both the data streams are synchronized with millisecond precision, and both fusion methods can improve the mild depression recognition accuracy, thus demonstrating the complementary nature of the modalities. Compared with the unimodal classification approach that uses only EEG or EM, the feature fusion method slightly improved the recognition accuracy by 1.88%, while the hidden layer fusion method significantly improved the classification rate by up to 7.36%. In particular, the highest classification accuracy achieved in this paper was 83.42%. These results indicate that the multimodal deep learning approaches with input data using a combination of EEG and EM signals are promising in achieving real-time monitoring and identification of mild depression.

With the expansion of the IoT, it is important to optimize available bandwidth to reliably support edge to device communications. Thus we propose a wireless network where each edge server communicates with its end devices using its wireless band as a primary channel, assisted by a secondary edge server that can relay communications via its own wireless band as a secondary channel. The network can optimize capacity by balancing load between primary and secondary wireless bands, and we analyze the geometry of achievable rate regions, depending on the state of bands modeled as Rayleigh fading channels. The allocation of a connection to the primary or secondary band is formulated as an optimization problem which is then solved, and illustrated with numerical examples.

With the expansion of the IoT, it is important to optimize available bandwidth to reliably support edge to device communications. Thus we propose a wireless network where each edge server communicates with its end devices using its wireless band as a primary channel, assisted by a secondary edge server that can relay communications via its own wireless band as a secondary channel. The network can optimize capacity by balancing load between primary and secondary wireless bands, and we analyze the geometry of achievable rate regions, depending on the state of bands modeled as Rayleigh fading channels. The allocation of a connection to the primary or secondary band is formulated as an optimization problem which is then solved, and illustrated with numerical examples.

Facial expression recognition has become a newly-emerging topic in recent decades, which has important value in the field of human-computer interaction. In this paper, we present a deep learning based approach, named frequency neural network (FreNet), for facial expression recognition. Different from convolutional neural network in spatial domain, FreNet inherits the advantages of processing image in frequency domain, such as efficient computation and spatial redundancy elimination. First, we propose the learnable multiplication kernel and construct multiple multiplication layers to learn features in frequency domain. Second, a summarization layer is proposed following multiplication layers to further yield high-level features. Third, based on the property of discrete cosine transform (DCT), we utilize multiplication layers and summarization layer to construct the Basic-FreNet, which can yield high-level features on the widely used DCT feature. Finally, to further achieve better performance on Basic-FreNet, we propose the Block-FreNet in which the weight-shared multiplication kernel is designed for feature learning and the block sub-sampling is designed for dimension reduction. The experimental results show that the Block-FreNet not only achieves superior performance, but also greatly reduces the computational cost. To our best knowledge, the proposed approach is the first attempt to fill in the blank of frequency based deep learning model for facial expression recognition.

The Internet of Medical Things (IoMT) aims to exploit the Internet-of-Things (IoT) techniques to provide better medical treatment scheme for patients with smart, automatic, timely, and emotion-aware clinical services. One of the IoMT instances is applying IoT techniques to sleep-aware smartphones or wearable devices’ applications to provide better sleep healthcare services. As we all know, sleep is vital to our daily health. What is more, studies have shown a strong relationship between sleep difficulties and various diseases such as COVID-19. Therefore, leveraging IoT techniques to develop a longer lifetime sleep healthcare IoMT system, with a tradeoff between data transferring/processing speed and battery energy efficiency, to provide longer time services for bad sleep condition persons, especially the COVID-19 patients or survivors, is a meaningful research topic. In this study, we propose an IoT-enabled sleep data fusion networks (SDFN) module with a star topology Bluetooth network to fuse data of sleep-aware applications. A machine learning model is built to detect sleep events through an audio signal. We design two data reprocessing mechanisms running on our IoT devices to alleviate the data jam problem and save the IoT devices’ battery energy. The experiments manifest that the presented module and mechanisms can save the energy of the system and alleviate the data jam problem of the device.

Classification of Alzheimer's Disease (AD) has been becoming a hot issue along with the rapidly increasing number of patients. This task remains tremendously challenging due to the limited data and the difficulties in detecting mild cognitive impairment (MCI). Existing methods use gait [or EEG (electroencephalogram)] data only to tackle this task. Although the gait data acquisition procedure is cheap and simple, the methods relying on gait data often fail to detect the slight difference between MCI and AD. The methods that use EEG data can detect the difference more precisely, but collecting EEG data from both HC (health controls) and patients is very time-consuming. More critically, these methods often convert EEG records into the frequency domain and thus inevitably lose the spatial and temporal information, which is essential to capture the connectivity and synchronization among different brain regions. This paper proposes a cascade neural network with two steps to achieve a faster and more accurate AD classification by exploiting gait and EEG data simultaneously. In the first step, we propose attention-based spatial temporal graph convolutional networks to extract the features from the skeleton sequences (i.e., gait) captured by Kinect (a commonly used sensor) to distinguish between HC and patients. In the second step, we propose spatial temporal convolutional networks to fully exploit the spatial and temporal information of EEG data and classify the patients into MCI or AD eventually. We collect gait and EEG data from 35 cognitively health controls, 35 MCI, and 17 AD patients to evaluate our proposed method. Experimental results show that our method significantly outperforms other AD diagnosis methods (91.07 vs. 68.18%) in the three-way AD classification task (HC, MCI, and AD). Moreover, we empirically found that the lower body and right upper limb are more important for the early diagnosis of AD than other body parts. We believe this interesting finding can be helpful for clinical researches.

Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. Existing solutions either rely on hand-crafted descriptors or supervised gait representation learning. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID. Specifically, we first create self-supervision by learning to reconstruct unlabeled skeleton sequences reversely, which involves richer high-level semantics to obtain better gait representations. Other pretext tasks are also explored to further improve self-supervised learning. Second, inspired by the fact that motion's continuity endows adjacent skeletons in one skeleton sequence and temporally consecutive skeleton sequences with higher correlations (referred as locality in 3D skeleton data), we propose a locality-aware attention mechanism and a locality-aware contrastive learning scheme, which aim to preserve locality-awareness on intra-sequence level and inter-sequence level respectively during self-supervised learning. Last, with context vectors learned by our locality-aware attention mechanism and contrastive learning scheme, a novel feature named Constrastive Attention-based Gait Encodings (CAGEs) is designed to represent gait effectively. Empirical evaluations show that our approach significantly outperforms skeleton-based counterparts by 15-40% Rank-1 accuracy, and it even achieves superior performance to numerous multi-modal methods with extra RGB or depth information. Our codes are available at https://github.com/Kali-Hac/Locality-Awareness-SGE., Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI). Journal version of https://www.ijcai.org/proceedings/2020/0125 (IJCAI 2020). Codes are available at https://github.com/Kali-Hac/Locality-Awareness-SGE. arXiv admin note: text overlap with arXiv:2008.09435

Human gait refers to a daily motion that represents not only mobility, but it can also be used to identify the walker by either human observers or computers. Recent studies reveal that gait even conveys information about the walker's emotion. Individuals in different emotion states may show different gait patterns. The mapping between various emotions and gait patterns provides a new source for automated emotion recognition. Compared to traditional emotion detection biometrics, such as facial expression, speech and physiological parameters, gait is remotely observable, more difficult to imitate, and requires less cooperation from the subject. These advantages make gait a promising source for emotion detection. This article reviews current research on gait-based emotion detection, particularly on how gait parameters can be affected by different emotion states and how the emotion states can be recognized through distinct gait patterns. We focus on the detailed methods and techniques applied in the whole process of emotion recognition: data collection, preprocessing, and classification. At last, we discuss possible future developments of efficient and effective gait-based emotion recognition using the state of the art techniques on intelligent computation and big data.

Gait-based person re-identification (Re-ID) is valuable for safety-critical applications, and using only 3D skeleton data to extract discriminative gait features for person Re-ID is an emerging open topic. Existing methods either adopt hand-crafted features or learn gait features by traditional supervised learning paradigms. Unlike previous methods, we for the first time propose a generic gait encoding approach that can utilize unlabeled skeleton data to learn gait representations in a self-supervised manner. Specifically, we first propose to introduce self-supervision by learning to reconstruct input skeleton sequences in reverse order, which facilitates learning richer high-level semantics and better gait representations. Second, inspired by the fact that motion's continuity endows temporally adjacent skeletons with higher correlations ("locality"), we propose a locality-aware attention mechanism that encourages learning larger attention weights for temporally adjacent skeletons when reconstructing current skeleton, so as to learn locality when encoding gait. Finally, we propose Attention-based Gait Encodings (AGEs), which are built using context vectors learned by locality-aware attention, as final gait representations. AGEs are directly utilized to realize effective person Re-ID. Our approach typically improves existing skeleton-based methods by 10-20% Rank-1 accuracy, and it achieves comparable or even superior performance to multi-modal methods with extra RGB or depth information. Our codes are available at https://github.com/Kali-Hac/SGE-LA., Comment: Accepted at IJCAI 2020 Main Track. Sole copyright holder is IJCAI. Codes are available at https://github.com/Kali-Hac/SGE-LA

As an emerging platform based on ITS, SIoV is promising for applications of traffic management and road safety in smart cities. However, the end-to-end delay is large in store-carry-and-forward-based vehicular networks, which has become the main obstacle for the implementation of large-scale SIoV. With the extensive applications of mobile devices, crowdsensing is promising to enable real-time content dissemination in a city-wide traffic management system. This article first provides an overview of several promising research areas for traffic management in SIoV. Given the significance of traffic management in urban areas, we investigate a crowdsensing-based framework to provide timely response for traffic management in heterogeneous SIoV. The participant vehicles based on D2D communications integrate trajectory and topology information to dynamically regulate their social behaviors according to network conditions. A real-world taxi trajectory analysis-based performance evaluation is provided to demonstrate the effectiveness of the designed framework. Furthermore, we discuss several future research challenges before concluding our work.

Aggregates are the basic structural units of soils. Fine soil aggregates are crucial pools for the retention of heavy metals. In this study, we evaluated the accumulation characteristics of exogenous Cu in the2 μm aggregate fractions from a Histosol for an aging of 28 months by nano-scale secondary ion mass spectrometry (NanoSIMS) and synchrotron micro-X-ray fluorescence (micro-XRF). Results showed that the correlations between Cu and Fe/Al/Mn increased from 0.10-0.17 to 0.55-0.63, while those between Cu and C decreased sharply from 0.61 to 0.10 and then increased to 0.36, indicating that exogenous Cu tended to accumulate on inorganic mineral components. The carbon NEXAFS data suggested that the relative content of aromatic and carboxyl carbon decreased from 8.1% and 30.8% to 3.6% and 17.8% at month 12, and increased to 5.9% and 26.0% at month 28, respectively. However, an opposite trend was found for alkyl and carbonyl carbon which showed an increase at month 12 followed by a decrease afterwards. The consistency for the correlations between Cu and C with the changes of aromatic and carboxyl carbon indicated their key roles in the binding of Cu on the organic components in the2 μm aggregates of Histosol. These direct observations offer a better understanding on the interactions of heavy metals with various soil components which is critical for the risk assessment and fate evaluation of exogenous Cu in soil ecosystems.