Search

Human activity recognition (HAR) using drone-mounted cameras has attracted considerable interest from the computer vision research community in recent years. A robust and efficient HAR system has a pivotal role in fields like video surveillance, crowd behavior analysis, sports analysis, and human-computer interaction. What makes it challenging are the complex poses, understanding different viewpoints, and the environmental scenarios where the action is taking place. To address such complexities, in this paper, we propose a novel Sparse Weighted Temporal Attention (SWTA) module to utilize sparsely sampled video frames for obtaining global weighted temporal attention. The proposed SWTA is comprised of two parts. First, temporal segment network that sparsely samples a given set of frames. Second, weighted temporal attention, which incorporates a fusion of attention maps derived from optical flow, with raw RGB images. This is followed by a basenet network, which comprises a convolutional neural network (CNN) module along with fully connected layers that provide us with activity recognition. The SWTA network can be used as a plug-in module to the existing deep CNN architectures, for optimizing them to learn temporal information by eliminating the need for a separate temporal stream. It has been evaluated on three publicly available benchmark datasets, namely Okutama, MOD20, and Drone-Action. The proposed model has received an accuracy of 72.76%, 92.56%, and 78.86% on the respective datasets thereby surpassing the previous state-of-the-art performances by a margin of 25.26%, 18.56%, and 2.94%, respectively., arXiv admin note: substantial text overlap with arXiv:2211.05531

Group Activity Recognition requires spatiotemporal modeling of an exponential number of semantic\ud and geometric relations among various individuals in a\ud scene. Previous attempts model these relations by aggregating independently derived spatial and temporal features. This increases the modeling complexity and results in sparse information due to lack of feature correlation. In this paper, we propose Weighted Temporal Attention Mechanism (WTM), a representational mechanism that combines spatial and temporal\ud features of a local subset of a visual sequence into a single 2D image representation, highlighting areas of a frame where actor motion is significant. Pairwise dense optical flow maps representing the temporal characteristic of individuals over a sequence are used as attention masks over raw RGB images through a multi-layer weighted aggregation. We demonstrate a strong correlation between spatial and temporal features, which\ud helps localize actions effectively in a multi-person scenario. The simplicity of the input representation allows the model to be trained by 2D image classification architectures in a plug-and-play fashion, which outperforms its multi-stream and multi-dimensional counterparts. The proposed method achieves the lowest computational complexity in comparison to other works. We demonstrate the performance of WTM on two widely used public benchmark datasets, namely the Collective\ud Activity Dataset (CAD) and the Volleyball Dataset. and achieve state-of-the-art accuracies of 95.1% and 94.6% respectively. We also discuss the application of this method to other datasets and general scenarios. The code is being made publicly available.

Human Activity Recognition is an important task in Computer Vision that involves the utilization of spatio-temporal features of videos to classify human actions. The temporal portion of videos contains vital information needed for accurate classification. However, common Deep Learning methods simply average the temporal features, thereby giving all frames equal importance\ud irrespective of their relevance, which negatively impacts\ud the accuracy of the model. To combat this adverse effect, this paper proposes a novel Transformer Based Attention Consensus (TBAC) module. The TBAC module can be used in a plug-and play manner as an alternate to the conventional consensus methods of any existing video action recognition network. The TBAC module contains four components: (i) Query Sampling Unit, (ii) Attention Extraction Unit, (iii) Softening Unit, and (iv) Attention\ud Consensus Unit. Our experiments demonstrate that the use of the TBAC module in place of classical consensus can improve the performance of the CNN-based action recognition models, such as Channel Separated Convolutional Network (CSN), Temporal Shift Module (TSM), and Temporal Segment Network (TSN). We also propose the Decision Consensus (DC) algorithm that\ud utilizes multiple independent but related action recognizer models in order to improve upon the performance of most of these constituent models, using a novel fusion algorithm. Results have been obtained on two benchmark human action recognition datasets, HMDB51 and HAA500. The use of the proposed TBAC\ud module along with Decision Consensus achieves state-of-the-art performances, with 85.23% and 83.73% classification accuracies on the two databases HMDB51 and HAA500, respectively. The code will be made publicly available.

Classifying complex human motion sequences is a major research challenge in the domain of human activity recognition. Currently, most popular datasets lack a specialized set of classes pertaining to similar action sequences (in terms of spatial trajectories). To recognize such complex action sequences with high inter-class similarity, such as those in karate, multiple streams are required. To fulfill this need, we propose MS-KARD, a Multi-Stream Karate Action Recognition Dataset that uses\ud multiple vision perspectives, as well as sensor data - accelerometer and gyroscope. It includes 1518 video clips along with their corresponding sensor data. Each video was shot at 30fps and lasts around one minute, equating to a total of 2,814,930 frames and 5,623,734 sensor data samples. The dataset has been collected for 23 classes like Jodan Zuki, Oi Zuki, etc. The data acquisition setting involves the combination of 2 orthogonal web cameras and 3 wearable inertial sensors recording both vision and inertial data respectively. The aim of this dataset is to aid research that deals with recognizing human actions that have similar spatial trajectories. The paper describes statistics of the dataset, acquisition setting, and provides baseline performance figures using popular action recognizers. We propose an ensemble-based method, KarateNet, that performs decision-level fusion on the\ud two input modalities (vision and sensor data) to classify actions. For the first stream, the RGB frames are extracted from the videos and passed into action recognition networks like Temporal Segment Network (TSN) and Temporal Shift Module (TSM). For the second stream, the sensor data is converted into a 2- D image and fed into a Convolutional Neural Network (CNN).\ud The results reported were obtained on performing a fusion of the 2 streams. We also report results on ablations that use fusion with various input settings. The dataset and code will be made publicly available.

Routing Protocol (RPL) is treated as a standard protocol for Low power and Lossy Networks (LLNs). It was introduced by the Internet Engineering Task Force (IETF) Routing over Low Power and Lossy network (ROLL) working group to deal with routing challenges that occur in LLNs. It is noted that RPL permits optimization at several levels in the networks. RPL uses an objective function that helps in evaluating network performance. The objective function can be created using single or composite metrics. Literature reveals that single metrics-based objective function showed poor performance whilst composite metrics have demonstrated excellent performance, but still there is plenty of scope for further improvement. This paper shows an extension of the composite metrics. The real problem of RPL concerning IoV is that the heterogeneous network undergoes extreme packet loss and congestion which disables the full utilization of network capacity. Thus, this paper presents an enhanced fuzzy-based objective function and analyzes its impact on the Internet of Vehicles (IoV) network. The objective function aims at reducing the Control Trafc Overhead (CTO) in the network and providing a high Packet Delivery Ratio (PDR).The contribution of this paper is as follows: First, network scalability is analyzed for (a) random and grid confgurations; (b) random and Self-similar Least Action Walk (SLAW) mobility models. Second, the performance of the proposed E-MOF is validated with the standard objective functions for both networks. Third, extensive computer simulations are performed for performance analysis. Simulation result reveals that the proposed E-MOF outperform OF-EC concerning the PDR, CTO and comparable latency at the expense of high energy consumption for both configurations and mobility models.\ud Finally, it is remarked that E-MOF extends the applicability of the RPL for IoV networks. Further, it commits a better PDR Quality of Service (QoS) and high network reliability. The results and discussion reported in this paper are outstanding, therefore, they will motivate other researchers to develop a novel approach in the future.

Correction: Cluster Computing \ud https://doi.org/10.1007/s10586-022-03689-x\ud \ud Linked to http://eprints.bournemouth.ac.uk/37417/

The Routing Protocol for Low power and Lossy networks (RPL) utilises the Objective Function (OF) to form a Destination Oriented Directed Acyclic Graph (DODAG) to reach the destination by selecting the best path. Many works in literature have explored this domain concerning the Internet of Things (IoT) applications. Although, the application of RPL protocol from IoT to the Internet of Vehicles (IoV) in the smart city still presents a big test. Since this gap has not been much traversed, it motivated us to present our findings on this research gap. This paper has realised the transition of RPL protocol from IoT to IoV for the first time. The network performance has been analysed using RPL in a static and mobile environment based on three configurations: Quality of Service (QoS) parameters, network scalability and mobility models. Also, a comprehensive analysis of the RPL performance in both environments has been bestowed in our paper. Finally, we have summarised our inputs and stated potential future directions for researchers. The experiments have been performed using Contiki OS/Cooja Simulator, BonnMotion tool and Wireshark. Simulation results have shown that Self-similar Least Action Walk (SLAW) has outperformed Random Way-Point (RWP) and Nomadic mobility model. High value of Packet Delivery Ratio (PDR) is achieved in mobile/dynamic environment than static. These findings can be directly applied to IoV and IoT applications using RPL protocol like Traffic Monitoring System (TMS), smart corridors, Electronic Toll Collection (ETC), etc. in smart city. Moreover, this article will help the researchers in gaining a better insight of RPL protocol in static and mobile environments for future works.

Publisher Copyright: IEEE DNA methylation is an epigenetic marker, that plays an important role in the biological processes of regulating gene expression, maintaining chromatin structure, imprinting genes, inactivating X chromosomes, and developing embryos. The traditional detection method is time-consuming. Currently, researchers have used effective computational methods to improve the efficiency of methylation detection. This study proposes a fuzzy model with correntropy induced loss (C-loss) function to identify DNA N4-methylcytosine (4mC) sites. To improve the robustness and performance of the model, we use kernel method and the C-loss function to build a higher-order fuzzy inference system (HFIS). To test performance, our model is implemented on six 4mC and eight UCI data sets. The experimental results show that our model achieves better prediction performance.

openaire: EC/H2020/721321/EU//QUARTZ Sarcasm detection in conversation (SDC), a theoretically and practically challenging artificial intelligence (AI) task, aims to discover elusively ironic, contemptuous and metaphoric information implied in daily conversations. Most of the recent approaches in sarcasm detection have neglected the intrinsic vagueness and uncertainty of human language in emotional expression and understanding. To address this gap, we propose a complex-valued fuzzy network (CFN) by leveraging the mathematical formalisms of quantum theory (QT) and fuzzy logic. In particular, the target utterance to be recognized is considered as a quantum superposition of a set of separate words. The contextual interaction between adjacent utterances is described as the interaction between a quantum system and its surrounding environment, constructing the quantum composite system, where the weight of interaction is determined by a fuzzy membership function. In order to model both the vagueness and uncertainty, the aforementioned superposition and composite systems are mathematically encapsulated in a density matrix. Finally, a quantum fuzzy measurement is performed on the density matrix of each utterance to yield the probabilistic outcomes of sarcasm recognition. Extensive experiments are conducted on the MUStARD and the 2020 sarcasm detection Reddit track datasets, and the results show that our model outperforms a wide range of strong baselines.

Publisher Copyright: IEEE Recently, unsupervised cross-dataset person reidentification (Re-ID) has attracted more and more attention, which aims to transfer knowledge of a labeled source domain to an unlabeled target domain. There are two common frameworks: one is pixel-alignment of transferring low-level knowledge, and the other is feature-alignment of transferring high-level knowledge. In this article, we propose a novel recurrent autoencoder (RAE) framework to unify these two kinds of methods and inherit their merits. Specifically, the proposed RAE includes three modules, i.e., a feature-transfer (FT) module, a pixel-transfer (PT) module, and a fusion module. The FT module utilizes an encoder to map source and target images to a shared feature space. In the space, not only features are identity-discriminative but also the gap between source and target features is reduced. The PT module takes a decoder to reconstruct original images with its features. Here, we hope that the images reconstructed from target features are in the sourcestyle. Thus, the low-level knowledge can be propagated to the target domain. After transferring both high- and low-level knowledge with the two proposed modules above, we design another bilinear pooling layer to fuse both kinds of knowledge. Extensive experiments on Market-1501, DukeMTMC-ReID, and MSMT17 datasets show that our method significantly outperforms either pixel-alignment or feature-alignment Re-ID methods and achieves new state-of-the-art results.

Babaee Tirkolaee, Erfan/0000-0003-1664-9210 The integration of big data analytics and cognitive computing results in a new model that can provide the utilization of the most complicated advances in industry and its relevant decision-making processes as well as resolving failures faced during big data analytics. In E-projects portfolio selection (EPPS) problem, big data-driven decision-making has a great importance in web development environments. EPPS problem deals with choosing a set of the best investment projects on social media such that maximum return with minimum risk is achieved. To optimize the EPPS problem on social media, this study aims to develop a hybrid fuzzy multi-objective optimization algorithm, named as NSGA-III-MOIWO encompassing the non-dominated sorting genetic algorithm III (NSGA-III) and multi-objective invasive weed optimization (MOIWO) algorithms. The objectives are to simultaneously minimize variance, skewness and kurtosis as the risk measures and maximize the total expected return. To evaluate the performance of the proposed hybrid algorithm, the data derived from 125 active E-projects in an Iranian web development company are analyzed and employed over the period 2014-2018. Finally, the obtained experimental results provide the optimal policy based on the main limitations of the system and it is demonstrated that the NSGA-III-MOIWO outperforms the NSGA-III and MOIWO in finding efficient investment boundaries in EPPS problems. Finally, an efficient statistical-comparative analysis is performed to test the performance of NSGA-III-MOIWO against some well-known multi-objective algorithms. Key-Area Research and Development Program of Guangdong Province [2019B010136001]; National Key Research and Development Plan [2017YFB0801801]; National Natural Science Foundation of China (NSFC)National Natural Science Foundation of China (NSFC) [61672186, 61872110] This work was supported in part by the Key-Area Research and Development Program of Guangdong Province under Grant 2019B010136001, in part by the National Key Research and Development Plan under Grant 2017YFB0801801, and in part by the National Natural Science Foundation of China (NSFC) under Grant 61672186 and Grant 61872110. WOS:000549502200044 Q2

Federated Learning is a framework that jointly trains a model \textit{with} complete knowledge on a remotely placed centralized server, but \textit{without} the requirement of accessing the data stored in distributed machines. Some work assumes that the data generated from edge devices are identically and independently sampled from a common population distribution. However, such ideal sampling may not be realistic in many contexts. Also, models based on intrinsic agency, such as active sampling schemes, may lead to highly biased sampling. So an imminent question is how robust Federated Learning is to biased sampling? In this work\footnote{\url{https://github.com/jiaqian/robustness_of_FL}}, we experimentally investigate two such scenarios. First, we study a centralized classifier aggregated from a collection of local classifiers trained with data having categorical heterogeneity. Second, we study a classifier aggregated from a collection of local classifiers trained by data through active sampling at the edge. We present evidence in both scenarios that Federated Learning is robust to data heterogeneity when local training iterations and communication frequency are appropriately chosen.

Traditional deep learning methods are suboptimal in classifying ambiguity features, which often arise in noisy and hard to predict categories, especially, to distinguish semantic scoring. Semantic scoring, depending on semantic logic to implement evaluation, inevitably contains fuzzy description and misses some concepts, for example, the ambiguous relationship between normal and probably normal always presents unclear boundaries (normal—more likely normal—probably normal). Thus, human error is common when annotating images. Differing from existing methods that focus on modifying kernel structure of neural networks, this article proposes a dominant fuzzy fully connected layer (FFCL) for breast imaging reporting and data system (BI-RADS) scoring and validates the universality of this proposed structure. This proposed model aims to develop complementary properties of scoring for semantic paradigms, while constructing fuzzy rules based on analyzing human thought patterns, and to particularly reduce the influence of semantic conglutination. Specifically, this semantic-sensitive defuzzifier layer projects features occupied by relative categories into semantic space, and a fuzzy decoder modifies probabilities of the last output layer referring to the global trend. Moreover, the ambiguous semantic space between two relative categories shrinks during the learning phases, as the positive and negative growth trends of one category appearing among its relatives were considered. We first used the Euclidean distance to zoom in the distance between the real scores and the predicted scores, and then employed two sample t test method to evidence the advantage of the FFCL architecture. Extensive experimental results performed on the curated breast imaging subset of digital database of screening mammography dataset show that our FFCL structure can achieve superior performances for both triple and multiclass classification in BI-RADS scoring, outperforming the state-of-the-art methods.

Refereed/Peer-reviewed The Semantic Internet of Things (SIoT) is the extension of the Internet of Things (IoT) and the Semantic Web, which aims to build an interoperable collaborative system to solve the heterogeneous problems in the IoT. However, the SIoT has the characteristics of both the IoT and the Semantic Web environment, and the corresponding semantic data present many new data features. In this article, we analyze the characteristics of semantic data and propose the concept of a uniform knowledge graph (UKG), allowing us to be applied to the environment of the SIoT better. Here, we design a semantic collaboration method based on a UKG. It can take the UKG as the form of knowledge organization and representation, and provide a useful data basis for semantic collaboration by constructing the semantic links to complete semantic relation between different data sets, to achieve the semantic collaboration in the SIoT. Our experiments show that the proposed method can analyze and understand the semantics of user requirements better and provide more satisfactory outcomes. usc

The Internet-of-Things (IoT) healthcare system monitors a patients’ condition and takes preventive measures in case of an emergency. The electrocardiogram (ECG) that measures the electrical activity of the heart is one of the important health indicators. Thanks to the wearable technology, nowadays, we can even measure the ECG using smart portable devices and send via a wireless channel. However, this wireless transmission has to minimize both energy and memory consumption. In this article, we propose CULT—an ECG compression technique using unsupervised dictionary learning. Our method achieves a high compression rate due to the essence of dictionary learning and is immune to the noise by integrating discrete cosine transformation. Moreover, it continuously expands the dictionary when the unseen pattern occurs and refines the dictionary when new input arrives, by imposing the double dictionary scheme. We show that our method has a better performance by comparing it with the other existing approaches.

Deep learning (DL) networks are composed of multiple processing layers that learn data representations with multiple levels of abstraction. In recent years, DL networks have significantly improved the state of the art across different domains, including speech processing, text mining, pattern recognition, object detection, robotics, and big data analytics. Generally, a researcher or practitioner who is planning to use DL networks for the first time faces difficulties in selecting suitable software tools. The present article provides a comprehensive list and taxonomy of current programming languages and software tools that can be utilized for implementation of DL networks. The motivation of this article is hence to create awareness among researchers, especially beginners, regarding the various languages and interfaces that are available to implement deep learning and to provide a simplified ontological basis for selecting between them.

A technique is developed using Self Organizing Maps (SOM) to efficiently cluster the data and it is compared with existing clustering Techniques such as K-Means clustering, Hierarchical clustering and SOM Clustering. The proposed technique is used to cluster an Earthquake dataset and the performance is compared with the other existing clustering technique. The experimental results show that the proposed clustering method demonstrated better results as compared to other clustering methods.

Security is the prominent concern for the network and maintaining security is highly recommended. There exists several approaches has been attempted to address this challenging task. This paper presents the applicability of the genetic algorithm (GA) for security concerns. The working of the GA heavily depends on the various factors includes: reproduction operators, selection techniques, chromosome representation and problem type. There exist several selection methods presents play a vital role, but identifying the suitable one is a grand and persistent challenge. In this paper, a comparison of various selection techniques in the GA has been reported. The GA utilizes operators: crossover, mutation and selection to guide the searching in an iterative manner. A significant work has been conducted explains the importance of crossover and mutation probabilities, but very few researchers (some of them has shown the comparison of selection methods) presented the importance of selection approaches. The comparison of three: Rank based, Roulette wheel and Tournament selection techniques have been presented in this paper over Travelling Salesman Problem. Computational experiments have been conducted and results are collected considering distance should be as minimum as possible. Statistical tests (Paired T-Test and two ways ANOVA) are conducted to report the performance significance of selection techniques considered.