Search

As a neural network's depth increases, it can achieve strong generalization performance. Training, however, becomes challenging due to gradient issues. Theoretical research and various methods have been introduced to address this issues. However, research on weight initialization methods that can be effectively applied to tanh neural networks of varying sizes still needs to be completed. This paper presents a novel weight initialization method for Feedforward Neural Networks with tanh activation function. Based on an analysis of the fixed points of the function $\tanh(ax)$, our proposed method aims to determine values of $a$ that prevent the saturation of activations. A series of experiments on various classification datasets demonstrate that the proposed method is more robust to network size variations than the existing method. Furthermore, when applied to Physics-Informed Neural Networks, the method exhibits faster convergence and robustness to variations of the network size compared to Xavier initialization in problems of Partial Differential Equations.

The success of a specific neural network architecture is closely tied to the dataset and task it tackles; there is no one-size-fits-all solution. Thus, considerable efforts have been made to quickly and accurately estimate the performances of neural architectures, without full training or evaluation, for given tasks and datasets. Neural architecture encoding has played a crucial role in the estimation, and graphbased methods, which treat an architecture as a graph, have shown prominent performance. For enhanced representation learning of neural architectures, we introduce FlowerFormer, a powerful graph transformer that incorporates the information flows within a neural architecture. FlowerFormer consists of two key components: (a) bidirectional asynchronous message passing, inspired by the flows; (b) global attention built on flow-based masking. Our extensive experiments demonstrate the superiority of FlowerFormer over existing neural encoding methods, and its effectiveness extends beyond computer vision models to include graph neural networks and auto speech recognition models. Our code is available at http://github.com/y0ngjaenius/CVPR2024_FLOWERFormer., Comment: CVPR 2024 Camera-Ready

In ambient assisted living, human activity recognition from time series sensor data mainly focuses on predefined activities, often overlooking new activity patterns. We propose CLAN, a two-tower contrastive learning-based novelty detection framework with diverse types of negative pairs for human activity recognition. It is tailored to challenges with human activity characteristics, including the significance of temporal and frequency features, complex activity dynamics, shared features across activities, and sensor modality variations. The framework aims to construct invariant representations of known activity robust to the challenges. To generate suitable negative pairs, it selects data augmentation methods according to the temporal and frequency characteristics of each dataset. It derives the key representations against meaningless dynamics by contrastive and classification losses-based representation learning and score function-based novelty detection that accommodate dynamic numbers of the different types of augmented samples. The proposed two-tower model extracts the representations in terms of time and frequency, mutually enhancing expressiveness for distinguishing between new and known activities, even when they share common features. Experiments on four real-world human activity datasets show that CLAN surpasses the best performance of existing novelty detection methods, improving by 8.3%, 13.7%, and 53.3% in AUROC, balanced accuracy, and FPR@TPR0.95 metrics respectively.

Human activity recognition (HAR) is a key challenge in pervasive computing and its solutions have been presented based on various disciplines. Specifically, for HAR in a smart space without privacy and accessibility issues, data streams generated by deployed pervasive sensors are leveraged. In this paper, we focus on a group activity by which a group of users perform a collaborative task without user identification and propose an efficient group activity recognition scheme which extracts causality patterns from pervasive sensor event sequences generated by a group of users to support as good recognition accuracy as the state-of-the-art graphical model. To filter out irrelevant noise events from a given data stream, a set of rules is leveraged to highlight causally related events. Then, a pattern-tree algorithm extracts frequent causal patterns by means of a growing tree structure. Based on the extracted patterns, a weighted sum-based pattern matching algorithm computes the likelihoods of stored group activities to the given test event sequence by means of matched event pattern counts for group activity recognition. We evaluate the proposed scheme using the data collected from our testbed and CASAS datasets where users perform their tasks on a daily basis and validate its effectiveness in a real environment. Experiment results show that the proposed scheme performs higher recognition accuracy and with a small amount of runtime overhead than the existing schemes.

A directed hypergraph, which consists of nodes and hyperarcs, is a higher-order data structure that naturally models directional group interactions (e.g., chemical reactions of molecules). Although there have been extensive studies on local structures of (directed) graphs in the real world, those of directed hypergraphs remain unexplored. In this work, we focus on measurements, findings, and applications related to local structures of directed hypergraphs, and they together contribute to a systematic understanding of various real-world systems interconnected by directed group interactions. Our first contribution is to define 91 directed hypergraphlets (DHGs), which disjointly categorize directed connections and overlaps among four node sets that compose two incident hyperarcs. Our second contribution is to develop exact and approximate algorithms for counting the occurrences of each DHG. Our last contribution is to characterize 11 real-world directed hypergraphs and individual hyperarcs in them using the occurrences of DHGs, which reveals clear domain-based local structural patterns. Our experiments demonstrate that our DHG-based characterization gives up to 12% and 33% better performances on hypergraph clustering and hyperarc prediction, respectively, than baseline characterization methods. Moreover, we show that CODA-A, which is our proposed approximate algorithm, is up to 32X faster than its competitors with similar characterization quality.

Hypergraphs naturally represent group interactions, which are omnipresent in many domains: collaborations of researchers, co-purchases of items, and joint interactions of proteins, to name a few. In this work, we propose tools for answering the following questions: (Q1) what are the structural design principles of real-world hypergraphs? (Q2) how can we compare local structures of hypergraphs of different sizes? (Q3) how can we identify domains from which hypergraphs are? We first define hypergraph motifs (h-motifs), which describe the overlapping patterns of three connected hyperedges. Then, we define the significance of each h-motif in a hypergraph as its occurrences relative to those in properly randomized hypergraphs. Lastly, we define the characteristic profile (CP) as the vector of the normalized significance of every h-motif. Regarding Q1, we find that h-motifs' occurrences in 11 real-world hypergraphs from 5 domains are clearly distinguished from those of randomized hypergraphs. Then, we demonstrate that CPs capture local structural patterns unique to each domain, and thus comparing CPs of hypergraphs addresses Q2 and Q3. The concept of CP is extended to represent the connectivity pattern of each node or hyperedge as a vector, which proves useful in node classification and hyperedge prediction. Our algorithmic contribution is to propose MoCHy, a family of parallel algorithms for counting h-motifs' occurrences in a hypergraph. We theoretically analyze their speed and accuracy and show empirically that the advanced approximate version MoCHy-A+ is more accurate and faster than the basic approximate and exact versions, respectively. Furthermore, we explore ternary hypergraph motifs that extends h-motifs by taking into account not only the presence but also the cardinality of intersections among hyperedges. This extension proves beneficial for all previously mentioned applications., Comment: Extended version of VLDB 2020 paper arXiv:2003.01853

Simplicial complexes are higher-order combinatorial structures which have been used to represent real-world complex systems. In this paper, we concentrate on the local patterns in simplicial complexes called simplets, a generalization of graphlets. We formulate the problem of counting simplets of a given size in a given simplicial complex. For this problem, we extend a sampling algorithm based on color coding from graphs to simplicial complexes, with essential technical novelty. We theoretically analyze our proposed algorithm named SC3, showing its correctness, unbiasedness, convergence, and time/space complexity. Through the extensive experiments on sixteen real-world datasets, we show the superiority of SC3 in terms of accuracy, speed, and scalability, compared to the baseline methods. Finally, we use the counts given by SC3 for simplicial complex analysis, especially for characterization, which is further used for simplicial complex clustering, where SC3 shows a strong ability of characterization with domain-based similarity., Comment: Accepted to WWW 2023 - The Web Conference 2023. Simplet 0 and Simplet 1 of size 4 have been swapped in Figure 2

Inferring the properties of exoplanets from their atmospheres, while confronting low resolution and low signal-to-noise in the context of the quantities we want to derive, poses rigorous demands upon the data collected from observation. Further compounding this challenge is that inferences of exoplanet properties are built from forward models, which can include errors due to incomplete or inaccurate assumptions in atmospheric physics and chemistry. The confluence of observational noise and model error makes developing techniques to identify predictive features that are robust to both low s/n and model error increasingly important for exoplanet science. We demonstrate how both issues can be addressed simultaneously by taking advantage of underutilized multivariate information already present in current atmospheric models, including thermodynamic statistics and reaction network structure. To do so, we provide a case study of the prediction of vertical mixing (parameterized as eddy diffusion) in hot Jupiter atmospheres and show how prediction efficacy depends on what model information is used - e.g. chemical species abundances, network statistics, and/or thermodynamic statistics. We also show how the variables with the most predictive power vary with planetary properties such as temperature and metallicity. Our results demonstrate how inferences built on single metrics do not have utility across all possible use cases. We also show how statistical measures derived from network analyses tend to be better predictors when accounting for the possibility of missing data or observational uncertainty. We discuss future directions applying multivariate and network model information as a framework for increasing confidence in inferences aimed at extracting features relevant to exoplanet atmospheres and future applications to the detection of life on terrestrial worlds., Comment: 52 pages (including supplement); 10 figures and 2 tables in main text, 17 figures in supplement

Efficient and fast predictor-corrector methods are proposed to deal with nonlinear Caputo-Fabrizio fractional differential equations, where Caputo-Fabrizio operator is a new proposed fractional derivative with a smooth kernel. The proposed methods achieve a uniform accuracy order with the second-order scheme for linear interpolation and the third-order scheme for quadratic interpolation. The convergence analysis is proved by using the discrete Gronwall's inequality. Furthermore, applying the recurrence relation of the memory term, it reduces CPU time executed the proposed methods. The proposed fast algorithm requires approximately $O(N)$ arithmetic operations while $O(N^2)$ is required in case of the regular predictor-corrector schemes, where $N$ is the total number of time step. The following numerical examples demonstrate the accuracy of the proposed methods as well as the efficiency; nonlinear fractional differential equations, time-fraction sub-diffusion, and time-fractional advection-diffusion equation. The theoretical convergence rates are also verified by numerical experiments.

This article proposes new strategies for solving two-point Fractional order Nonlinear Boundary Value Problems(FN-BVPs) with Robin Boundary Conditions(RBCs). In the new numerical schemes, a two-point FNBVP is transformed into a system of Fractional order Initial Value Problems(FIVPs) with unknown Initial Conditions(ICs). To approximate ICs in the system of FIVPs, we develop nonlinear shooting methods based on Newton's method and Halley's method using the RBC at the right end point. To deal with FIVPs in a system, we mainly employ High-order Predictor-Corrector Methods(HPCMs) with linear interpolation and quadratic interpolation into Volterra integral equations which are equivalent to FIVPs. The advantage of proposed schemes with HPCMs is that even though they are designed for solving two-point FNBVPs, they can handle both linear and nonlinear two-point Fractional order Boundary Value Problems(FBVPs) with RBCs and have uniform convergence rates of HPCMs, $O(h^2)$ and $O(h^3)$ for shooting techniques with Newton's method and Halley's method, respectively. A variety of numerical examples are demonstrated to confirm the effectiveness and performance of the proposed schemes. Also we compare the accuracy and performance of our schemes with another method.

On May $28^{th}$ and $29^{th}$, a two day workshop was held virtually, facilitated by the Beyond Center at ASU and Moogsoft Inc. The aim was to bring together leading scientists with an interest in Network Science and Epidemiology to attempt to inform public policy in response to the COVID-19 pandemic. Epidemics are at their core a process that progresses dynamically upon a network, and are a key area of study in Network Science. In the course of the workshop a wide survey of the state of the subject was conducted. We summarize in this paper a series of perspectives of the subject, and where the authors believe fruitful areas for future research are to be found., Comment: Ayan Paul references preprints (DESY 20-142, HU-EP-20/23)

One of the more widely advocated solutions for slowing down the spread of COVID-19 has been automated contact tracing. Since proximity data can be collected by personal mobile devices, the natural proposal has been to use this for automated contact tracing providing a major gain over a manual implementation. In this work, we study the characteristics of voluntary and automated contact tracing and its effectiveness for mapping the spread of a pandemic due to the spread of SARS-CoV-2. We highlight the infrastructure and social structures required for automated contact tracing to work. We display the vulnerabilities of the strategy to inadequate sampling of the population, which results in the inability to sufficiently determine significant contact with infected individuals. Of crucial importance will be the participation of a significant fraction of the population for which we derive a minimum threshold. We conclude that relying largely on automated contact tracing without population-wide participation to contain the spread of the SARS-CoV-2 pandemic can be counterproductive and allow the pandemic to spread unchecked. The simultaneous implementation of various mitigation methods along with automated contact tracing is necessary for reaching an optimal solution to contain the pandemic., Comment: 10 pages and 2 figures

The complex dynamics of gene expression in living cells can be well-approximated using Boolean networks. The average sensitivity is a natural measure of stability in these systems: values below one indicate typically stable dynamics associated with an ordered phase, whereas values above one indicate chaotic dynamics. This yields a theoretically motivated adaptive advantage to being near the critical value of one, at the boundary between order and chaos. Here, we measure average sensitivity for 66 publicly available Boolean network models describing the function of gene regulatory circuits across diverse living processes. We find the average sensitivity values for these networks are clustered around unity, indicating they are near critical. In many types of random networks, mean connectivity and the average activity bias of the logic functions have been found to be the most important network properties in determining average sensitivity, and by extension a network's criticality. Surprisingly, many of these gene regulatory networks achieve the near-critical state with and far from that predicted for critical systems: randomized networks sharing the local causal structure and local logic of biological networks better reproduce their critical behavior than controlling for macroscale properties such as and alone. This suggests the local properties of genes interacting within regulatory networks are selected to collectively be near-critical, and this non-local property of gene regulatory network dynamics cannot be predicted using the density of interactions alone., Comment: 10 pages, 7 figures

Standard techniques for studying biological systems largely focus on their dynamical, or, more recently, their informational properties, usually taking either a reductionist or holistic perspective. Yet, studying only individual system elements or the dynamics of the system as a whole disregards the organisational structure of the system - whether there are subsets of elements with joint causes or effects, and whether the system is strongly integrated or composed of several loosely interacting components. Integrated information theory (IIT), offers a theoretical framework to (1) investigate the compositional cause-effect structure of a system, and to (2) identify causal borders of highly integrated elements comprising local maxima of intrinsic cause-effect power. Here we apply this comprehensive causal analysis to a Boolean network model of the fission yeast (Schizosaccharomyces pombe) cell-cycle. We demonstrate that this biological model features a non-trivial causal architecture, whose discovery may provide insights about the real cell cycle that could not be gained from holistic or reductionist approaches. We also show how some specific properties of this underlying causal architecture relate to the biological notion of autonomy. Ultimately, we suggest that analysing the causal organisation of a system, including key features like intrinsic control and stable causal borders, should prove relevant for distinguishing life from non-life, and thus could also illuminate the origin of life problem., Comment: 15 pages, 4 figures, to appear in Philosophical Transactions of the Royal Society A