Search

We are interested in investigating the statistical properties of extreme values for strongly correlated variables. The starting motivation is to understand how the strong-correlation properties of power-law distributed processes affect the possibility of exploring the whole domain of a stochastic process (the real axis in most cases) when performing time-average numerical simulations and how this relates to the numerical evaluation of the autocorrelation function. We show that correlations decrease the heterogeneity of the maximum values. Specifically, through numerical simulations we observe that for strongly correlated variables whose probability distribution function decays like a power-law $1/x^\alpha$, the maximum distribution has a tail compatible with a $1/x^{\alpha+2}$ decay, while for i.i.d. variables we expect a $1/x^\alpha$ decay. As a consequence, we also show that the numerically estimated autocorrelation function converges to the theoretical prediction according to a factor that depends on the length of the simulated time-series $n$ according to a power-law: $1/n^{\alpha^\delta}$ with $\delta<1$, This accounts for a very slow convergence rate., Comment: 6 pages, 4 figures

We investigate block diagonal and hierarchical nested stochastic multivariate Gaussian models by studying their sample cross-correlation matrix on high dimensions. By performing numerical simulations, we compare a filtered sample cross-correlation with the population cross-correlation matrices by using several rotationally invariant estimators (RIE) and hierarchical clustering estimators (HCE) under several loss functions. We show that at large but finite sample size, sample cross-correlation filtered by RIE estimators are often outperformed by HCE estimators for several of the loss functions. We also show that for block models and for hierarchically nested block models the best determination of the filtered sample cross-correlation is achieved by introducing two-step estimators combining state-of-the-art non-linear shrinkage models with hierarchical clustering estimators., Comment: 14 pages, 6 figures, 6 tables

Many natural phenomena exhibit a stochastic nature that one attempts at modeling by using stochastic processes of different types. In this context, often one is interested in investigating the memory properties of the natural phenomenon at hand. This is usually accomplished by computing the autocorrelation function of the numerical series describing the considered phenomenon. Often, especially when considering real world data, the autocorrelation function must be computed starting from a single numerical series: i.e. with a time-average approach. Hereafter, we will propose a novel way of evaluating the time-average autocorrelation function, based on the preliminary evaluation of the quantity N({\tau},g{\mu},g{\nu}), that, apart from normalization factors, represents a numerical estimate, based on a single realization of the process, of the 2-point joint probability density function P(x2,{\tau};x1,0). The main advantage of the proposed method is that it allows to quantitatively assess what is the error that one makes when numerically evaluating the autocorrelation function due to the fact that any simulated time series is necessarily bounded. In fact, we show that, for a wide class of stochastic processes admitting a nonlinear Langevin equation with white noise and that can be described by using a Fokker-Planck equation, the way the numerical estimate of the autocorrelation function converges to its theoretical prediction depends on the pdf tails. Moreover, the knowledge of N({\tau},g{\mu},g{\nu}) allows to easily compute the process histogram and to characterize processes with multiple timescales. We will show the effectiveness of our new methodology by considering three stochastic processes whose autocorrelation function and two-point probability density function are both known in an analytical or numerical form, thus allowing direct comparisons., Comment: 15 ages, 3 tables, 4 figures

Network dismantling is a relevant research area in network science, gathering attention both from a theoretical and an operational point of view. Here, we propose a general framework for dismantling that prioritizes the removal of nodes that bridge together different network communities. The strategies we detect are not unique, as they depend on the specific realization of the community detection algorithm considered. However, when applying the methodology to some real-world networks we find that the percolation threshold at which dismantling occurs is strongly robust, and it does not depend on the specific algorithm. Thus, the stochasticity inherently present in many community detection algorithms allows to identify several strategies that have comparable effectiveness but require the removal of distinct subsets of nodes. This feature is highly relevant in operational contexts in which the removal of nodes is costly and allows to identify the least expensive strategy that still holds high effectiveness., Comment: 9 pages, 5 figures, 2 tables

We develop a greedy algorithm that is fast and scalable in the detection of a nested partition extracted from a dendrogram obtained from hierarchical clustering of a multivariate series. Our algorithm provides a $p$-value for each clade observed in the hierarchical tree. The $p$-value is obtained by computing a number of bootstrap replicas of the dissimilarity matrix and by performing a statistical test on each difference between the dissimilarity associated with a given clade and the dissimilarity of the clade of its parent node. We prove the efficacy of our algorithm with a set of benchmarks generated by using a hierarchical factor model. We compare the results obtained by our algorithm with those of Pvclust. Pvclust is a widely used algorithm developed with a global approach originally motivated by phylogenetic studies. In our numerical experiments we focus on the role of multiple hypothesis test correction and on the robustness of the algorithms to inaccuracy and errors of datasets. We also apply our algorithm to a reference empirical dataset. We verify that our algorithm is much faster than Pvclust algorithm and has a better scalability both in the number of elements and in the number of records of the investigated multivariate set. Our algorithm provides a hierarchically nested partition in much shorter time than currently widely used algorithms allowing to perform a statistically validated cluster analysis detection in very large systems.

In this contribution we discuss some approaches of network analysis providing information about single links or single nodes with respect to a null hypothesis taking into account the heterogeneity of the system empirically observed. With this approach, a selection of nodes and links is feasible when the null hypothesis is statistically rejected. We focus our discussion on approaches using (i) the so-called disparity filter and (ii) statistically validated network in bipartite networks. For both methods we discuss the importance of using multiple hypothesis test correction. Specific applications of statistically validated networks are discussed. We also discuss how statistically validated networks can be used to (i) pre-process large sets of data and (ii) detect cores of communities that are forming the most close-knit and stable subsets of clusters of nodes present in a complex system., Comment: 13 pages, 2 figures. Lecture notes for the International School of Physics "Enrico Fermi" Computational Social Science and Complex Systems 16-21 July 2018

We describe two different bootstrap methods applied to the detection of a minimum spanning tree obtained from a set of multivariate variables. We show that two different bootstrap procedures provide partly distinct information that can be highly informative about the investigated complex system. Our case study, based on the investigation of daily returns of a portfolio of stocks traded in the US equity markets, shows the degree of robustness and completeness of the information extracted with popular information filtering methods such as the minimum spanning tree and the planar maximally filtered graph. The first method performs a "row bootstrap" whereas the second method performs a "pair bootstrap". We show that the parallel use of the two methods is suggested especially for complex systems presenting both a nested hierarchical organization together with the presence of global feedback channels., Comment: 17 pages, 7 figures

We use the information present in a bipartite network to detect cores of communities of each set of the bipartite system. Cores of communities are found by investigating statistically validated projected networks obtained using information present in the bipartite network. Cores of communities are highly informative and robust with respect to the presence of errors or missing entries in the bipartite network. We assess the statistical robustness of cores by investigating an artificial benchmark network, the co-authorship network, and the actor-movie network. The accuracy and precision of the partition obtained with respect to the reference partition are measured in terms of the adjusted Rand index and of the adjusted Wallace index respectively. The detection of cores is highly precise although the accuracy of the methodology can be limited in some cases., Comment: 9 pages, 6 figures

In this paper we present a simulator allowing to perform policy experiments relative to the air traffic management. Different SESAR solutions can be implemented in the model to see the reaction of the different stakeholders as well as other relevant metrics (delays, safety, etc). The model describes both the strategic phase associated to the planning of the flight trajectories and the tactical modifications occurring in the en-route phase. An implementation of the model is available as open-source and freely accessible by any user. More specifically, different procedures related to business trajectories and free-routing are tested and we illustrate the capabilities of the model on airspace implementing these concepts. After performing numerical simulations with the model, we show that in a free-routing scenario the controllers perform less operations although they are dispersed over a larger portion of the airspace. This can potentially increase the complexity of conflict detection and resolution for controllers. In order to investigate this specific aspect, we consider some metrics used to measure traffic complexity. We first show that in non-free-routing situations our simulator deals with complexity in a way similar to what humans would do. This allows us to be confident that the results of our numerical simulations relative to the free-routing can reasonably forecast how human controllers would behave in this new situation. Specifically, our numerical simulations show that most of the complexity metrics decrease with free-routing, while the few metrics which increase are all linked to the flight level changes. This is a non-trivial result since intuitively the complexity should increase with free-routing because of problematic geometries and more dispersed conflicts over the airspace.

We detect the backbone of the weighted bipartite network of the Japanese credit market relationships. The backbone is detected by adapting a general method used in the investigation of weighted networks. With this approach we detect a backbone that is statistically validated against a null hypothesis of uniform diversification of loans for banks and firms. Our investigation is done year by year and it covers more than thirty years during the period from 1980 to 2011. We relate some of our findings with economic events that have characterized the Japanese credit market during the last years. The study of the time evolution of the backbone allows us to detect changes occurred in network size, fraction of credit explained, and attributes characterizing the banks and the firms present in the backbone., Comment: 14 pages, 8 figures

We investigate the trading behavior of Finnish individual investors trading the stocks selected to compute the OMXH25 index in 2003 by tracking the individual daily investment decisions. We verify that the set of investors is a highly heterogeneous system under many aspects. We introduce a correlation based method that is able to detect a hierarchical structure of the trading profiles of heterogeneous individual investors. We verify that the detected hierarchical structure is highly overlapping with the cluster structure obtained with the approach of statistically validated networks when an appropriate threshold of the hierarchical trees is used. We also show that the combination of the correlation based method and of the statistically validated method provides a way to expand the information about the clusters of investors with similar trading profiles in a robust and reliable way., Comment: 25 pages, 8 figures

We present an analysis of the credit market of Japan. The analysis is performed by investigating the bipartite network of banks and firms which is obtained by setting a link between a bank and a firm when a credit relationship is present in a given time window. In our investigation we focus on a community detection algorithm which is identifying communities composed by both banks and firms. We show that the clusters obtained by directly working on the bipartite network carry information about the networked nature of the Japanese credit market. Our analysis is performed for each calendar year during the time period from 1980 to 2011. Specifically, we obtain communities of banks and networks for each of the 32 investigated years, and we introduce a method to track the time evolution of these communities on a statistical basis. We then characterize communities by detecting the simultaneous over-expression of attributes of firms and banks. Specifically, we consider as attributes the economic sector and the geographical location of firms and the type of banks. In our 32 year long analysis we detect a persistence of the over-expression of attributes of clusters of banks and firms together with a slow dynamics of changes from some specific attributes to new ones. Our empirical observations show that the credit market in Japan is a networked market where the type of banks, geographical location of firms and banks and economic sector of the firm play a role in shaping the credit relationships between banks and firms., Comment: 9 pages, 4 figures, 2 Tables

Big data open up unprecedented opportunities to investigate complex systems including the society. In particular, communication data serve as major sources for computational social sciences but they have to be cleaned and filtered as they may contain spurious information due to recording errors as well as interactions, like commercial and marketing activities, not directly related to the social network. The network constructed from communication data can only be considered as a proxy for the network of social relationships. Here we apply a systematic method, based on multiple hypothesis testing, to statistically validate the links and then construct the corresponding Bonferroni network, generalized to the directed case. We study two large datasets of mobile phone records, one from Europe and the other from China. For both datasets we compare the raw data networks with the corresponding Bonferroni networks and point out significant differences in the structures and in the basic network measures. We show evidence that the Bonferroni network provides a better proxy for the network of social interactions than the original one. By using the filtered networks we investigated the statistics and temporal evolution of small directed 3-motifs and conclude that closed communication triads have a formation time-scale, which is quite fast and typically intraday. We also find that open communication triads preferentially evolve to other open triads with a higher fraction of reciprocated calls. These stylized facts were observed for both datasets., Comment: 19 pages, 8 figures, 5 tables

Interbank markets are fundamental for bank liquidity management. In this paper, we introduce a model of interbank trading with memory. Our model reproduces features of preferential trading patterns in the e-MID market recently empirically observed through the method of statistically validated networks. The memory mechanism is used to introduce a proxy of trust in the model. The key idea is that a lender, having lent many times to a borrower in the past, is more likely to lend to that borrower again in the future than to other borrowers, with which the lender has never (or has in- frequently) interacted. The core of the model depends on only one parameter representing the initial attractiveness of all the banks as borrowers. Model outcomes and real data are compared through a variety of measures that describe the structure and properties of trading networks, including number of statistically validated links, bidirectional links, and 3-motifs. Refinements of the pairing method are also proposed, in order to capture finite memory and reciprocity in the model. The model is implemented within the Mason framework in Java., Comment: 37 pages, 10 figures

Mobile phone calling is one of the most widely used communication methods in modern society. The records of calls among mobile phone users provide us a valuable proxy for the understanding of human communication patterns embedded in social networks. Mobile phone users call each other forming a directed calling network. If only reciprocal calls are considered, we obtain an undirected mutual calling network. The preferential communication behavior between two connected users can be statistically tested and it results in two Bonferroni networks with statistically validated edges. We perform a comparative analysis of the statistical properties of these four networks, which are constructed from the calling records of more than nine million individuals in Shanghai over a period of 110 days. We find that these networks share many common structural properties and also exhibit idiosyncratic features when compared with previously studied large mobile calling networks. The empirical findings provide us an intriguing picture of a representative large social network that might shed new lights on the modelling of large social networks.

We show that the European airspace can be represented as a multi-scale traffic network whose nodes are airports, sectors, or navigation points and links are defined and weighted according to the traffic of flights between the nodes. By using a unique database of the air traffic in the European airspace, we investigate the architecture of these networks with a special emphasis on their community structure. We propose that unsupervised network community detection algorithms can be used to monitor the current use of the airspaces and improve it by guiding the design of new ones. Specifically, we compare the performance of three community detection algorithms, also by using a null model which takes into account the spatial distance between nodes, and we discuss their ability to find communities that could be used to define new control units of the airspace., Comment: 22 pages, 14 figures

We propose a setup for which a power-law decay is predicted to be observable for generic and realistic conditions. The system we study is very simple: A quantum wave packet initially prepared in a potential well with (i) tails asymptotically decaying like ~ x^{-2} and (ii) an eigenvalues spectrum that shows a continuous part attached to the ground or equilibrium state. We analytically derive the asymptotic decay law from the spectral properties for generic, confined initial states. Our findings are supported by realistic numerical simulations for state-of-the-art expansion experiments with cold atoms., Comment: improved and extended version

The last decade has seen the advent and consolidation of ontology based tools for the identification and biological interpretation of classes of genes, such as the Gene Ontology. The information accumulated time-by-time and included in the GO is encoded in the definition of terms and in the setting up of semantic relations amongst terms. This approach might be usefully complemented by a bottom-up approach based on the knowledge of relationships amongst genes. To this end, we investigate the Gene Ontology from a complex network perspective. We consider the semantic network of terms naturally associated with the semantic relationships provided by the Gene Ontology consortium and a gene-based weighted network in which the nodes are the terms and a link between any two terms is set up whenever genes are annotated in both terms. One aim of the present paper is to understand whether the semantic and the gene-based network share the same structural properties or not. We then consider network communities. The identification of communities in the SVNs network can therefore be the basis of a simple protocol aiming at fully exploiting the possible relationships amongst terms, thus improving the knowledge of the semantic structure of GO. This is also important from a biomedical point of view, as it might reveal how genes over-expressed in a certain term also affect other biological functions not directly linked by the GO semantics. As a by-product, we present a simple methodology that allows to have a first glance insight about the biological characterization of groups of GO terms., Comment: 10 pages, 5 figures, 2 Tables

We investigate the trading behavior of a large set of single investors trading the highly liquid Nokia stock over the period 2003-2008 with the aim of determining the relative role of endogenous and exogenous factors that may affect their behavior. As endogenous factors we consider returns and volatility, whereas the exogenous factors we use are the total daily number of news and a semantic variable based on a sentiment analysis of news. Linear regression and partial correlation analysis of data show that different categories of investors are differently correlated to these factors. Governmental and non profit organizations are weakly sensitive to news and returns or volatility, and, typically, they are more correlated with the former than with the latter. Households and companies, on the contrary, are very sensitive to both endogenous and exogenous factors, and volatility and returns are, on average, much more relevant than the number of news and sentiment, respectively. Finally, financial institutions and foreign organizations are intermediate between these two cases, in terms of both the total explanatory power of these factors and their relative importance., Comment: 30 pages, 4 figures and 5 tables

We introduce a new statistical test of the hypothesis that a balanced panel of firms have the same growth rate distribution or, more generally, that they share the same functional form of growth rate distribution. We applied the test to European Union and US publicly quoted manufacturing firms data, considering functional forms belonging to the Subbotin family of distributions. While our hypotheses are rejected for the vast majority of sets at the sector level, we cannot rejected them at the subsector level, indicating that homogenous panels of firms could be described by a common functional form of growth rate distribution., Comment: 17 pages, 3 figures, 2 tables

We introduce an analytical statistical method to characterize the communities detected in heterogeneous complex systems. By posing a suitable null hypothesis, our method makes use of the hypergeometric distribution to assess the probability that a given property is over-expressed in the elements of a community with respect to all the elements of the investigated set. We apply our method to two specific complex networks, namely a network of world movies and a network of physics preprints. The characterization of the elements and of the communities is done in terms of languages and countries for the movie network and of journals and subject categories for papers. We find that our method is able to characterize clearly the identified communities. Moreover our method works well both for large and for small communities., Comment: 8 pages, 1 figure and 2 tables

Many complex systems present an intrinsic bipartite nature and are often described and modeled in terms of networks [1-5]. Examples include movies and actors [1, 2, 4], authors and scientific papers [6-9], email accounts and emails [10], plants and animals that pollinate them [11, 12]. Bipartite networks are often very heterogeneous in the number of relationships that the elements of one set establish with the elements of the other set. When one constructs a projected network with nodes from only one set, the system heterogeneity makes it very difficult to identify preferential links between the elements. Here we introduce an unsupervised method to statistically validate each link of the projected network against a null hypothesis taking into account the heterogeneity of the system. We apply our method to three different systems, namely the set of clusters of orthologous genes (COG) in completely sequenced genomes [13, 14], a set of daily returns of 500 US financial stocks, and the set of world movies of the IMDb database [15]. In all these systems, both different in size and level of heterogeneity, we find that our method is able to detect network structures which are informative about the system and are not simply expression of its heterogeneity. Specifically, our method (i) identifies the preferential relationships between the elements, (ii) naturally highlights the clustered structure of investigated systems, and (iii) allows to classify links according to the type of statistically validated relationships between the connected nodes., Comment: Main text: 13 pages, 3 figures, and 1 Table. Supplementary information: 15 pages, 3 figures, and 2 Tables

The aim of the paper is to understand how the inclusion of more and more time-scales into a stochastic stationary Markovian process affects its conditional probability. To this end, we consider two Gaussian processes: (i) a short-range correlated process with an infinite set of time-scales bounded from below, and (ii) a power-law correlated process with an infinite and unbounded set of time-scales. For these processes we investigate the equal position conditional probability P(x,t|x,0) and the mean First Passage Time T(L). The function P(x,t|x,0) can be considered as a proxy of the persistence, i.e. the fact that when a process reaches a position x then it spends some time around that position value. The mean First Passage Time can be considered as a proxy of how fast is the process in reaching a position at distance L starting from position x. In the first investigation we show that the more time-scales the process includes, the larger the persistence. Specifically, we show that the power-law correlated process shows a slow power-law decay of P(x,t|x,0) to the stationary pdf. By contrast, the short range correlated process shows a decay dominated by an exponential cut-off. Moreover, we also show that the existence of an infinite and unbouded set of time-scales is a necessary and not sufficient condition for observing a slow power-law decay of P(x,t|x,0). In the second investigation, we show that for large values of L the more time-scales the process includes, the larger the mean First Passage Time, i.e. the slowest the process. On the other hand, for small values of L, the more time-scales the process includes, the smaller the mean First Passage Time, i.e. when a process statistically spends more time in a given position the likelihood that it reached nearby positions by chance is also enhanced., Comment: 11 pages, 7 figure

In this paper we give explicit examples of power-law correlated stationary Markovian processes y(t) where the stationary pdf shows tails which are gaussian or exponential. These processes are obtained by simply performing a coordinate transformation of a specific power-law correlated additive process x(t), already known in the literature, whose pdf shows power-law tails 1/x^a. We give analytical and numerical evidence that although the new processes (i) are Markovian and (ii) have gaussian or exponential tails their autocorrelation function still shows a power-law decay =1/T^b where b grows with a with a law which is compatible with b=a/2-c, where c is a numerical constant. When a<2(1+c) the process y(t), although Markovian, is long-range correlated. Our results help in clarifying that even in the context of Markovian processes long-range dependencies are not necessarily associated to the occurrence of extreme events. Moreover, our results can be relevant in the modeling of complex systems with long memory. In fact, we provide simple processes associated to Langevin equations thus showing that long-memory effects can be modeled in the context of continuous time stationary Markovian processes., Comment: 5 figures

By performing a comprehensive study on 1832 segments of 1212 complete genomes of viruses, we show that in viral genomes the hairpin structures of thermodynamically predicted RNA secondary structures are more abundant than expected under a simple random null hypothesis. The detected hairpin structures of RNA secondary structures are present both in coding and in noncoding regions for the four groups of viruses categorized as dsDNA, dsRNA, ssDNA and ssRNA. For all groups hairpin structures of RNA secondary structures are detected more frequently than expected for a random null hypothesis in noncoding rather than in coding regions. However, potential RNA secondary structures are also present in coding regions of dsDNA group. In fact we detect evolutionary conserved RNA secondary structures in conserved coding and noncoding regions of a large set of complete genomes of dsDNA herpesviruses., Comment: 9 pages, 2 figures

We investigate the emergence of a structure in the correlation matrix of assets' returns as the time-horizon over which returns are computed increases from the minutes to the daily scale. We analyze data from different stock markets (New York, Paris, London, Milano) and with different methods. Result crucially depends on whether the data is restricted to the ``internal'' dynamics of the market, where the ``center of mass'' motion (the market mode) is removed or not. If the market mode is not removed, we find that the structure emerges, as the time-horizon increases, from splitting a single large cluster. In NYSE we find that when the market mode is removed, the structure of correlation at the daily scale is already well defined at the 5 minutes time-horizon, and this structure accounts for 80 % of the classification of stocks in economic sectors. Similar results, though less sharp, are found for the other markets. We also find that the structure of correlations in the overnight returns is markedly different from that of intraday activity., Comment: 12 pages, 17 figures

We study theoretical and empirical aspects of the mean exit time of financial time series. The theoretical modeling is done within the framework of continuous time random walk. We empirically verify that the mean exit time follows a quadratic scaling law and it has associated a pre-factor which is specific to the analyzed stock. We perform a series of statistical tests to determine which kind of correlation are responsible for this specificity. The main contribution is associated with the autocorrelation property of stock returns. We introduce and solve analytically both a two-state and a three-state Markov chain models. The analytical results obtained with the two-state Markov chain model allows us to obtain a data collapse of the 20 measured MET profiles in a single master curve., Comment: REVTeX 4, 11 pages, 8 figures, 1 table, submitted for publication

We investigate the time series of the degree of minimum spanning trees obtained by using a correlation based clustering procedure which is starting from (i) asset return and (ii) volatility time series. The minimum spanning tree is obtained at different times by computing correlation among time series over a time window of fixed length $T$. We find that the minimum spanning tree of asset return is characterized by stock degree values, which are more stable in time than the ones obtained by analyzing a minimum spanning tree computed starting from volatility time series. Our analysis also shows that the degree of stocks has a very slow dynamics with a time-scale of several years in both cases., Comment: 9 pages, 3 figures

We introduce a new class of stochastic processes which are stationary, Markovian and characterized by an infinite range of time-scales. By transforming the Fokker-Planck equation of the process into a Schrodinger equation with an appropriate quantum potential we determine the asymptotic behavior of the autocorrelation function of the process in an analytical way. We find the conditions needed to observe a stationary long-range correlated Markovian process. In the presence of long-range correlation, for selected values of the control parameters, the process has a 1/f-like spectral density for low frequency values., Comment: 4 pages

We investigate the historical volatility of the 100 most capitalized stocks traded in US equity markets. An empirical probability density function (pdf) of volatility is obtained and compared with the theoretical predictions of a lognormal model and of the Hull and White model. The lognormal model well describes the pdf in the region of low values of volatility whereas the Hull and White model better approximates the empirical pdf for large values of volatility. Both models fails in describing the empirical pdf over a moderately large volatility range., Comment: 6 pages, 2 figures