Search

The characterization of the subsurface profile properties that are susceptible to liquefaction during earthquakes is of great importance in accident prevention. Field data can be used in a stochastic inverse problem to estimate the material properties using Markov chain Monte Carlo Methods (McMC) that incorporates prior knowledge about the unknown parameters as well as the data available. Here, we numerically study the Random Walk (RW) and Differential Evolution (DE) variations of the Metropolis algorithm in the context of seismic modeling considering the viscoelastic wave equation. The algorithms are tested with data from the 1987 Superstition Hills earthquake recorded at the Wildlife Site. The results show the effectiveness and accuracy of these algorithms, with emphasis to DE which reached convergence earlier compared to RW. [ABSTRACT FROM AUTHOR]

Traffic congestion in urban areas poses several challenges to municipal authorities including pollution, productivity loss, reckless driving, and delays in dealing with emergencies. Smart cities can use modern IoT infrastructure to solve the congestion problem and reduce pollution and delays. In this article, we focus on congestion mapping and mitigation for emergency vehicles in smart cities. We use a novel traffic light control technique to change the flow of cars on lights of interest thereby making way for emergency vehicles. We use a simulation model for a selected area of Manhattan to implement congestion mapping and to help find the fastest path for routing emergency vehicles based on the congestion metrics. The system controls traffic lights to block off the roads feeding into congestion and allows flow away from the congested path. This helps in clearing the preferred route to help emergency vehicles reach the destination faster. We show that the proposed algorithm can map congestion on city roads with accuracy thus helping to improve the response time of the emergency services and saving precious lives. [ABSTRACT FROM AUTHOR]

We consider the propagation of a stochastic SIR-type epidemic in two connected populations: a relatively small local population of interest which is surrounded by a much larger external population. External infectives can temporarily enter the small population and contribute to the spread of the infection inside this population. The rules for entry of infectives into the small population as well as their length of stay are modeled by a general Markov queueing system. Our main objective is to determine the distribution of the total number of infections within both populations. To do this, the approach we propose consists of deriving a family of martingales for the joint epidemic processes and applying classical stopping time or convergence theorems. The study then focuses on several particular cases where the external infection is described by a linear branching process and the entry of external infectives obeys certain specific rules. Some of the results obtained are illustrated by numerical examples. [ABSTRACT FROM AUTHOR]

We study Markov population processes on large graphs, with the local state transition rates of a single vertex being a linear function of its neighborhood. A simple way to approximate such processes is by a system of ODEs called the homogeneous mean-field approximation (HMFA). Our main result is showing that HMFA is guaranteed to be the large graph limit of the stochastic dynamics on a finite time horizon if and only if the graph-sequence is quasi-random. An explicit error bound is given and it is 1 N plus the largest discrepancy of the graph. For Erdős–Rényi and random regular graphs we show an error bound of order the inverse square root of the average degree. In general, diverging average degrees is shown to be a necessary condition for the HMFA to be accurate. Under special conditions, some of these results also apply to more detailed type of approximations like the inhomogenous mean field approximation (IHMFA). We pay special attention to epidemic applications such as the SIS process. [ABSTRACT FROM AUTHOR]

In the Fastest Mixing Markov Chain problem, we are given a graph G = (V , E) and desire the discrete-time Markov chain with smallest mixing time τ subject to having equilibrium distribution uniform on V and non-zero transition probabilities only across edges of the graph. It is well-known that the mixing time τ RW of the lazy random walk on G is characterised by the edge conductance Φ of G via Cheeger's inequality: Φ - 1 ≲ τ RW ≲ Φ - 2 log | V | . Analogously, we characterise the fastest mixing time τ ⋆ via a Cheeger-type inequality but for a different geometric quantity, namely the vertex conductance Ψ of G: Ψ - 1 ≲ τ ⋆ ≲ Ψ - 2 (log | V |) 2 . This characterisation forbids fast mixing for graphs with small vertex conductance. To bypass this fundamental barrier, we consider Markov chains on G with equilibrium distribution which need not be uniform, but rather only ε -close to uniform in total variation. We show that it is always possible to construct such a chain with mixing time τ ≲ ε - 1 (diam G) 2 log | V | . Finally, we discuss analogous questions for continuous-time and time-inhomogeneous chains. [ABSTRACT FROM AUTHOR]

A copula-based Markov chain model can flexibly capture serial dependence in a time series. However, the computational developments for copula-based Markov models remain insufficient for discrete marginal models compared with continuous ones. In this article, we develop computational methods for a binomial time series under the Clayton and Joe copulas. The methods include the data-generation, parameter estimation, model selection, and goodness-of-fit tests. We implement the methods in our R package Copula.Markov (). We conduct simulations to see the performance of the developed methods. Finally, the proposed method is illustrated by a real dataset. [ABSTRACT FROM AUTHOR]

Controlled discrete time Markov processes are studied first with long run general discounting functional. It is shown that optimal strategies for average reward per unit time problem are also optimal for average generally discounting functional. Then long run risk sensitive reward functional with general discounting is considered. When risk factor is positive then optimal value of such reward functional is dominated by the reward functional corresponding to the long run risk sensitive control. In the case of negative risk factor we get an asymptotical result, which says that optimal average reward per unit time control is nearly optimal for long run risk sensitive reward functional with general discounting, assuming that risk factor is close to 0. For this purpose we show in Appendix upper estimates for large deviations of weighted empirical measures, which are of independent interest. [ABSTRACT FROM AUTHOR]

This paper discusses a generalized SIR epidemic model that incorporates infectives with or without symptoms, allows arbitrary distributions for infectious periods and assumes infection rates depending on the current size of susceptibles. Our interest lies in the joint distribution of the state of the population and the severity of the disease at the end of infection. The approach is based on a formulation of the epidemic as a so-called collective model. First, a set of martingales is constructed which provides the distribution of this final epidemic outcome. Then, its corresponding distributional transform is expressed in terms of a family of pseudo-polynomials of Abel-Gontcharoff type. Some of the results obtained are illustrated numerically. [ABSTRACT FROM AUTHOR]

We consider populations with time-varying growth rates living in sinks. Each population, when isolated, would become extinct. Dispersal-induced growth (DIG) occurs when the populations are able to persist and grow exponentially when dispersal among the populations is present. We provide a mathematical analysis of this surprising phenomenon, in the context of a deterministic model with periodic variation of growth rates and non-symmetric migration which are assumed to be piecewise continuous. We also consider a stochastic model with random variation of growth rates and migration. This work extends existing results of the literature on the DIG effects obtained for periodic continuous growth rates and time independent symmetric migration. [ABSTRACT FROM AUTHOR]

In this study, we consider disorder detection problems for statistical models with dependent observations defined by Markov chains in a Bayesian setting for a uniform prior distribution when the number of observations is limited by some known quantity. To achieve this, a new optimal sequential detection procedure is constructed using optimal stopping methods. This procedure minimizes the average delay time in the class of sequential procedures with false alarm probabilities that do not exceed a fixed value. The main difference between the proposed detection procedure and the conventional ones is that it is based not on the posterior probabilities but on the weighted Shiryaev-Roberts statistic. This allows for optimal detection in a nonasymptotic sense over any bounded time interval. Thereafter, we applied the constructed procedures to the early detection problem for the beginning of the epidemic spread. We used two epidemic models: the binomial model proposed by Baron, Choudhary, and Yu (2013) and the model based on the Gaussian approximation introduced by Pergamenchtchikov, Tartakovsky, and Spivak (2022). The theoretical results were confirmed through numerical simulations using the Monte Carlo method. [ABSTRACT FROM AUTHOR]

We derive asymptotic formulae in the limit when population size N tends to infinity for mean fixation times (conditional and unconditional) in a population with two types of individuals, A and B, governed by the Moran process. We consider only the case in which the fitness of the two types do not depend on the population frequencies. Our results start with the important cases in which the initial condition is a single individual of any type, but we also consider the initial condition of a fraction x ∈ (0 , 1) of A individuals, where x is kept fixed and the total population size tends to infinity. In the cases covered by Antal and Scheuring (Bull Math Biol 68(8):1923–1944, 2006), i.e. conditional fixation times for a single individual of any type, it will turn out that our formulae are much more accurate than the ones they found. As quoted, our results include other situations not treated by them. An interesting and counterintuitive consequence of our results on mean conditional fixation times is the following. Suppose that a population consists initially of fitter individuals at fraction x and less fit individuals at a fraction 1 - x . If population size N is large enough, then in the average the fixation of the less fit individuals is faster (provided it occurs) than fixation of the fitter individuals, even if x is close to 1, i.e. fitter individuals are the majority. [ABSTRACT FROM AUTHOR]

Bayesian Learning is an inference method designed to tackle exploration-exploitation trade-off as a function of the uncertainty of a given probability model from observations within the Reinforcement Learning (RL) paradigm. It allows the incorporation of prior knowledge, as probabilistic distributions, into the algorithms. Finding the resulting Bayes-optimal policies is notorious problem. We focus our attention on RL of a special kind of ergodic and controllable Markov games. We propose a new framework for computing the near-optimal policies for each agent, where it is assumed that the Markov chains are regular and the inverse of the behavior strategy is well defined. A fundamental result of this paper is the development of a theoretical method that, based on the formulation of a non-linear problem, computes the near-optimal adaptive-behavior strategies and policies of the game under some restrictions that maximize the expected reward. We prove that such behavior strategies and the policies satisfy the Bayesian-Nash equilibrium. Another important result is that the RL process learn a model through the interaction of the agents with the environment, and shows how the proposed method can finitely approximate and estimate the elements of the transition matrices and utilities maintaining an efficient long-term learning performance measure. We develop the algorithm for implementing this model. A numerical empirical example shows how to deploy the estimation process as a function of agent experiences. [ABSTRACT FROM AUTHOR]

We study an energy-constrained random walker on a length-N interval of the one-dimensional integer lattice, with boundary reflection. The walker consumes one unit of energy for every step taken in the interior, and energy is replenished up to a capacity of M on each boundary visit. We establish large N, M distributional asymptotics for the lifetime of the walker, i.e., the first time at which the walker runs out of energy while in the interior. Three phases are exhibited. When M ≪ N 2 (energy is scarce), we show that there is an M-scale limit distribution related to a Darling–Mandelbrot law, while when M ≫ N 2 (energy is plentiful) we show that there is an exponential limit distribution on a stretched-exponential scale. In the critical case where M / N 2 → ρ ∈ (0 , ∞) , we show that there is an M-scale limit in terms of an infinitely-divisible distribution expressed via certain theta functions. [ABSTRACT FROM AUTHOR]

A two-class Processor-Sharing queue with one impatient class is studied. Local exponential decay rates for its stationary distribution (N (∞) , M (∞)) are established in the heavy traffic regime where the arrival rate of impatient customers grows proportionally to a large factor A. This regime is characterized by two time-scales, so that no general Large Deviations result is applicable. In the framework of singular perturbation methods, we instead assume that an asymptotic expansion of the solution of associated Kolmogorov equations exists for large A and derive it in the form P (N (∞) = A x , M (∞) = A y) ∼ g (x , y) 2 π A · e − A H (x , y) , x > 0 , y > 0 , with explicit functions g and H. This result is then applied to the model of mobile networks proposed in [Olivier and Simonian] and accounting for the spatial movement of users. We give further evidence of a unusual growth behavior in heavy traffic in that the stationary mean queue length E (N mob (∞)) and E M mob (∞)) of each customer-class increases proportionally to E (N mob (∞)) ∝ E (M mob (∞)) ∝ log (1 1 − ϱ tot ) with system load ϱ tot tending to 1, instead of the usual 1 / (1 − ϱ tot) growth behavior. [ABSTRACT FROM AUTHOR]

In this article we propose a new online control system aiming to lower the instants in which the production process migrates from being in- to out-of-control state, which generates an increase in the non-conformity rates. As shifts from in- to out-of-control are non-deterministic, a sample of size n is collected, for each m or L units produced, and each element from the sample is imprecisely classified as conform or non-conform (that is, there may be classification errors). If the amount of conform units from the sample is equal or greater than a, the process would not be adjusted and another sample would be collected after m units produced. If the quantity of conform units is inferior to a, the process would be adjusted and another sample would be collected after L units produced, given that L > m. A genetic algorithm is proposed to approximately find the values of a, n, m, and L that minimize all costs involved in the process being controlled. All procedures are illustrated through a detailed numerical example that attests the efficacy and efficiency of the proposed online control system. [ABSTRACT FROM AUTHOR]

In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the aforementioned Wasserstein-2 convergence result can be applied to establish a non-asymptotic error bound for the expected excess risk. Crucially, these results are obtained under a local Lipschitz condition and a local dissipativity condition where we remove the uniform dependence in the data stream. We illustrate the importance of this relaxation by presenting examples from variational inference and from index tracking optimization. [ABSTRACT FROM AUTHOR]

In this paper, we consider constrained discounted stochastic games with a countably generated state space and norm continuous transition probability having a density function. We prove existence of approximate stationary equilibria and stationary weak correlated equilibria. Our results imply the existence of stationary Nash equilibrium in ARAT stochastic games. [ABSTRACT FROM AUTHOR]

Our main goal in this paper is to propose a mathematical model in order to use it to study the historical achievements of a particular team in order to stand on the evolutionary behavior of the team and to be able to predict the future results of the team by measuring the team's performance through the total number of victories, losses and draws in official matches approved. Our main idea is to use a Markov model to find the results of matches, so that the outcome of each match is one of the specified cases: win or lose or draw. [ABSTRACT FROM AUTHOR]

A controlled discrete-time stochastic process is proposed as a model for the state of a machine. In the proposed model, normal and random wear of the machine are considered. The random wear of the machine during a time unit can be either a discrete or a continuous random variable. The objective is to find a control policy that maximises the profits generated by the machine over its useful lifetime. In this problem, the optimiser must decide whether to do or not to do the maintenance work on the machine at each time unit. The significance of the paper is that the final time in the optimal control problem is a random variable denoting the first time the machine must be replaced. To obtain the optimal control, one can try to solve the dynamic programming equation, which reduces to a difference or an integral equation, satisfied by the value function. Finally, particular cases are solved exactly, or approximately, and explicitly to demonstrate and validate the proposed model. [ABSTRACT FROM AUTHOR]

Autocatalysis underlies the ability of chemical and biochemical systems to replicate. Recently, Blokhuis et al. (PNAS 117(41):25230–25236, 2020) gave a stoechiometric definition of autocatalysis for reaction networks, stating the existence of a combination of reactions such that the balance for all autocatalytic species is strictly positive, and investigated minimal autocatalytic networks, called autocatalytic cores. By contrast, spontaneous autocatalysis—namely, exponential amplification of all species internal to a reaction network, starting from a diluted regime, i.e. low concentrations—is a dynamical property. We introduce here a topological condition (Top) for autocatalysis, namely: restricting the reaction network description to highly diluted species, we assume existence of a strongly connected component possessing at least one reaction with multiple products (including multiple copies of a single species). We find this condition to be necessary and sufficient for stoechiometric autocatalysis. When degradation reactions have small enough rates, the topological condition further ensures dynamical autocatalysis, characterized by a strictly positive Lyapunov exponent giving the instantaneous exponential growth rate of the system. The proof is generally based on the study of auxiliary Markov chains. We provide as examples general autocatalytic cores of Type I and Type III in the typology of Blokhuis et al. (PNAS 117(41):25230–25236, 2020). In a companion article (Unterberger in Dynamical autocatalysis for autocatalytic cores, 2021), Lyapunov exponents and the behavior in the growth regime are studied quantitatively beyond the present diluted regime. [ABSTRACT FROM AUTHOR]

The IEEE 802.15.4 standard is known as the most used technology for Wireless Sensor Networks (WSNs) because of their low rate and especially their energy efficiency. To better support the requirements of Low Energy Critical Infrastructure Monitoring (LECIM) applications, the IEEE 802.15.4k is approved. This amendment is intended for critical packets transmission using the Priority Channel Access (PCA) that has become a real issue in infrastructures applications. The analysis presented in this paper investigates, for the first time in the literature, the performance analysis of the IEEE 802.15.4k in non beacon-enabled mode assuming star topology, saturated traffic and fading channel conditions. Where fading channel is a very important factor in the network as it affects its performance. We emphasize that this is an original work that has never been treated in literature. In this paper, we have developed a two dimensional Markov chain model for the IEEE 802.15.4k unslotted Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) with PCA backoff mechanism, and, a three dimensional Markov chain model for the IEEE 802.15.4k unslotted CSMA/CA mechanism. Furthermore, to test the efficiency of the proposed model, we have analyzed the impact of variation of network size, probability that the available packet is critical, Bit Error Rate (BER), and packet length on the reliability, power consumption and throughput in order to compare the performance of both mechanisms. With the increase of network size and packet length, it is observed that PCA achieves a reduction in energy, a higher throughput and a lower reliability compared to CSMA/CA. The obtained results are satisfactory because packet length and network size are the most important parameters in industrial applications especially in critical systems. Finally, a comparative table was given to compare the behavior of the IEEE 802.15.4k network in terms of performance metrics. [ABSTRACT FROM AUTHOR]

In this paper, we extend the classic optimal research and development (R&D) investment model into the regime-switching environment. We formulate a robust model to obtain the maximal net value function of the R&D project under a family of real-world measures, which can also be regarded as a stochastic differential game. Then, the verification argument is provided. Though we do not find the closed-form solution, we give a numerical simulation to further study the qualities of the solution to our model. [ABSTRACT FROM AUTHOR]

We present a Reinforcement Learning (RL) algorithm to solve infinite horizon asymptotic Mean Field Game (MFG) and Mean Field Control (MFC) problems. Our approach can be described as a unified two-timescale Mean Field Q-learning: The same algorithm can learn either the MFG or the MFC solution by simply tuning the ratio of two learning parameters. The algorithm is in discrete time and space where the agent not only provides an action to the environment but also a distribution of the state in order to take into account the mean field feature of the problem. Importantly, we assume that the agent cannot observe the population's distribution and needs to estimate it in a model-free manner. The asymptotic MFG and MFC problems are also presented in continuous time and space, and compared with classical (non-asymptotic or stationary) MFG and MFC problems. They lead to explicit solutions in the linear-quadratic (LQ) case that are used as benchmarks for the results of our algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we consider a large class of constrained non-cooperative stochastic Markov games with countable state spaces and discounted cost criteria. In one-player case, i.e., constrained discounted Markov decision models, it is possible to formulate a static optimisation problem whose solution determines a stationary optimal strategy (alias control or policy) in the dynamical infinite horizon model. This solution lies in the compact convex set of all occupation measures induced by strategies, defined on the set of state–action pairs. In case of n-person discounted games the occupation measures are induced by strategies of all players. Therefore, it is difficult to generalise the approach for constrained discounted Markov decision processes directly. It is not clear how to define the domain for the best-response correspondence whose fixed point induces a stationary equilibrium in the Markov game. This domain should be the Cartesian product of compact convex sets in a locally convex topological vector space. One of our main results shows how to overcome this difficulty and define a constrained non-cooperative static game whose Nash equilibrium induces a stationary Nash equilibrium in the Markov game. This is done for games with bounded cost functions and positive initial state distribution. An extension to a class of Markov games with unbounded costs and arbitrary initial state distribution relies on an approximation of the unbounded game by bounded ones with positive initial state distributions. In the unbounded case, we assume the uniform integrability of the discounted costs with respect to all probability measures induced by strategies of the players, defined on the space of plays (histories) of the game. Our assumptions are weaker than those applied in earlier works on discounted dynamic programming or stochastic games using the so-called weighted norm approach. [ABSTRACT FROM AUTHOR]

Self-exciting point processes have been proposed as models for the location of criminal events in space and time. Here we consider the case where the triggering function is isotropic and takes a non-parametric form that is determined from data. We pay special attention to normalisation issues and to the choice of spatial distance measure, thereby extending the current methodology. After validating these ideas on synthetic data, we perform inference and prediction tests on public domain burglary data from Chicago. We show that the algorithmic advances that we propose lead to improved predictive accuracy. [ABSTRACT FROM AUTHOR]

The problem of removing or blocking the minimum number of individuals to control the infectious spread in the network is an NP-hard problem. In this paper a heuristic algorithm, named, BDEC, based on the Principal Component Analysis (PCA) is introduced to deal with this problem. A mathematical model with a PCA approach is formulated, the dynamical behaviour of the model is studied, and also the basic reproductive number R0 is determined. It is proved that the disease-free equilibrium is stable if R0 < 1, and it is unstable if R0 > 1. Then, we introduce a function to discuss the rate of epidemic spread in the social networks by simulation the epidemic spread based on the birth and death process. [ABSTRACT FROM AUTHOR]

We consider a system of N interacting particles, governed by transport and diffusion, that converges in a mean-field limit to the solution of a McKean–Vlasov equation. From the observation of a trajectory of the system over a fixed time horizon, we investigate nonparametric estimation of the solution of the associated nonlinear Fokker–Planck equation, together with the drift term that controls the interactions, in a large population limit N → ∞ . We build data-driven kernel estimators and establish oracle inequalities, following Lepski's principle. Our results are based on a new Bernstein concentration inequality in McKean–Vlasov models for the empirical measure around its mean, possibly of independent interest. We obtain adaptive estimators over anisotropic Hölder smoothness classes built upon the solution map of the Fokker–Planck equation, and prove their optimality in a minimax sense. In the specific case of the Vlasov model, we derive an estimator of the interaction potential and establish its consistency. [ABSTRACT FROM AUTHOR]

In a recent breakthrough, Teyssier (Ann Probab 48(5):2323–2343, 2020) introduced a new method for approximating the distance from equilibrium of a random walk on a group. He used it to study the limit profile for the random transpositions card shuffle. His techniques were restricted to conjugacy-invariant random walks on groups; we derive similar approximation lemmas for random walks on homogeneous spaces and for general reversible Markov chains. We illustrate applications of these lemmas to some famous problems: the k-cycle shuffle, sharpening results of Hough (Probab Theory Relat Fields 165(1–2):447–482, 2016) and Berestycki, Schramm and Zeitouni (Ann Probab 39(5):1815–1843, 2011), the Ehrenfest urn diffusion with many urns, sharpening results of Ceccherini-Silberstein, Scarabotti and Tolli (J Math Sci 141(2):1182–1229, 2007), a Gibbs sampler, which is a fundamental tool in statistical physics, with Binomial prior and hypergeometric posterior, sharpening results of Diaconis, Khare and Saloff-Coste (Stat Sci 23(2):151–178, 2008). [ABSTRACT FROM AUTHOR]

During their lifetimes, individuals in populations pass through different states, and the notion of an occupancy time describes the amount of time an individual spends in a given set of states. Questions related to this idea were studied in a recent paper by Roth and Caswell for cases where the environmental conditions are constant. However, it is truly important to consider the case where environments are changing randomly or in directional way through time, so the transition probabilities between different states change over time, motivating the use of time-dependent stage-structured models. Using absorbing inhomogenous Markov chains and the discrete-time McKendrick–von Foerster equation, we derive explicit formulas for the occupancy time, its expectation, and its higher-order moments for stage-structured models with time-dependent transition rates. The results provide insights into the dynamics of long lived plant or animal populations where individuals transition in both directions between reproductive and non reproductive stages. We apply our approach to study a specific time-dependent model of the Southern Fulmar, and obtain insights into how the number of breeding attempts depends on external conditions that vary through time. [ABSTRACT FROM AUTHOR]

By making use of Kullback-Leibler information, we develop a new approach for the quickest detection problem for general statistical models with dependent observations and unknown postchange distributions; the postchange distribution depends on either unknown informative parameters or unknown nonparametric infinite-dimensional nuisance functions. For such models, we introduce a robust risk as the supremum of the mean detection delay over the class of postchange distributions. On the basis of the window-limited cumulative sum rules developed by Lai in 1988, we propose new detection procedures, making use of the noise density that minimizes the Kullback-Leibler divergence. Then for the constructed detection procedures, we provide sufficient conditions on the considered statistical models that ensure minimax optimality properties with respect to the robust risk. We apply the developed methods to the quick detection problems for both scalar and multivariate autoregressive processes with unknown postchange parameters and unknown noise distributions. [ABSTRACT FROM AUTHOR]

Motivated by modelling epidemics like COVID-19, this paper proposes a generalized chain binomial process which integrates two types of infectives, those with symptoms and those without. Testing of infectives and vaccination of susceptibles are then incorporated as preventive protective measures. Our interest relates to the distribution of the state of the population at the end of infection and to the reproduction number R 0 with the associated extinction condition. The method uses the construction of a family of martingales and a branching approximation for large populations, respectively. A more general branching process for epidemics is also constructed and studied. Finally, some results obtained are illustrated by numerical examples. [ABSTRACT FROM AUTHOR]

Investigating serial dependence is an important step in statistical process control (SPC). One recent approach is to fit a copula-based Markov chain model to perform SPC, which provides an attractive alternative to the traditional AR1 model. However, methodologies for model diagnostic have not been considered. In this paper, we develop two different approaches for model diagnostic procedures for copula-based Markov chain models. The first approach employs a formal test based on the Kolmogorov-Smirnov or the Cramér-von Mises statistics with aid of a parametric bootstrap. The second approach employs the second-order Markov chain model to examine the Markov property in the model. This second approach itself is a new SPC method. We made all the computing methodologies available in the R Copula.Markov package, and check their performance by simulations. We analyze three datasets for illustration. [ABSTRACT FROM AUTHOR]

Models of evolution by natural selection often make the simplifying assumption that populations are infinitely large. In this infinite population limit, rare mutations that are selected against always go extinct, whereas in finite populations they can persist and even reach fixation. Nevertheless, for mutations of arbitrarily small phenotypic effect, it is widely believed that in sufficiently large populations, if selection opposes the invasion of rare mutants, then it also opposes their fixation. Here, we identify circumstances under which infinite-population models do or do not accurately predict evolutionary outcomes in large, finite populations. We show that there is no population size above which considering only invasion generally suffices: for any finite population size, there are situations in which selection opposes the invasion of mutations of arbitrarily small effect, but favours their fixation. This is not an unlikely limiting case; it can occur when fitness is a smooth function of the evolving trait, and when the selection process is biologically sensible. Nevertheless, there are circumstances under which opposition of invasion does imply opposition of fixation: in fact, for the n -player snowdrift game (a common model of cooperation) we identify sufficient conditions under which selection against rare mutants of small effect precludes their fixation—in sufficiently large populations—for any selection process. We also find conditions under which—no matter how large the population—the trait that fixes depends on the selection process, which is important because any particular selection process is only an approximation of reality. [ABSTRACT FROM AUTHOR]

The efficient evaluation of high-dimensional integrals is important from both theoretical and practical points of view. In particular, multidimensional integration plays a central role in Bayesian inference, statistical physics, data science, and machine learning. However, due to the curse of dimensionality, deterministic numerical methods are inefficient in the high-dimensional setting. Consequentially, for many practical problems one must resort to approximate estimation techniques such as Monte Carlo methods. In this article, we introduce a novel sequential Monte Carlo algorithm called stratified splitting. The method provides unbiased estimates and can handle various integrand types including indicator functions, which are important for rare-event probability estimation problems. We provide rigorous analysis of the efficiency of the proposed method and present a numerical demonstration of the algorithmic performance when applied to practical application domains. Our numerical experiments suggest that the stratified splitting method is capable of delivering accurate results for a variety of integration problems while requiring reasonable computational effort. [ABSTRACT FROM AUTHOR]

Resource management is the process of task scheduling and resource provisioning to provide requirements of cloud users. Since cloud resources are often heterogeneous, task scheduling and resource provisioning are major challenges in this area. Various methods have been introduced to improve resource utilization and thus increase the efficiency of cloud computing. Existing methods can be divided into several categories, including mathematical and statistical methods, heuristic- and meta-heuristic-based methods, and machine-learning-based methods. Since the resource management problem is NP-complete, several optimization methods have been also exploited in this area. Coral reefs algorithm is an evolutionary method that has showed appropriate convergence and response time for some problems, and thus is used in this paper to combine with reinforcement learning to improve efficiency of resource management in cloud environments. The proposed method of this paper consists of two phases. The initial allocation of resources to ready-to-perform tasks is done using the coral reefs algorithm in the first phase. The tasks are considered as corals and the resources are considered reefs in this method. The second phase utilizes reinforcement learning to avoid falling into the local optima and to make optimal use of resources using a long-term approach. The proposed model of this paper, called MO-CRAML, introduces a new hybrid algorithm for improving utilization and load balancing of cloud resources using the combination of coral reefs optimization algorithm and reinforcement learning. The results of the experiments show that the proposed algorithm has better performance in cloud resource utilization and load balancing in comparison with some other important methods of the literature. [ABSTRACT FROM AUTHOR]

In this paper, a three-state multinomial hidden Markov model (HMM) is formulated to handle the problem of intra-category heterogeneity caused by latent factors for transition flows of a hierarchical manpower system. The model, which incorporates mover, mediocre and stayer latent subclasses for each personnel category, is applied in analyzing manpower data for the academic staff of a University system in Nigeria. An EM algorithm is used to estimate the probabilities of transition of members from each of the subclasses. The principles of Likelihood Ratio Test, Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) for model comparison/validation gave evidence in favor of the HMM over the classical Markov model for the system. [ABSTRACT FROM AUTHOR]

This paper studies the nonstandardized Student's t-distribution for fitting serially correlated observations where serial dependence is described by the copula-based Markov chain. Due to the computational difficulty of obtaining maximum likelihood estimates, alternatively, we develop Bayesian inference using the empirical Bayes method through the resampling procedure. We provide the simulations to examine the performance and also analyze the stock price data in empirical studies for illustration. [ABSTRACT FROM AUTHOR]

We propose an adaptive estimator for the stationary distribution of a bifurcating Markov Chain on R d . Bifurcating Markov chains (BMC for short) are a class of stochastic processes indexed by regular binary trees. A kernel estimator is proposed whose bandwidths are selected by a method inspired by the works of Goldenshluger and Lepski [(2011), 'Bandwidth Selection in Kernel Density Estimation: Oracle Inequalities and Adaptive Minimax Optimality', The Annals of Statistics 3: 1608–1632). Drawing inspiration from dimension jump methods for model selection, we also provide an algorithm to select the best constant in the penalty. Finally, we investigate the performance of the method by simulation studies and application to real data. [ABSTRACT FROM AUTHOR]

In statistical learning theory, numerous works established non-asymptotic bounds assessing the generalization capacity of empirical risk minimizers under a large variety of complexity assumptions for the class of decision rules over which optimization is performed, by means of sharp control of uniform deviation of i.i.d. averages from their expectation, while fully ignoring the possible dependence across training data in general. It is the purpose of this paper to show that similar results can be obtained when statistical learning is based on a data sequence drawn from a (Harris positive) Markov chain X, through the running example of estimation of minimum volume sets (MV-sets) related to X's stationary distribution, an unsupervised statistical learning approach to anomaly/novelty detection. Based on novel maximal deviation inequalities we establish, using the regenerative method, learning rate bounds that depend not only on the complexity of the class of candidate sets but also on the ergodicity rate of the chain X, expressed in terms of tail conditions for the length of the regenerative cycles. In particular, this approach fully tailored to Markovian data permits to interpret the rate bound results obtained in frequentist terms, in contrast to alternative coupling techniques based on mixing conditions: the larger the expected number of cycles over a trajectory of finite length, the more accurate the MV-set estimates. Beyond the theoretical analysis, this phenomenon is supported by illustrative numerical experiments. [ABSTRACT FROM AUTHOR]

We study fixation probabilities for the Moran stochastic process for the evolution of a population with three or more types of individuals and frequency-dependent fitnesses. Contrary to the case of populations with two types of individuals, in which fixation probabilities may be calculated by an exact formula, here we must solve a large system of linear equations. We first show that this system always has a unique solution. Other results are upper and lower bounds for the fixation probabilities obtained by coupling the Moran process with three strategies with birth–death processes with only two strategies. We also apply our bounds to the problem of evolution of cooperation in a population with three types of individuals already studied in a deterministic setting by Núñez Rodríguez and Neves (J Math Biol 73:1665–1690, 2016). We argue that cooperators will be fixated in the population with probability arbitrarily close to 1 for a large region of initial conditions and large enough population sizes. [ABSTRACT FROM AUTHOR]

A spatial Markov-chain model is formulated for the progression of skin cancer. The model is based on the division of the computational domain into nodal points, that can be in a binary state: either in 'cancer state' or in 'non-cancer state'. The model assigns probabilities for the non-reversible transition from 'non-cancer' state to the 'cancer state' that depend on the states of the neighbouring nodes. The likelihood of transition further depends on the life burden intensity of the UV-rays that the skin is exposed to. The probabilistic nature of the process and the uncertainty in the input data is assessed by the use of Monte Carlo simulations. A good fit between experiments on mice and our model has been obtained. [ABSTRACT FROM AUTHOR]

This paper is addressed to give a generalization of the classical Markov methodology allowing the treatment of the entries of the transition matrix and initial condition as random variables instead of deterministic values lying in the interval [ 0 , 1 ]. This permits the computation of the first probability density function (1-PDF) of the solution stochastic process taking advantage of the so-called Random Variable Transformation technique. From the 1-PDF relevant probabilistic information about the evolution of Markov models can be calculated including all one-dimensional statistical moments. We are also interested in determining the computation of distribution of some important quantities related to randomized Markov chains (steady state, hitting times, etc.). All theoretical results are established under general assumptions and they are illustrated by modelling the spread of a technology using real data. [ABSTRACT FROM AUTHOR]

Motivated by the cost savings that can be obtained by sharing resources in a network context, we consider a stylized, yet representative, model for the coordination of maintenance and service logistics for a geographic network of assets. Capital assets, such as wind turbines in a wind park, require maintenance throughout their long lifetimes. Two types of preventive maintenance are considered: planned maintenance at periodic, scheduled opportunities, and opportunistic maintenance at unscheduled opportunities. The latter type of maintenance arises due to the network context: When an asset in the network fails, this constitutes an opportunity for preventive maintenance for the other assets in the network. So as to increase the realism of the model at hand and its applicability to various sectors, we consider the option of not-deferring and of deferring planned maintenance after the occurrence of opportunistic maintenance. We also assume that preventive maintenance may not always restore the condition of the system to 'as good as new.' By formulating this problem as a semi-Markov decision process, we characterize the optimal policy as a control limit policy (depending on the remaining time until the next planned maintenance) that indicates on the one hand when it is optimal to perform preventive maintenance and on the other hand when maintenance resources should be shared if an opportunity in the network arises. In order to facilitate managerial insights on the effect of each parameter on the cost, we provide a closed-form expression for the long-run rate of cost for any given control limit policy (depending on the remaining time until the next planned maintenance) and compare the costs (under the optimal policy) to those of suboptimal policies that neglect the opportunity for resource sharing. We illustrate our findings using data from the wind energy industry. [ABSTRACT FROM AUTHOR]

Stochastic fluid models have been widely used to model the level of a resource that changes over time, where the rate of variation depends on the state of some continuous-time Markov process. Latouche and Taylor (Queueing Syst 63:109–129, 2009) introduced an approach, using matrix analytic methods and the reduced load approximation for loss networks, to analyse networks of fluid models all driven by the same modulating process where the buffers are infinite. We extend the method to networks involving finite buffer models and illustrate the approach by deriving performance measures for a simple network as characteristics such as buffer size are varied. Our results provide insight into the situations where the infinite buffer model is a reasonable approximation to the finite buffer model. [ABSTRACT FROM AUTHOR]

We discuss lifetimes for a family of population-dependent branching processes. The attenuation factor (due to environment or competition, for example) is of Ricker type, i.e., the probability of an individual having offspring at all is of the form if the total population is n. Equivalently we can write the probability as where the carrying capacity K is the inverse of the attenuating factor. It is well known that the expected lifetime of such a process is exponential in K. If the carrying capacities vary much over time, for instance, if they are i.i.d. with a heavy-tailed distribution, the extinction scenario may change to a growth-catastrophe one with expected lifetimes much shorter. In addition to Ricker's model, production functions of the Beverton–Holt and Hassell types are also discussed. [ABSTRACT FROM AUTHOR]

Stochastic processes are approved presentation of real systems which its development in space or time can be supposed as random. A semi-hidden Markov model as a type of stochastic processes is a modification of hidden Markov models with states that are no longer totally unobservable and are less hidden. This mathematical model is employed for modeling data sequences with long runs, memory and statistical inertia. In this article, we investigate the theory of the semi-hidden Markov model along with its parameter estimation and order estimation methods. Moreover, the proposed model is applied to model the error traces generated by the wireless channels. A new Markov-based trace analysis algorithm is suggested to divide a non-stationary network error trace into stationary parts. By means of the best semi-hidden Markov model and fitting probability distribution, we would be able to model these parts accurately. Calculating the information measure criteria and the autocorrelation function by running the modified Baum–Welch algorithm several times help us to find the optimal order of the semi-hidden Markov model. [ABSTRACT FROM AUTHOR]

This paper is based on the complete classification of evolutionary scenarios for the Moran process with two strategies given by Taylor et al. (Bull Math Biol 66(6):1621-1644, 2004. 10.1016/j.bulm.2004.03.004). Their classification is based on whether each strategy is a Nash equilibrium and whether the fixation probability for a single individual of each strategy is larger or smaller than its value for neutral evolution. We improve on this analysis by showing that each evolutionary scenario is characterized by a definite graph shape for the fixation probability function. A second class of results deals with the behavior of the fixation probability when the population size tends to infinity. We develop asymptotic formulae that approximate the fixation probability in this limit and conclude that some of the evolutionary scenarios cannot exist when the population size is large. [ABSTRACT FROM AUTHOR]

Wilson (Proceedings of the twenty-eight annual acm symposium on the theory of computing, pp. 296-303, 1996) in the 1990s described a simple and efficient algorithm based on loop-erased random walks to sample uniform spanning trees and more generally weighted trees or forests spanning a given graph. This algorithm provides a powerful tool in analyzing structures on networks and along this line of thinking, in recent works (Avena and Gaudillière in A proof of the transfer-current theorem in absence of reversibility, in Stat. Probab. Lett. 142, 17-22 (2018); Avena and Gaudillière in J Theor Probab, 2017. 10.1007/s10959-017-0771-3; Avena et al. in Approximate and exact solutions of intertwining equations though random spanning forests, 2017. arXiv:1702.05992v1; Avena et al. in Intertwining wavelets or multiresolution analysis on graphs through random forests, 2017. arXiv:1707.04616, to appear in ACHA (2018)) we focused on applications of spanning rooted forests on finite graphs. The resulting main conclusions are reviewed in this paper by collecting related theorems, algorithms, heuristics and numerical experiments. A first foundational part on determinantal structures and efficient sampling procedures is followed by four main applications: (1) a random-walk-based notion of well-distributed points in a graph, (2) a framework to describe metastable-like dynamics in finite settings by means of Markov intertwining dualities, (3) coarse graining schemes for networks and associated processes, (4) wavelets-like pyramidal algorithms for graph signals. [ABSTRACT FROM AUTHOR]

Transmission delay, throughput and energy are important criterions to consider in wireless sensor networks (WSN). In this way, IEEE 802.15.4 standard was conceived with the objective to reduce resource’s consumption in both WSN and wireless personal area networks. In such networks, the slotted CSMA/CA still occupies a prominent place as a channel control access mechanism with its inherent simplicity and reduced complexity. In this paper, we propose to introduce a network allocation vector (NAV), to reduce energy consumption and collisions in IEEE 802.15.4 networks. A Markov chain-based analytical model of the fragmentation mechanism, in a saturated traffic, is given as well as a model of the energy consumption using the NAV mechanism. The obtained results show that the fragmentation technique improves at the same time the throughput, the access delay and the bandwidth occupation. They, also, show that using the NAV allows reducing significantly the energy consumption when applying the fragmentation technique in slotted CSMA/CA under saturated traffic conditions. [ABSTRACT FROM AUTHOR]

We introduce a dynamic optimization framework in which collateral is used to mitigate losses arising at counterparty’s default. The investor faces two sources of risk: the default risk of the entity referencing the traded credit swap security, and counterparty risk generated from the default event of the trading counterparty. We show that the value function of the control problem coincides with the classical solution of a nonlinear dynamic programming equation. We provide an explicit characterization of the optimal investment strategy, and show that the investor does not trade if counterparty risk is sufficiently high. These findings suggest that moving credit swap trades into well-designed clearinghouses may stimulate economic activities. [ABSTRACT FROM AUTHOR]