Search

Latent classification model is a class of statistical methods for identifying unobserved class membership among the study samples using some observed data. In this study, we proposed a latent classification model that takes a censored longitudinal binary outcome variable and uses its changing pattern over time to predict individuals' latent class membership. Assuming the time‐dependent outcome variables follow a continuous‐time Markov chain, the proposed method has two primary goals: (1) estimate the distribution of the latent classes and predict individuals' class membership, and (2) estimate the class‐specific transition rates and rate ratios. To assess the model's performance, we conducted a simulation study and verified that our algorithm produces accurate model estimates (ie, small bias) with reasonable confidence intervals (ie, achieving approximately 95% coverage probability). Furthermore, we compared our model to four other existing latent class models and demonstrated that our approach yields higher prediction accuracies for latent classes. We applied our proposed method to analyze the COVID‐19 data in Houston, Texas, US collected between January first 2021 and December 31st 2021. Early reports on the COVID‐19 pandemic showed that the severity of a SARS‐CoV‐2 infection tends to vary greatly by cases. We found that while demographic characteristics explain some of the differences in individuals' experience with COVID‐19, some unaccounted‐for latent variables were associated with the disease. [ABSTRACT FROM AUTHOR]

Even in large systems, the effect of noise arising from when populations are initially small can persist to be measurable on the macroscale. A deterministic approximation to a stochastic model will fail to capture this effect, but it can be accurately approximated by including an additional random time-shift to the initial conditions. We present a efficient numerical method to compute this time-shift distribution for a large class of stochastic models. The method relies on differentiation of certain functional equations, which we show can be effectively automated by deriving rules for different types of model rates that arise commonly when mass-action mixing is assumed. Explicit computation of the time-shift distribution can be used to build a practical tool for the efficient generation of macroscopic trajectories of stochastic population models, without the need for costly stochastic simulations. Full code is provided to implement the calculations and we demonstrate the method on an epidemic model and a model of within-host viral dynamics. [ABSTRACT FROM AUTHOR]

When initially introduced into a susceptible population, a disease may die out or result in a major outbreak. We present a Continuous- Time Markov Chain model for enzootic WNV transmission between two avian host species and a single vector, and use multitype branching process theory to determine the probability of disease extinction based upon the type of infected individual initially introducing the disease into the population -- an exposed vector, infectious vector, or infectious host of either species. We explore how the likelihood of disease extinction depends on the ability of each host species to transmit WNV, vector biting rates on host species, and the relative abundance of host species, as well as vector abundance. Theoretical predictions are compared to the outcome of stochastic simulations. We find the community composition of hosts and vectors, as well as the means of disease introduction, can greatly affect the probability of disease extinction. [ABSTRACT FROM AUTHOR]

When initially introduced into a susceptible population, a disease may die out or result in a major outbreak. We present a Continuous-Time Markov Chain model for enzootic WNV transmission between two avian host species and a single vector, and use multitype branching process theory to determine the probability of disease extinction based upon the type of infected individual initially introducing the disease into the population – an exposed vector, infectious vector, or infectious host of either species. We explore how the likelihood of disease extinction depends on the ability of each host species to transmit WNV, vector biting rates on host species, and the relative abundance of host species, as well as vector abundance. Theoretical predictions are compared to the outcome of stochastic simulations. We find the community composition of hosts and vectors, as well as the means of disease introduction, can greatly affect the probability of disease extinction.

AbstractWe propose a general framework for pricing both discretely and continuously monitored arithmetic average Asian options whose underlying asset price satisfies the rough stochastic local volatility model. We use a semimartingale approximation approach for the rough stochastic local volatility model to obtain its Markovian representation. For each type of Asian options, we use the double-layer continuous-time Markov chain to approximate the transformed underlying asset price, and derive the double transform of the Asian option price in terms of the unique bounded solution to a related functional equation. We invert the analytical double transforms of Asian option prices under the approximate continuous-time Markov chain numerically to obtain the approximations for Asian option prices. Monte Carlo simulation demonstrates the accuracy and efficiency of the proposed method for Asian option pricing. [ABSTRACT FROM AUTHOR]

In order to investigate the impact of two immunization strategies—vaccination targeting susceptible individuals to reduce their infection rate and clinical medical interventions targeting infected individuals to enhance their recovery rate—on the spread of infectious diseases in complex networks, this study proposes a bilinear SIR infectious disease model that considers bidirectional immunization. By analyzing the conditions for the existence of endemic equilibrium points, we derive the basic reproduction numbers and outbreak thresholds for both homogeneous and heterogeneous networks. The epidemic model is then reconstructed and extensively analyzed using continuous-time Markov chain (CTMC) methods. This analysis includes the investigation of transition probabilities, transition rate matrices, steady-state distributions, and the transition probability matrix based on the embedded chain. In numerical simulations, a notable concordance exists between the outcomes of CTMC and mean-field (MF) simulations, thereby substantiating the efficacy of the CTMC model. Moreover, the CTMC-based model adeptly captures the inherent stochastic fluctuation in the disease transmission, which is consistent with the mathematical properties of Markov chains. We further analyze the relationship between the system's steady-state infection density and the immunization rate through MCS. The results suggest that the infection density decreases with an increase in the immunization rate among susceptible individuals. The current research results will enhance our understanding of infectious disease transmission patterns in real-world scenarios, providing valuable theoretical insights for the development of epidemic prevention and control strategies. [ABSTRACT FROM AUTHOR]

The advancement of cloud computing technologies has positioned virtual machine (VM) migration as a critical area of research, essential for optimizing resource management, bolstering fault tolerance, and ensuring uninterrupted service delivery. This paper offers an exhaustive analysis of VM migration processes within cloud infrastructures, examining various migration types, server load assessment methods, VM selection strategies, ideal migration timing, and target server determination criteria. We introduce a queuing theory-based model to scrutinize VM migration dynamics between servers in a cloud environment. By reinterpreting resource-centric migration mechanisms into a task-processing paradigm, we accommodate the stochastic nature of resource demands, characterized by random task arrivals and variable processing times. The model is specifically tailored to scenarios with two servers and three VMs. Through numerical examples, we elucidate several performance metrics: task blocking probability, average tasks processed by VMs, and average tasks managed by servers. Additionally, we examine the influence of task arrival rates and average task duration on these performance measures. [ABSTRACT FROM AUTHOR]

The present paper considers a service system without the input flow of claims, that is, all claims are present in the queue initially from the beginning. The number of such claims equals Θmax, whereas the number of servers equals k. The time of one service has the Erlang distribution. The paper aims to calculate the time distribution until the completion of all services, as well as the average waiting time of one claim. [ABSTRACT FROM AUTHOR]

Dependency plays a key role in the design stage of multi-component manufacturing systems. To capture stochastic dependencies between subsystems, this paper considers a load-sharing k -out-of- n : G system which contains non-identical multi-state subsystems. The state evolution of each subsystem follows a continuous-time homogeneous Markov chain. Most existing research on k -out-of- n : G systems focuses on independent subsystems. However, a subsystem failure usually leads to a higher failure rate for each surviving subsystem within the system. This paper aims to study the interaction among subsystems and analyse the system reliability performance. The transition rate matrix for a given subsystem is assumed to depend on the total number of the remaining surviving subsystems, which can be represented as the sum of a baseline transition rate matrix and a stochastic dependency matrix. This paper proves that the transition rate matrix of the system is the generalised Kronecker sum of transition rate matrices of subsystems, and develops an explicit method to calculate the generalised Kronecker sum. The theory of aggregated stochastic processes is employed to obtain closed-form formulas for reliability indexes. A case study of multi-engine aircraft systems is provided where numerical examples are given to illustrate the developed model and obtained results. [ABSTRACT FROM AUTHOR]

A stochastic continuous-time Markov chain tuberculosis model with fast-slow progression and relapse is established to explore the impact of the demographic variation on TB transmission. At first, the extinction threshold and probability of the disease extinction and outbreak are obtained by applying the multitype Galton-Waston branching process for the stochastic model. In numerical simulations, the probability of the disease extinction and outbreak and expected epidemic duration of the disease are estimated. To see how demographic stochasticity affects TB dynamics, we compare dynamical behaviors of both stochastic and deterministic models, and these results show that the disease extinction in stochastic model would occur while the disease is persistent for the deterministic model. Our results suggest that minimizing the contact between the infectious and the susceptible, and detecting the latently infected as early as possible, etc., could effectively prevent the spread of tuberculosis.

With the large ongoing number of aged people and Alzheimer's disease (AD) patients worldwide, unpaid caregivers have become the primary sources of their daily caregiving. Alzheimer's family caregivers often suffer from physical and mental morbidities owing to various reasons. The aims of this paper were to develop alternate methods to understand the transition properties, the dynamic change, and the long‐run behavior of AD caregivers' stress levels, by assuming their transition to the next level only depends on the duration of the current stress level. In this paper, we modeled the transition rates in the semi‐Markov Process with log‐logistic hazard functions. We assumed the transition rates were non‐monotonic over time and the scale of transition rates depended on covariates. We also extended the uniform accelerated expansion to calculate the long‐run probability distribution of stress levels while adjusting for multiple covariates. The proposed methods were evaluated through an empirical study. The application results showed that all the transition rates of caregivers' stress levels were right skewed. Care recipients' baseline age was significantly associated with the transitions. The long‐run probability of severe state was slightly higher, implying a prolonged recovery time for severe stress patients. [ABSTRACT FROM AUTHOR]

This study is aimed at exploring the stochastic process in modelling the distribution of rainfall from some selected stations in Northern region of Nigeria, using continuous-time Markov chain to determine the level of its persistence based on its relative frequencies. The rainfall data were collected from the stations spread across the Sahel, Sudan and Guinea savanna. An exponential probability distribution was used to model the distribution of rainfall intensity after clustering the average rainfall experienced in all the stations. The extreme rainfall and the intensity dryness, over the recorded period, across all the stations in the Northern region were observed. We observed that change in climatic conditions of each station depends on the amount of rainfall experience annually. This study will help in simulating the likely rainfall to be experienced in each station of the Northern Nigeria. It will assist Meteorologists to make a short term probabilities prediction for Aviation sector and agricultural production. [ABSTRACT FROM AUTHOR]

The effects of seasonality on disease invasion and superinfection are investigated in dengue epidemic models with two viral strains. A two-strain host–vector nonautonomous ODE is formulated with seasonal periodicity in either vector recruitment or transmission. The basic reproduction number is derived and the dynamics of the single-strain periodic subsystem are summarized. Conditions for uniform persistence and existence of a single-strain periodic solution are verified and the invasion reproduction number is derived. A time-nonhomogeneous, continuous-time Markov chain (CTMC) is formulated from the ODE framework. The basic and invasion reproduction numbers apply to the ODE and CTMC. In the case of invasion, after one strain is established and the invasion reproduction number exceeds one, the ODE predicts a successful invasion, but in the CTMC, there is a periodic probability of invasion that depends on time and size of the invasion. A multitype branching process approximation of the CTMC provides an analytical method to predict the periodic probability of invasion. The numerical results show that the peak time of invasion is related to the current endemic strain and the dominant seasonal driver but it often precedes the peak time of the seasonal driver. [ABSTRACT FROM AUTHOR]

• A new stochastic SIRD model with imperfect immunity based on a continuous time Markov process is proposed. • We explore novel stochastic properties like the extinction and alarm time beside the infection and mortality time of a tagged case. • Theorems and recursive algorithms for the computation of the stochastic properties are presented. • An extensive sensitivity analysis reveals monotonous and unimodal tendencies for the considered stochastic indicators. • Health authorities can employ this information to achieve the best balance between health benefits and economic drawbacks. Efficient assessment of epidemic phenomena has an important role in modern epidemiology, while many novel methods propose reliable estimates for epidemic evolution in a population. In this paper, we focus on the stochastic modeling of a novel epidemiological (susceptible-infected-recovered-deceased) SIRD model with imperfect immunity based on a continuous-time Markov process adapted to the characteristics of the epidemic model. We investigate several novel stochastic properties of the SIRD scheme that are both population- and individual-oriented, such as the extinction and alert time of an epidemic, in parallel with the infection and mortality times of a tagged individual. We provide propositions and detailed recursive algorithms for computing probabilities and moments, giving additional information beyond the mean and variance of the stochastic quantities. Important remarks for representing higher-order matrices and system solving are provided, decreasing significantly the operation time of these algorithms. Extensive sensitivity analysis gives light to the influence of the system's parameters, while enhancing the validity of our methodology. Unlike other analyses that focus mainly on fitting the evolution of various diseases, our goal is to complement these studies by highlighting additional noteworthy features that determine an epidemic's future. The proposed methodology is applied on the monkeypox outbreak of 2022 in India and the first COVID infection wave in Barbados. Finally, public health authorities can use the information provided by these indicators to adjust the duration of lockdowns, accordingly, achieving the best possible balance between health benefits and economic disadvantages. Simultaneously, knowledge of mortality probabilities and timing of infection can support early coordination of hospitals and intensive care units, which could notably reduce the high risk of mortality. [ABSTRACT FROM AUTHOR]

We consider a variation of an Erlang loss system in which jobs are routed to servers according to the Shortest Idle Server First service discipline. Specifically, we consider a system in which idle servers are arranged in a stack; servers are returned to the top of the stack upon service completion; and arriving jobs are assigned to the server currently at the top of the stack. When busy, servers accumulate age and incur an age-dependent operating cost. For such systems, we (i) formulate a continuous-time Markov chain model to characterize the system's transient behavior, and (ii) develop maintenance policies consisting of two possible actions: server group replacement and stack inversion. The stack inversion may be performed at any time prior to group replacement to achieve a more evenly distributed utilization among servers. We develop an optimization model to determine the optimal inversion and replacement times so as to minimize the long-run expected cost rate. Because the model is nonlinear and non-convex, we develop a set of algorithms to solve for the optimal replacement and inversion time. Lastly, we establish a lower bound for the inversion cost threshold below which it is optimal to invert the stack of servers before their replacement. [ABSTRACT FROM AUTHOR]

Malaria is the world’s most fatal and challenging parasitic disease, caused by the Plasmodium parasite and transmitted to humans by the bites of infected female mosquitos. This study presents a stochastic process representing the evolution in time of an unexpected phenomenon. We use different probability techniques to assume the possible outcome in a Malaria model since state variables or parameters are random in a stochastic model. This study considers a deterministic vector-host malaria model and converts this model into the corresponding stochastic model with a Continuous-time Markov chain (CTMC) and Stochastic differential equation (SDE). We prove the positivity and boundedness of solutions and calculate the equilibrium points. The threshold values are evaluated using different approaches. Moreover, the Latin Hypercube Sampling (LHS) and Partial Rank Correlation Coefficients (PRCC) procedures are used to analyze the sensitivity of each model parameter. Finally, a transient numerical simulation is discussed, and their outcomes strongly correlate with the actual scenario.

Fifth generation (5G) networks support ultra-reliable low-latency communications (URLLC) and enhanced mobile broadband (eMBB). The coexistence of URLLC and eMBB is often organized by non-orthogonal multiple access (NOMA), giving priority to URLLC and resulting in eMBB quality of service (QoS) degradation. In this paper, we address the issue of joint URLLC and eMBB transmission, focusing on the problem from the perspective of delay-tolerant eMBB. Due to the priority given to URLLC, we assume that an eMBB session may be interrupted if there are no free resources available for URLLC or delayed when a new eMBB session arrives. To make the scheme more flexible, we propose that interrupted and delayed eMBB sessions periodically check for free resources, rather than continuously. To analyze this scenario, we propose a retrial queuing system with two retrial buffers (orbits) for interrupted and delayed eMBB sessions. The stationary probability distribution, provided in matrix form by recursive formulas, is presented. The paper concludes with a numerical example showing that the scheme with two buffers, compared to one buffer, practically doubles the average number of active eMBB sessions while keeping the interruption probability below 0.001. We provide an illustration of the configuration of eMBB retrial rates to meet its QoS requirements. [ABSTRACT FROM AUTHOR]

In oncology/hematology early phase clinical trials, efficacies were often observed in terms of response rate, depth, timing, and duration. However, the true clinical benefits that eventually support registrational purpose are progression‐free survival (PFS) and/or overall survival (OS), the follow‐up of which are typically not long enough in early phase trials. This gap imposes challenges in strategies for late phase drug development. In this article, we tackle the question by leveraging published study to establish a quantitative link between early efficacy outcomes and late phase efficacy endpoints. We used solid tumor cancer as disease model. We modeled the disease course of a RECISTv1.1 assessed solid tumor with a continuous Markov chain (CMC) model. We parameterize the transition intensity matrix of a CMC model based on published aggregate‐level summary statistics, and then simulate subject‐level time‐to‐event data. The simulated data is shown to have good approximation to published studies. PFS and/or OS could be predicted with the transition intensity matrix modified given clinical knowledge to reflect various assumptions on response rate, depth, timing, and duration. The authors have built a R shiny application named PubPredict, the tool implements the algorithm described above and allows customized features including multiple response levels, treatment crossover and varying follow‐up duration. This toolset has been applied to advise phase 3 trial design when only early efficacy data are available from phase 1 or 2 studies. [ABSTRACT FROM AUTHOR]

We consider the pricing of variable annuities (VAs) with general fee structures under a class of stochastic volatility models which includes the Heston, Hull-White, Scott, α-Hypergeometric, 3/2, and 4/2 models. In particular, we analyze the impact of different VIX-linked fee structures on the optimal surrender strategy of a VA contract with guaranteed minimum maturity benefit (GMMB). Under the assumption that the VA contract can be surrendered before maturity, the pricing of a VA contract corresponds to an optimal stopping problem with an unbounded, time-dependent, and discontinuous payoff function. We develop efficient algorithms for the pricing of VA contracts using a two-layer continuous-time Markov chain approximation for the fund value process. When the contract is kept until maturity and under a general fee structure, we show that the value of the contract can be approximated by a closed-form matrix expression. We also provide a quick and simple way to determine the value of early surrenders via a recursive algorithm and give an easy procedure to approximate the optimal surrender surface. We show numerically that the optimal surrender strategy is more robust to changes in the volatility of the account value when the fee is linked to the VIX index. [ABSTRACT FROM AUTHOR]

In order to investigate the impact of two immunization strategies—vaccination targeting susceptible individuals to reduce their infection rate and clinical medical interventions targeting infected individuals to enhance their recovery rate—on the spread of infectious diseases in complex networks, this study proposes a bilinear SIR infectious disease model that considers bidirectional immunization. By analyzing the conditions for the existence of endemic equilibrium points, we derive the basic reproduction numbers and outbreak thresholds for both homogeneous and heterogeneous networks. The epidemic model is then reconstructed and extensively analyzed using continuous-time Markov chain (CTMC) methods. This analysis includes the investigation of transition probabilities, transition rate matrices, steady-state distributions, and the transition probability matrix based on the embedded chain. In numerical simulations, a notable concordance exists between the outcomes of CTMC and mean-field (MF) simulations, thereby substantiating the efficacy of the CTMC model. Moreover, the CTMC-based model adeptly captures the inherent stochastic fluctuation in the disease transmission, which is consistent with the mathematical properties of Markov chains. We further analyze the relationship between the system’s steady-state infection density and the immunization rate through MCS. The results suggest that the infection density decreases with an increase in the immunization rate among susceptible individuals. The current research results will enhance our understanding of infectious disease transmission patterns in real-world scenarios, providing valuable theoretical insights for the development of epidemic prevention and control strategies.

The advancement of cloud computing technologies has positioned virtual machine (VM) migration as a critical area of research, essential for optimizing resource management, bolstering fault tolerance, and ensuring uninterrupted service delivery. This paper offers an exhaustive analysis of VM migration processes within cloud infrastructures, examining various migration types, server load assessment methods, VM selection strategies, ideal migration timing, and target server determination criteria. We introduce a queuing theory-based model to scrutinize VM migration dynamics between servers in a cloud environment. By reinterpreting resource-centric migration mechanisms into a task-processing paradigm, we accommodate the stochastic nature of resource demands, characterized by random task arrivals and variable processing times. The model is specifically tailored to scenarios with two servers and three VMs. Through numerical examples, we elucidate several performance metrics: task blocking probability, average tasks processed by VMs, and average tasks managed by servers. Additionally, we examine the influence of task arrival rates and average task duration on these performance measures.

5G networks are expected to deliver very high data rates with enhanced quality of service (QoS) at a very low cost. To meet the demands of high data rates and improved QoS, ultra-densification is the key solution used in 5G networks. In this paper we propose a call admission control (CAC) policy by considering the main components of an ultra-dense 5G network, namely, macrocell, picocell and femtocell. The calls in the network are accepted based on the SNIR ranges which help in deciding whether the call will be connected to a macrocell or picocell or femtocell. We also come up with a handover management scheme which regulates the handover of calls between macrocell to macrocell (picocell or femtocell), picocell to macrocell (picocell or femtocell) and femtocell to macrocell (picocell or femtocell). The performance analysis of the proposed CAC policy and the handover management scheme is also conducted in the paper. For this, we construct analytical models with the help of continuous-time Markov chains for the macrocell, picocell and femtocell network to obtain the performance metrics such as blocking probability of new calls and dropping probability of handover calls. The analytical results are further investigated to establish the importance of ultra-densification in 5G network. The proposed analytical models are also validated via simulation. A comparative analysis is also performed between the proposed CAC policy and an existing CAC policy in literature which considers only macrocells and femtocells in the network. [ABSTRACT FROM AUTHOR]

Long-term behaviors of biochemical systems are described by steady states in deterministic models and stationary distributions in stochastic models. Obtaining their analytic solutions can be done for limited cases, such as linear or finite-state systems, as it generally requires solving many coupled equations. Interestingly, analytic solutions can be easily obtained when underlying networks have special topologies, called weak reversibility (WR) and zero deficiency (ZD), and the kinetic law follows a generalized form of mass-action kinetics. However, such desired topological conditions do not hold for the majority of cases. Thus, translating networks to have WR and ZD while preserving the original dynamics was proposed. Yet, this approach is limited because manually obtaining the desired network translation among the large number of candidates is challenging. Here, we prove necessary conditions for having WR and ZD after translation, and based on these conditions, we develop a user-friendly computational package, TOWARDZ, that automatically and efficiently identifies translated networks with WR and ZD. This allows us to quantitatively examine how likely it is to obtain WR and ZD after translation depending on the number of species and reactions. Importantly, we also describe how our package can be used to analytically derive steady states of deterministic models and stationary distributions of stochastic models. TOWARDZ provides an effective tool to analyze biochemical systems. [ABSTRACT FROM AUTHOR]

We investigate the valuation of exchange options when the market is affected by changing economic conditions as well as liquidity risks. The volatility and expected returns of both stocks are assumed to be controlled by a continuous‐time Markov chain to reflect the effects of varying economic conditions, and a liquidity discounting factor is employed to capture the impact of market liquidity on stock prices. Once the model has been established, we construct a risk‐neutral measure with the use of regime‐switching Esscher transform, and the characteristic function is then derived in an analytical form, so that a closed‐form formula for exchange options can be presented. We further analyze the effects of the two considered factors on exchange option prices numerically. [ABSTRACT FROM AUTHOR]

Bayesian inference for Continuous-Time Markov chains (CTMCs) on countably infinite spaces is notoriously difficult because evaluating the likelihood exactly is intractable. One way to address this challenge is to first build a nonnegative and unbiased estimate of the likelihood—involving the matrix exponential of finite truncations of the true rate matrix—and then to use the estimates in a pseudo-marginal inference method. In this work, we show that we can dramatically increase the efficiency of this approach by avoiding the computation of exact matrix exponentials. In particular, we develop a general methodology for constructing an unbiased, nonnegative estimate of the likelihood using doubly-monotone matrix exponential approximations. We further develop a novel approximation in this family—the skeletoid—as well as theory regarding its approximation error and how that relates to the variance of the estimates used in pseudo-marginal inference. Experimental results show that our approach yields more efficient posterior inference for a wide variety of CTMCs. for this article are available online. [ABSTRACT FROM AUTHOR]

Stochastic modeling of chemical reaction systems based on master equations has been an indispensable tool in physical sciences. In the long-time limit, the properties of these systems are characterized by stationary distributions of chemical master equations. In this paper, we describe a novel method for computing stationary distributions analytically, based on a parallel formalism between stochastic chemical reaction systems and second quantization. Anderson, Craciun, and Kurtz showed that, when the rate equation for a reaction network admits a complex-balanced steady-state solution, the corresponding stochastic reaction system has a stationary distribution of a product form of Poisson distributions. In a formulation of stochastic reaction systems using the language of second quantization initiated by Doi, product-form Poisson distributions correspond to coherent states. Pursuing this analogy further, we study the counterpart of squeezed states in stochastic reaction systems. Under the action of a squeeze operator, the time-evolution operator of the chemical master equation is transformed, and the resulting system describes a different reaction network, which does not admit a complex-balanced steady state. A squeezed coherent state gives the stationary distribution of the transformed network, for which analytic expression is obtained. [ABSTRACT FROM AUTHOR]

The more complex the software system, the greater the number of possible combinations of defects that can cause errors, resulting in software defects that are difficult to isolate and expensive to correct in the development stage. An essential feature of such defects is a gradual deterioration in software performance finishing with software failure—software aging. Mobile devices are particularly vulnerable to software aging. Thus, there is a constant need for methods and tools to eliminate the effects of aging in mobile systems based on modeling the aging process, including the study of metrics and aging factors and the development of more reliable and adequate aging and rejuvenation models. This paper summarizes the previously developed Android software aging and rejuvenation models and presents a comprehensive model of aging and rejuvenation for the Android operating system. The comprehensive model is based on continuous-time Markov Chains and considers different aging levels, mobile device activity, and battery status. The aging and rejuvenation model can be used to assess the software quality, allows obtaining expressions for indicators of software rejuvenation efficiency, and can be used to design and select parameters of the software rejuvenation method considering battery saving. [ABSTRACT FROM AUTHOR]

This work has three important purposes: first it is the study of Markov Chains, the second is to show that Markov chains have different applications and finally it is to model a process of this behaves. Throughout this work we will describe what a Markov chain is, what these processes are for and how these chains are classified. We will describe a Markov Chain, that is, analyze what are the primary elements that make up a Markov chain, among others.

We present new methodologies for Bayesian inference on the rate parameters of a discretely observed continuous-time Markov jump process with a countably infinite statespace. The usual method of choice for inference, particle Markov chain Monte Carlo (particle MCMC), struggles when the observation noise is small. We consider the most challenging regime of exact observations and provide two new methodologies for inference in this case: the minimal extended statespace algorithm (MESA) and the nearly minimal extended statespace algorithm (nMESA). By extending the Markov chain Monte Carlo statespace, both MESA and nMESA use the exponentiation of finite rate matrices to perform exact Bayesian inference on the Markov jump process even though its statespace is countably infinite. Numerical experiments show improvements over particle MCMC of between a factor of three and several orders of magnitude. for this article are available online. [ABSTRACT FROM AUTHOR]

Ambulance offload delays have become a challenging concern for emergency healthcare service providers. These delays often occur when the number of patients in the emergency department (ED) exceed the designed capacity such that ED cannot accept an incoming patient immediately, thereby forcing the ambulance and crew to wait with the patient until a bed becomes available. In this paper, we analyze and optimize the emergency medical services network, including ambulance stations and EDs. The objective is to reduce ambulance offload delays and lessen the congestion of EDs. To this end, we first build a continuous-time Markov chain to characterize this network analytically. Next, from the perspectives of both ambulance stations and EDs, we develop resource configuration and optimization models for this network. We investigate the reasons for ED overcapacity and ambulance offload delays. Finally, we design an effective approach to reconfigure the resources in the emergency medical services network, leading to a new and better equilibrium. Note to Practitioners—This article is motivated by our collaborations with the Emergency Medical Service Center (also called 120 Center) and several hospitals in Shanghai, China. The Emergency Medical Service Center and ED of hospitals are the frontlines of healthcare services in Shanghai. They provide medical treatment services for acutely ill and injured patients, so the operation of this system is critical to the health of such patients. Today, the ambulance offloading delay poses a challenge to the Emergency Medical Service Center, as it reduces the usage of ambulances and crews, as well as putting patients at risk. Meanwhile, the EDs sometimes suffer from bed shortages and overcrowding. Emergency Medical Service Centers and hospitals are both striving to improve the performance of this system. We analyze the network, including both the ambulance station (AS) and ED, and formulate a continuous-time Markov chain model to describe the network states. Then, we propose two optimization models from both AS and ED perspectives with a series of approximation methods to overcome computational difficulties. We obtain the equilibrium through joint optimization of AS and ED and demonstrate some valuable insights for managing ambulances and beds in the ED. Methods presented in this article may help decision-makers in emergency medical services systems. [ABSTRACT FROM AUTHOR]

Blockchain technology has gained prominence over the last decade. Numerous achievements have been made regarding how this technology can be utilized in different aspects of the industry, market, and governmental departments. Due to the safety-critical and security-critical nature of their uses, it is pivotal to model the dependability of blockchain-based systems. In this study, we focus on Bitcoin, a blockchain-based peer-to-peer cryptocurrency system. A continuous-time Markov chain-based analytical method is put forward to model and quantify the dependability of the Bitcoin system under selfish mining attacks. Numerical results are provided to examine the influences of several key parameters related to selfish miners’ computing power, attack triggering, and honest miners’ recovery capability. The conclusion made based on this research may contribute to the design of resilience algorithms to enhance the self-defense and robustness of cryptocurrency systems.

Many important stochastic counting models can be written as general birth-death processes (BDPs). BDPs are continuous-time Markov chains on the non-negative integers in which only jumps to adjacent states are allowed. BDPs can be used to easily parameterize a rich variety of probability distributions on the non-negative integers, and straightforward conditions guarantee that these distributions are proper. BDPs also provide a mechanistic interpretation - birth and death of actual particles or organisms - that has proven useful in evolution, ecology, physics, and chemistry. Although the theoretical properties of general BDPs are well understood, traditionally statistical work on BDPs has been limited to the simple linear (Kendall) process. Aside from a few simple cases, it remains impossible to find analytic expressions for the likelihood of a discretely-observed BDP, and computational difficulties have hindered development of tools for statistical inference. But the gap between BDP theory and practical methods for estimation has narrowed in recent years. There are now robust methods for evaluating likelihoods for realizations of BDPs: finite-time transition, first passage, equilibrium probabilities, and distributions of summary statistics that arise commonly in applications. Recent work has also exploited the connection between continuously- and discretely-observed BDPs to derive EM algorithms for maximum likelihood estimation. Likelihood-based inference for previously intractable BDPs is much easier than previously thought and regression approaches analogous to Poisson regression are straightforward to derive. In this review, we outline the basic mathematical theory for BDPs and demonstrate new tools for statistical inference using data from BDPs.

Continuous-time Markov chains have transition matrices that vary continuously in time. Classical theory of nonnegative matrices, M-matrices and matrix exponentials is used in the literature to study their dynamics, probability distributions and other stochastic properties. For the benefit of Perron-Frobenius cognoscentes, this theory is surveyed and further adapted to study continuous-time Markov chains on finite state spaces.

The object under investigation is a controllable linear stochastic differential system affected by some external statistically uncertain piecewise continuous disturbances. They are directly unobservable but assumed to be a continuous-time Markov chain. The problem is to stabilize the system output concerning a quadratic optimality criterion. As is known, the separation theorem holds for the system. The goal of the paper is performance analysis of various numerical schemes applied to the filtering of the external Markov input for system stabilization purposes. The paper briefly presents the theoretical solution to the considered problem of optimal stabilization for systems with the Markov jump external disturbances: the conditions providing the separation theorem, the equations of optimal control, and the ones defining the Wonham filter. It also contains a complex of the stable numerical approximations of the filter, designed for the time-discretized observations, along with their accuracy characteristics. The approximations of orders 1 2 , 1, and 2 along with the classical Euler–Maruyama scheme are chosen for the comparison of the Wonham filter numerical realization. The filtering estimates are used in the practical stabilization of the various linear systems of the second order. The numerical experiments confirm the significant influence of the filtering precision on the stabilization performance and superiority of the proposed stable schemes of numerical filtering. [ABSTRACT FROM AUTHOR]

We study the implementation of the optimal control strategy obtained in [1] and supplemented in [2]. The algorithm for optimal stabilization of a linear stochastic differential system in a position determined by a piecewise constant Markov drift has been tested in a substantial number of model experiments. The drift value is observed indirectly; i.e., the control problem is solved in the statement with incomplete information. Practical implementation is complicated by the instability of Euler–Maruyama numerical schemes that implement the Wonham filter, which is a key element of the optimal control strategy. To perform calculations, the Wonham filter is approximated by stable schemes based on the optimal filtering of Markov chains by discretized observations [3]. These schemes have different implementation complexity and orders of accuracy. The paper presents a comparative analysis of the control performance for various stable approximations to the Wonham filter and its typical implementation using the Euler–Maruyama scheme. In addition, three versions of discretized filters are compared and final recommendations are given for their application in the problem of stabilizing a system with hopping drift. [ABSTRACT FROM AUTHOR]

Stochastic epidemic models describe the dynamics of an epidemic as a disease spreads through a population. Typically, only a fraction of cases are observed at a set of discrete times. The absence of complete information about the time evolution of an epidemic gives rise to a complicated latent variable problem in which the state space size of the epidemic grows large as the population size increases. This makes analytically integrating over the missing data infeasible for populations of even moderate size. We present a data augmentation Markov chain Monte Carlo (MCMC) framework for Bayesian estimation of stochastic epidemic model parameters, in which measurements are augmented with subject-level disease histories. In our MCMC algorithm, we propose each new subject-level path, conditional on the data, using a time-inhomogeneous continuous-time Markov process with rates determined by the infection histories of other individuals. The method is general, and may be applied to a broad class of epidemic models with only minimal modifications to the model dynamics and/or emission distribution. We present our algorithm in the context of multiple stochastic epidemic models in which the data are binomially sampled prevalence counts, and apply our method to data from an outbreak of influenza in a British boarding school.

Modeling of spectrum occupancy by both primary and secondary users in the Cognitive Radio Ad-hoc Network (CRAHN) can be helpful to improve spectrum usage efficiency. Modelling the prioritized spectrum access policy is a significant need to analyze a CRAHN. Sensing the spectrum is a basic aspect in a CRAHN, and in the design of sensing strategies, there are some trade-off between the PU occupancy rate, throughput, and delay and energy consumption. This paper provides a Continuous-Time Markov chain (CTMC) to describe the spectrum usage by both primary and secondary users in a CRAHN that enables the evaluation of throughput and delay. The steady-state analysis is performed to analytically derive the probability of this new packet loss in CRAHN. In addition, the average end-to-end delay together with the throughput, as two main parameters of quality of service, are improved as per the modification based on the proposed CTMC model. The results derived by these parameters are compared with some well-known transport protocols. The comparison confirms the accuracy of the proposed model. The experimental results have justified the fitness of the joint analysis model in the practical environment using the NS2. [ABSTRACT FROM AUTHOR]

We study a two-dimensional discounted optimal stopping problem related to the pricing of perpetual commodity equities in a model of financial markets in which the behaviour of the underlying asset price follows a generalized geometric Brownian motion and the dynamics of the convenience yield are described by an unobservable continuous-time Markov chain with two states. It is shown that the optimal time of exercise is the first time at which the commodity spot price paid in return to the fixed coupon rate hits a lower stochastic boundary being a monotone function of the running value of the filtering estimate of the state of the chain. We rigorously prove that the optimal stopping boundary is regular for the stopping region relative to the resulting two-dimensional diffusion process and the value function is continuously differentiable with respect to the both variables. It is verified by means of a change-of-variable formula with local time on surfaces that the value function and the boundary are determined as a unique solution of the associated parabolic-type free-boundary problem. We also give a closed-form solution to the optimal stopping problem for the case of an observable Markov chain. [ABSTRACT FROM AUTHOR]

Sensitivity analysis plays a critical role in quantifying uncertainty in the design of computer systems. In particular, a var iance-based global sensitivity analysis is often used to rank the importance of input factors based on their contribution to the variance of the output measure of interest. The variance-based sensitivity analysis is sampling-based and therefore usually applies simulation methods such as Monte Carlo simulation. That means the traditional methods for variance-based sensitivity analysis based on simulation do not need the analytic structure of the model to be analyzed. However, the simulation usually needs a huge number of realizations to obtain stable results, which incurs an undesired high computational cost. In this paper, we present an analytic approach to compute the variance-based sensitivity based on moment approximation. More specifically, we formulate the output measure of continuous-time Markov chains (CTMCs) and investigate the relationship between input parameters and output measure through variance-based sensitivity analysis. The numerical results showing the main effects of model parameters in both parallel and series system configurations indicate that a component's effect on the uncertainty in system reliability depends largely on the system structure. [ABSTRACT FROM AUTHOR]

We consider a model for a queue in which only a fixed number N of customers can join. Each customer joins the queue independently at an exponentially distributed time. Assuming further that the service times are independent and follow an exponential distribution, this system can be described as a two-dimensional Markov chain on a finite triangular region S of the square lattice. We interpret the resulting random walk on S as a Dyck path that is weighted according to some state-dependent transition probabilities that are constant along one axis, but are rather general otherwise. We untangle the resulting intricate combinatorial structure by introducing appropriate generating functions that exploit the recursive structure of the model. This allows us to derive an explicit expression for the probability mass function of the number of customers served in any busy period (equivalently, of the length of any excursion of the Dyck path above the diagonal) as a weighted sum with alternating sign over a certain subclass of Dyck paths. [ABSTRACT FROM AUTHOR]

In this paper, we consider a zero-sum differential game that can be stopped by any of the participants at one of time points known in advance. The cost functional may depend both on the time when the game ends and the position of the system at that time and on the player who initiates the game stopping. When optimizing the expectation of this functional, each player, based on the information the player has about the realized trajectory of the system, makes decisions both about his/her probability of stopping the game and about his/her own control of the conflict-controlled system; however, nondeterministic distribution rules are also allowed. The existence of the game value is shown under the assumption of a saddle point condition in the small game (the Isaacs condition). The construction of a stochastic guide whose model is a continuous-time Markov chain was used to construct approximately optimal strategies for the players. [ABSTRACT FROM AUTHOR]

HIV infection is still a serious worldwide public health problem. During the early stage of HIV infection, the number of infected cells and viruses are extremely low and the process of infection is stochastic. In this paper, we use a stochastic model consisting of continuous-time Markov chain to investigate CTL immune response and two modes of infection (viral transmission and cellular transmission). A multitype branching process is used to approximate the probability of clearance and outbreak of HIV infection near the infection-free equilibrium. In addition, we compare the dynamics of the deterministic model with the results of the stochastic model, and the differences are as follows: (1) when R 0 > 1 , the deterministic model shows HIV infection outbreak, while the stochastic model indicates a positive probability of HIV infection elimination; (2) the initial number of infected cells and viruses have no effect on the dynamics of the deterministic model, whereas the dynamics of the stochastic model are highly dependent on the initial sizes and cannot be ignored. [ABSTRACT FROM AUTHOR]

We consider a zero-sum differential game on a finite interval, in which the players not only control the system's trajectory but also influence the terminal time of the game. It is assumed that the early terminal time is an absolutely continuous random variable, and its density is given by bounded measurable functions of time assigned by both players (the intensities of the influence of each player on the termination of the game). The payoff function may depend both on the terminal time of the game together with the position of the system at this time and on the player who initiates the termination. The strategies are formalized by using nonanticipating càdlàg processes. The existence of the game value is shown under the Isaacs condition. For this, the original game is approximated by an auxiliary game based on a continuous-time Markov chain, which depends on the controls and intensities of the players. Based on the strategies optimal in this Markov game, a control procedure with a stochastic guide is proposed for the original game. It is shown that, under an unlimited increase in the number of points in the Markov game, this procedure leads to a near-optimal strategy in the original game. [ABSTRACT FROM AUTHOR]

We study zero-sum optimal stopping games associated with perpetual convertible bonds in an extension of the Black-Merton-Scholes model with random dividends under various information flows. In this type of contracts, the writers have the right to withdraw the bonds, before the holders convert them into assets. We derive closed-form expressions for the associated value function and optimal exercise boundaries in the model with an accessible dividend rate policy which is described by a continuous-time Markov chain with two states. We further consider the optimal stopping game in the model with inaccessible dividend rate policy and prove that the optimal exercise times are the first times at which the asset price process hits monotone boundaries depending on the running state of the filtering dividend rate estimate. We finally present the value of the optimal stopping game for the model in which the dividend rate policy is accessible to the writers but remains inaccessible to the holders of the bonds. [ABSTRACT FROM AUTHOR]

In the given paper a general analytical model for a queuing system with limited capacity buffer intended to control packets' traffic in Internet of Things applications is proposed. This model is based on the following assumptions. There is a fixed number of packets' classes. For each pair of these classes either preemptive or non-preemptive priority is set. Packets of each class arrive according to the Poisson process with the given arrival rate, and are transmitted without errors with the given transmission rate. The criterion on the structure of the set of priorities between the classes of packets avoiding unnecessary push-out of packets being in the transmission is proved. Continuous-Time Markov Chain associated with the proposed model has been defined and analyzed. Basic characteristics including blocking probability, push-out probability, delay, and utilization have been estimated for each class of packets. Basic measures for the proposed model, such as Grade of Service, cost function of operation, and performance are established.

Abstract We propose a unified framework to represent a wide range of continuous-time discrete-state Markov processes on networks, and show how many network dynamics models in the literature can be represented in this unified framework. We show how a particular sub-set of these models, referred to here as single-vertex-transition (SVT) processes, lead to the analysis of quasi-birth-and-death (QBD) processes in the theory of continuous-time Markov chains. We illustrate how to analyse a number of summary statistics for these processes, such as absorption probabilities and first-passage times. We extend the graph-automorphism lumping approach [Kiss, Miller, Simon, Mathematics of Epidemics on Networks, 2017; Simon, Taylor, Kiss, J. Math. Bio. 62(4), 2011], by providing a matrix-oriented representation of this technique, and show how it can be applied to a very wide range of dynamical processes on networks. This approach can be used not only to solve the master equation of the system, but also to analyse the summary statistics of interest. We also show the interplay between the graph-automorphism lumping approach and the QBD structures when dealing with SVT processes. Finally, we illustrate our theoretical results with examples from the areas of opinion dynamics and mathematical epidemiology.

Cognitive Radio Ad-hoc Network (CRAHN) is one of the building blocks of heterogeneous networks to improve accessibility and spectral efficiency. The prioritized spectrum access is the key to maintain the efficiency of CRAHN. However, in CRAHN, TCP results in severe throughput reduction since it cannot differentiate between the packet loss due to SU's transmission-blocking and packet loss due to congestion. In this paper, two significant events are investigated that caused secondary user blocking. Besides, using a Continuous-Time Markov chain (CTMC), we develop an analytical model to estimate the TCP throughput and end to end delay. The experimental results based on the NS2 confirm the accuracy of the proposed model and show that the throughput and the average end-to-end delay are improved based on the proposed CTMC model comparing with some transport protocol in the cognitive radio networks. The performance results through simulation show that the proposed model not only provides less energy consumption, also achieves up to 40% and 20% improvement of the throughput comparing with the classical TFRC and TFRC-CR, respectively. [ABSTRACT FROM AUTHOR]