Search

In this paper, we consider a zero-sum differential game that can be stopped by any of the participants at one of time points known in advance. The cost functional may depend both on the time when the game ends and the position of the system at that time and on the player who initiates the game stopping. When optimizing the expectation of this functional, each player, based on the information the player has about the realized trajectory of the system, makes decisions both about his/her probability of stopping the game and about his/her own control of the conflict-controlled system; however, nondeterministic distribution rules are also allowed. The existence of the game value is shown under the assumption of a saddle point condition in the small game (the Isaacs condition). The construction of a stochastic guide whose model is a continuous-time Markov chain was used to construct approximately optimal strategies for the players. [ABSTRACT FROM AUTHOR]

In this paper, the pure strategy subgame perfect equilibria of a general class of stopping time games are studied. It is shown that there always exists a natural class of Markov Perfect Equilibria, called stopping equilibria. Such equilibria can be computed as a solution of a single agent stopping time problem, rather than of a fixed point problem. A complete characterization of stopping equilibria is presented. Conditions are given under which the outcomes of such equilibria span the set of all possible outcomes from perfect equilibria. Two economic applications of the theory, product innovations and the timing of asset sales, are discussed. [ABSTRACT FROM AUTHOR]

A multi-armed bandit with finitely many arms is studied when each arm is a homogeneous Markov process on an underlying finite state space. The transition law of one of the arms, referred to as the odd arm, is different from the common transition law of all other arms. A learner, who has no knowledge of the above transition laws, has to devise a sequential test to identify the index of the odd arm as quickly as possible, subject to an upper bound on the probability of error. For this problem, we derive an asymptotic lower bound on the expected stopping time of any sequential test of the learner, where the asymptotics is as the probability of error vanishes. Furthermore, we propose a sequential test, and show that the asymptotic behaviour of its expected stopping time comes arbitrarily close to that of the lower bound. Prior works deal with independent and identically distributed arms, whereas our work deals with Markov arms. Our analysis of the rested Markov setting is a key first step in understanding the difficult case of restless Markov setting, which is still open. [ABSTRACT FROM AUTHOR]

Spectrum sensing is a key technology enabling the cognitive radio system. In this paper, the problem of how to quickly and accurately find an unoccupied channel from a large amount of potential channels is considered. The cognitive radio system under consideration is equipped with a narrow band sensor, hence it can only sense those potential channels in a sequential manner. In this scenario, we propose a novel two-stage mixed-observation sensing strategy. In the first stage, which is named as scanning stage, the sensor observes a linear combination of the signals from a pair of channels. The purpose of the scanning stage is to quickly identify a pair of channels such that at least one of them is highly likely to be unoccupied. In the second stage, which is called refinement stage, the sensor only observers the signal from one of those two channels identified from the first stage, and selects one of them as the unoccupied channel. The problem under this setup is an ordered two concatenated Markov stopping time problem. The optimal solution is solved using the tools from the multiple stopping time theory. It turns out that the optimal solution has a rather complex structure, hence a low complexity algorithm is proposed to facilitate the implementation. In the proposed low complexity algorithm, the cumulative sum test is adopted in the scanning stage and the sequential probability ratio test is adopted in the refinement stage. The performance of this low complexity algorithm is analyzed when the presence of unoccupied channels is rare. Numerical simulation results show that the proposed sensing strategy can significantly reduce the sensing time when the majority of potential channels are occupied. [ABSTRACT FROM PUBLISHER]

The paper considers the problem of optimal control in models with disorder and provides the computational scheme of its solution. The proposed method is based on replacing the disorder with its estimate by the stopping time. The calculations take place on the binary tree with nodes divided into two classes. The division of tree nodes into two classes is carried out using a decision rule associated either with testing two hypothesis, or with the optimal stopping of a Markov process developing on a binary tree. The estimate of the disorder is the moment at which the walk on the tree is placed in a node of the first class for the first time. The example of the optimal control problem is related to the problem of quantile hedging of a payoff. [ABSTRACT FROM AUTHOR]

This paper considers single-item inventory systems with immediate delivery and no economies of scale. Bounds are provided on the value loss relative to optimal cost for restricting attention to the class of inventory stocking policies that behave myopically up to a specified stopping time. The stopping time may be random, and may depend on demand histories as well as information exogenous to the firm. The bounds are robust to the nature of the demand process faced after the stopping time, so are applicable when the statistics of demand after the stopping time are unknown. Stopping times that are large with high probability imply that near-term decisions are completely specified with high probability. The general bounding results allow one to consider demand processes that may otherwise be analytically intractable. It is shown that the class of demand models for which the assumption of additive i.i.d. shocks is appropriate is the same class admitting effective myopic stocking policies. The bounds also make rigorous the intuitive notion that myopic policies are least effective in systems with precipitous drops in demand coupled with an inability to recover cash invested in inventory. The option of selling inventory at discount is a reality in many real systems and enhances the attractiveness of the myopic stocking policy. In numerical examples, the myopic policy is shown to be an effective competitor in a range of systems for which optimal policies are not known. The results suggest that for systems with immediate delivery, no economics of scale, and no currently known optimal policy, the myopic stocking rule is a reasonable default policy to adopt. [ABSTRACT FROM AUTHOR]

The paper discusses several techniques which may be used for applying the coupling method to solutions of stochastic differential equations (SDEs). The coupling techniques traditionally consist of two components: one is local mixing, the other is recurrence. Often in the articles they do not split. Yet, they are quite different in their nature, and this paper separates them, concentrating only on the former. Most of the techniques discussed here work in dimension d ≥ 1, although, in d = 1 there is one additional option to use intersections of trajectories, which requires nothing but the strong Markov property and nondegeneracy of the diffusion coefficient. In dimensions d > 1 it is possible to use embedded Markov chains either by considering discrete times n = 0, 1, ..., or by arranging special stopping time sequences and to use the local Markov-Dobrushin (MD) condition, which is one of the most efficient versions of local mixing. Further applications may be based on one or another version of the MD condition; respectively, this paper is devoted to various methods of verifying one or another form of it. [ABSTRACT FROM AUTHOR]

The problem of exactly generating a general random process (target process) by using another general random process (coin process) is studied. The performance of the interval algorithm, introduced by Han and Hoshi, is analyzed from the perspective of information spectrum approach. When either the coin process or the target process has one point spectrum, the asymptotic optimality of the interval algorithm among any random number generation algorithms is proved, which demonstrates utility of the interval algorithm beyond the ergodic process. Furthermore, the feasibility condition of exact random number generation is also elucidated. Finally, the obtained general results are illustrated by the case of generating a Markov process from another Markov process. [ABSTRACT FROM AUTHOR]

This research is devoted to studying a geometric Brownian motion with drift switching driven by a 2 χ 2 Markov chain. A discrete-time multiplicative approximation scheme was developed, and its convergence in Skorokhod topology to the continuous-time geometric Brownian motion with switching has been proved. Furthermore, in a financial market where the discounted asset price follows a geometric Brownian motion with drift switching, market incompleteness was established, and multiple equivalent martingale measures were constructed. [ABSTRACT FROM AUTHOR]

In this paper, we develop an energy-based, large-scale dynamical system model driven by Markov diffusion processes to present a unified framework for statistical thermodynamics predicated on a stochastic dynamical systems formalism. Specifically, using a stochastic state space formulation, we develop a nonlinear stochastic compartmental dynamical system model characterized by energy conservation laws that is consistent with statistical thermodynamic principles. In particular, we show that the difference between the average supplied system energy and the average stored system energy for our stochastic thermodynamic model is a martingale with respect to the system filtration. In addition, we show that the average stored system energy is equal to the mean energy that can be extracted from the system and the mean energy that can be delivered to the system in order to transfer it from a zero energy level to an arbitrary nonempty subset in the state space over a finite stopping time. [ABSTRACT FROM AUTHOR]

In epidemic modeling, interpretation of compartment quantities, such as s, i, and r in relevant equations, is not always straightforward. Ambiguities regarding whether these quantities represent numbers or fractions of individuals in each compartment rise questions about significance of the involved parameters. In this paper, we address these challenges by considering a density-dependent epidemic modelling by a birth-death process approach inspired by Kurtz from 1970s'. In contrast to existing literature, which employs population size scaling under constant population condition, we scale with respect to the area. Namely, under the assumption of spatial homogeneity of the population, we consider the quantities of susceptible, infective and recovered per unit area. This spatial scaling allows diffusion approximation for birth-death type epidemic models with varying population size. By adopting this approach, we anticipate to contribute to a clear and transparent description of compartment quantities and parameters in epidemic modeling. [ABSTRACT FROM AUTHOR]

This paper deals with the ruin problem of an insurance company investing its capital reserve in a risky asset with the price dynamics given by a conditional geometric Brownian motion whose parameters depend on a Markov process describing random variations in the economic and financial environments. We prove a sufficient condition on the distribution of jumps of the business process ensuring the smoothness of the ruin probability as a function of the initial capital and obtain for this function an integro-differential equation. [ABSTRACT FROM AUTHOR]

The goal of this paper is to establish Green function estimates for a class of purely discontinuous symmetric Markov processes with jump kernels degenerate at the boundary and critical killing potentials. The jump kernel and the killing potential depend on several parameters. We establish sharp two-sided estimates on the Green functions of these processes for all admissible values of the parameters involved. Depending on the regions where the parameters belong, the estimates on the Green functions are different. In fact, the estimates have three different forms. As applications, we prove that the boundary Harnack principle holds in certain region of the parameters and fails in some other region of the parameters. Together with the main results of our previous paper [Potential Anal., online, 2021], we completely determine the region of the parameters where the boundary Harnack principle holds. [ABSTRACT FROM AUTHOR]

This paper deals with minimization of the variances of the total discounted costs for constrained Continuous-Time Markov Decision Processes (CTMDPs). The costs consist of cumulative costs incurred between jumps and instant costs incurred at jump epochs. We interpret discounting as an exponentially distributed stopping time. According to existing theory, for the expected total discounted costs optimal policies exist in the forms of randomized stationary and switching stationary policies. While the former is typically unique, the latter forms a finite set whose number of elements grows exponentially with the number of constraints. This paper investigates the problem when the process stops immediately after the first jump. For costs up to the first jump we provide an index for selection of actions by switching stationary policies and show that the indexed switching policy achieves a smaller variance than the randomized stationary policy. For problems without instant costs, the indexed switching policy achieves the minimum variance of costs up to the first jump among all the equivalent switching policies. [ABSTRACT FROM AUTHOR]

We show that the optimal decision policy for several types of Bayesian sequential detection problems has a threshold switching curve structure on the space of posterior distributions. This is established by using lattice programming and stochastic orders in a partially observed Markov decision process (POMDP) framework. A stochastic gradient algorithm is presented to estimate the optimal linear approximation to this threshold curve. We illustrate these results by first considering quickest time detection with phase-type distributed change time and a variance stopping penalty. Then it is proved that the threshold switching curve also arises in several other Bayesian decision problems such as quickest transient detection, exponential delay (risk-sensitive) penalties, stopping time problems in social learning, and multi-agent scheduling in a changing world. Using Blackwell dominance, it is shown that for dynamic decision making problems, the optimal decision policy is lower bounded by a myopic policy. Finally, it is shown how the achievable cost of the optimal decision policy varies with change time distribution by imposing a partial order on transition matrices. [ABSTRACT FROM AUTHOR]

This paper is concerned with finding the best partial differential equation-based denoising process, out of a set of possible ones. We focus on either finding the proper weight of the fidelity term in the energy minimization formulation or on determining the optimal stopping time of a nonlinear diffusion process. A necessary condition for achieving maximal SNR is stated, based on the covariance of the noise and the residual part. We provide two practical alternatives for estimating this condition by observing that the filtering of the image and the noise can be approximated by a decoupling technique, with respect to the weight or time parameters. Our automatic algorithm obtains quite accurate results on a variety of synthetic and natural images, including piecewise smooth and textured ones. We assume that the statistics of the noise were previously estimated. No a priori knowledge regarding the characteristics of the clean image is required. A theoretical analysis is carried out, where several SNR performance bounds are established for the optimal strategy and for a widely used method, wherein the variance of the residual part equals the variance of the noise. [ABSTRACT FROM AUTHOR]

Many owners of growing privately held firms make operational and financial decisions in an effort to maximize the expected present value of the proceeds from an initial public offering (IPO). We ask: "What is the right time to make an IPO?" and "How should operational and financial decisions be coordinated to increase the likelihood of a successful IPO?" Financial and operational decisions in this problem are linked because adequate financial capital is crucial for operational decisions to be feasible and operational decisions affect the firm's access to financial resources. The IPO event is treated as a stopping time in an infinite-horizon discounted Markov decision process. Unlike traditional stopping-time models, at every stage the model includes other decisions such as production, sales, and loan size. The results include (1) characterization of an optimal capacity-expansion policy, (2) sufficient conditions for a monotone threshold rule to yield an optimal IPO decision, and (3) algorithmic implications of results in (1) and (2). [ABSTRACT FROM AUTHOR]