Search

Algorithms for system identification, estimation, and adaptive control in stochastic systems rely mostly on different types of signal averaging to achieve uncertainty reduction, convergence, stability, and performance enhancement. The core of such algorithms is various types of laws of large numbers that reduce the effect of noises when they are averaged. Many of the noise sequences encountered are often correlated and nonwhite. In the case of state estimation using quantized information such as in networked systems, convergence must be analyzed on double-indexed and randomly weighted sums of mixing-type stochastic processes, which are correlated with the remote past and distant future being asymptotically independent. This paper presents new results on convergence analysis of such processes. Strong laws of large numbers and convergence rates for such problems are established. These results resolve some fundamental issues in state observer designs with random sampling times, quantized information processing, and other applications. [ABSTRACT FROM AUTHOR]

Focusing on two-component Markov processes having continuous dynamics and discrete events, this work develops asymptotic properties of regime-switching diffusions. First, strong Feller property is established. Next, classifying the underlying processes as recurrence (positive recurrence or null recurrence) or transience, sufficient conditions for such processes are obtained. In addition to providing general criteria for recurrent and transient switching diffusions, for processes that are linearizable with respect to the continuous component, easily verifiable conditions are established. Also provided are conditions for null-recurrent processes. Furthermore, conditions for controlled switching diffusions in which the feedback controls ensure weak stability (or ergodicity) are given. Finally, simple examples are used for demonstration. [ABSTRACT FROM AUTHOR]

This work is concerned with rates of convergence of Markov chain approximation methods for controlled switching diffusions. The cost function is defined on an infinite horizon with stopping times and without discount. Displaying both continuous dynamics and discrete events, the discrete events are modeled by continuous-time Markov chains to delineate a random environment and other random factors that cannot be represented by diffusion processes. This paper presents a first attempt using a probabilistic approach to treat such rates of convergence problems. In addition, in contrast to the significant developments in the literature using partial differential equation (PDE) methods for the approximation of controlled diffusions, there do not yet appear to be any PDE results to date for rates of convergence of numerical solutions for controlled switching diffusions, to the best of our knowledge. Although some of the working conditions in this paper such as the one-dimensional continuous state variable, nondegenerate diffusions, and control only on the drift may be seemingly strong, they are adequate as the starting point for using this new approach to treat the rates of convergence problems. Moreover, in the literature, to prove the convergence using Markov chain approximation methods for control problems involving cost functions with stopping (even for uncontrolled diffusion without switching), an added assumption was used to avoid the socalled tangency problem. As a by-product of our approach, by modifying the value function, it is demonstrated that the anticipated tangency problem will not arise in the sense of convergence in probability and in the sense of L1. [ABSTRACT FROM AUTHOR]

In response to the increasing needs for control and optimization of hybrid systems, this work is concerned with such asymptotic properties as recurrence (also known as weak stochastic stability in the literature) and ergodicity of regime-switching diffusions. Using Liapunov functions, necessary and sufficient conditions for positive recurrence are developed. Then, ergodicity of positive recurrent regime-switching diffusions is obtained by constructing cycles using the associated discretetime Markov chains. [ABSTRACT FROM AUTHOR]

This work develops a filtering scheme for hybrid systems. The process dictating the configuration or regimes is a continuous-time Markov chain with a finite state space. Exploiting hierarchical structure of the underlying system, the states of the Markov chain are divided into a number of groups so that it jumps rapidly within each group and slowly among different groups. Focusing on reduction of computational complexity, the filtering scheme includes the following steps: (1) partition the state space of the Markov chain into subspaces, (2) derive a limit system in which the states are averaged out with respect to the invariant distributions of the Markov chain, (3) use the limit system to design quadratic variation test statistics, and (4) use the test statistics to identify which ergodic class the aggregated process belongs to and to construct near-optimal filter. For demonstration, a numerical example is presented. [ABSTRACT FROM AUTHOR]

A continuous-time version of the Markowitz mean-variance portfolio selection model is proposed and analyzed for a market consisting of one bank account and multiple stocks. The market parameters, including the bank interest rate and the appreciation and volatility rates of the stocks, depend on the market mode that switches among a finite number of states. The random regime switching is assumed to be independent of the underlying Brownian motion. This essentially renders the underlying market incomplete. A Markov chain modulated diffusion formulation is employed to model the problem. Using techniques of stochastic linear-quadratic control, mean-variance efficient portfolios and efficient frontiers are derived explicitly in closed forms, based on solutions of two systems of linear ordinary differential equations. Related issues such as a minimum-variance portfolio and a mutual fund theorem are also addressed. All the results are markedly different from those for the case when there is no regime switching. An interesting observation is, however, that if the interest rate is deterministic, then the results exhibit (rather unexpected) similarity to their no-regime-switching counterparts, even if the stock appreciation and volatility rates are Markov-modulated. [ABSTRACT FROM AUTHOR]

Presents objectives to reduce the complexity of the underlying problems associated with the weak convergence of hybrid filtering problems involving nearly completely decomposable hidden Markov chains. Transition of probability matrix of the Markov chain which can be viewed as a nearly decomposable one; Illustration of how a reduced system of filtering equations can be obtained by aggregating the states of each recurrent class into one state.