Search

From 1972 to 1974, I was working on a PhD thesis entitled Multiple Server Queues with Service Time Depending on Waiting Time.The method of analysis was the embedded Markov chain technique, described in the papers [82] and [77]. My analysis involved lengthy, tedious deri- tions of systems of integral equations for the probability density function (pdf) of the waiting time. After pondering for many months whether there might be a faster, easier way to derive the integral equations, I?nally discovered the basic theorems for such a method in August, 1974. The theorems establish a connection between sample-path level-crossing rates of the virtual wait process and the pdf of the waiting time. This connection was not found anywhere else in the literature at the time. I immediately developed a comprehensive new methodology for deriving the integral equations based on these theorems, and called it system point theory. (Subsequently it was called system point method,or system point level crossing method: SPLC or simply LC.) I rewrote the entire PhD thesis from November 1974 to March 1975, using LC to reach solutions. The new thesis was called System Point Theory in Exponential Queues. On June 12, 1975 I presented an invited talk on the new methodology at the Fifth Conference on Stochastic Processes and their Applications at the University of Maryland. Many queueing theorists were present.

Peter Kall and János Mayer are distinguished scholars and professors of Operations Research and their research interest is particularly devoted to the area of stochastic optimization. Stochastic Linear Programming: Models, Theory, and Computation is a definitive presentation and discussion of the theoretical properties of the models, the conceptual algorithmic approaches, and the computational issues relating to the implementation of these methods to solve problems that are stochastic in nature. The application area of stochastic programming includes portfolio analysis, financial optimization, energy problems, random yields in manufacturing, risk analysis, etc. In this book, models in financial optimization and risk analysis are discussed as examples, including solution methods and their implementation. Stochastic programming is a fast developing area of optimization and mathematical programming. Numerous papers and conference volumes, and several monographs have been published in the area; however, the Kall and Mayer book will be particularly useful in presenting solution methods including their solid theoretical basis and their computational issues, based in many cases on implementations by the authors. The book is also suitable for advanced courses in stochastic optimization.

Objectives The current global environmental crisis has reinforced the need for developing flexible mathematical models to obtain a better understanding of environmental problems so that effective remedial action can be taken. Because natural phenomena occurring in hydrology and environmental engineering usually behave in random and probabilistic fashions, stochastic and statistical models have major roles to play in the protection and restoration of our natural environment. Consequently, the main objective of this edited volume is to present some of the most up-to-date and promising approaches to stochastic and statistical modelling, especially with respect to groundwater and surface water applications. Contents As shown in the Table of Contents, the book is subdivided into the following main parts: GENERAL ISSUES PART I PART II GROUNDWATER PART III SURFACE WATER PART IV STOCHASTIC OPTIMIZATION PART V MOMENT ANALYSIS PART VI OTHER TOPICS Part I raises some thought-provoking issues about probabilistic modelling of hydro­ logical and environmental systems. The first two papers in Part I are, in fact, keynote papers delivered at an international environmetrics conference held at the University of Waterloo in June, 1993, in honour of Professor T. E. Unny. In his keynote pa­ per, Dr. S. J. Burges of the University of Washington places into perspective the historical and future roles of stochastic modelling in hydrology and environmental engineering. Additionally, Dr. Burges stresses the need for developing a sound scien­ tific basis for the field of hydrology. Professor P. E.

Introduction to Stochastic Level Crossing Techniques describes stochastic models and their analysis using the System Point Level Crossing method (abbreviated SPLC or LC). This involves deriving probability density functions (pdfs) or cumulative probability distribution functions (cdfs) of key random variables, applying simple level-crossing limit theorems developed by the author. The pdfs and/or cdfs are used to specify operational characteristics about the stochastic model of interest. The chapters describe distinct stochastic models and associated key random variables in the models. For each model, a figure of a typical sample path (realization, i.e., tracing over time) of the key random variable is displayed. For each model, an analytic (Volterra) integral equation for the stationary pdf of the key random variable is created−by inspection of the sample path, using the simple LC limit theorems. This LC method bypasses a great deal of algebra, usually required by other methods of analysis. The integral equations will be solved directly, or computationally. This book is meant for students of mathematics, management science, engineering, natural sciences, and researchers who use applied probability. It will also be useful to technical workers in a range of professions.Key Features: A description of one representative stochastic model (e.g., a single-server M/G/1 queue; a multiple server M/M/c queue; an inventory system; etc.) Construction of a typical sample path of the key random variable of interest (e.g., the virtual waiting time or workload in queues; the net on-hand inventory in inventory systems; etc.) Statements of the simple LC theorems, which connect the sample-path upcrossing and downcrossing rates across state-space levels, to simple mathematical functions of the stationary pdf of the key random variable, at those state-space levels Creation of (usually Volterra) integral equations for the stationary pdf of the key random variable, by inspection of the sample path Direct analytic solution of the integral equations, where feasible; or, computational solutions of the integral equations Use of the derived stationary pdfs for obtaining operational characteristics of the model

This open access book focuses on both the theory and practice associated with the tools and approaches for decisionmaking in the face of deep uncertainty. It explores approaches and tools supporting the design of strategic plans under deep uncertainty, and their testing in the real world, including barriers and enablers for their use in practice. The book broadens traditional approaches and tools to include the analysis of actors and networks related to the problem at hand. It also shows how lessons learned in the application process can be used to improve the approaches and tools used in the design process. The book offers guidance in identifying and applying appropriate approaches and tools to design plans, as well as advice on implementing these plans in the real world. For decisionmakers and practitioners, the book includes realistic examples and practical guidelines that should help them understand what decisionmaking under deep uncertainty is and how it may be of assistance to them. Decision Making under Deep Uncertainty: From Theory to Practice is divided into four parts. Part I presents five approaches for designing strategic plans under deep uncertainty: Robust Decision Making, Dynamic Adaptive Planning, Dynamic Adaptive Policy Pathways, Info-Gap Decision Theory, and Engineering Options Analysis. Each approach is worked out in terms of its theoretical foundations, methodological steps to follow when using the approach, latest methodological insights, and challenges for improvement. In Part II, applications of each of these approaches are presented. Based on recent case studies, the practical implications of applying each approach are discussed in depth. Part III focuses on using the approaches and tools in real-world contexts, based on insights from real-world cases. Part IV contains conclusions and a synthesis of the lessons that can be drawn for designing, applying, and implementing strategic plans under deep uncertainty, as well as recommendations for future work.The publication of this book has been funded by the Radboud University, the RAND Corporation, Delft University of Technology, and Deltares.

Building upon the previous editions, this textbook is a first course in stochastic processes taken by undergraduate and graduate students (MS and PhD students from math, statistics, economics, computer science, engineering, and finance departments) who have had a course in probability theory. It covers Markov chains in discrete and continuous time, Poisson processes, renewal processes, martingales, and option pricing. One can only learn a subject by seeing it in action, so there are a large number of examples and more than 300 carefully chosen exercises to deepen the reader's understanding. Drawing from teaching experience and student feedback, there are many new examples and problems with solutions that use TI-83 to eliminate the tedious details of solving linear equations by hand, and the collection of exercises is much improved, with many more biological examples. Originally included in previous editions, material too advanced for this first course in stochastic processes has been eliminated while treatment of other topics useful for applications has been expanded. In addition, the ordering of topics has been improved; for example, the difficult subject of martingales is delayed until its usefulness can be applied in the treatment of mathematical finance.

This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data.This book consists of eight chapters. Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods for solving linear systems will be introduced for finding the stationary distribution of a Markov chain. The chapter then covers the basic theories and algorithms for hidden Markov models (HMMs) and Markov decision processes (MDPs).Chapter 2 discusses the applications of continuous time Markov chains to model queueing systems and discrete time Markov chain for computing the PageRank, the ranking of websites on the Internet. Chapter 3 studies Markovian models for manufacturing and re-manufacturing systems and presents closed form solutions and fast numerical algorithms for solving the captured systems. In Chapter 4, the authors present a simple hidden Markov model (HMM) with fast numerical algorithms for estimating the model parameters. An application of the HMM for customer classification is also presented. Chapter 5 discusses Markov decision processes for customer lifetime values. Customer Lifetime Values (CLV) is an important concept and quantity in marketing management. The authors present an approach based on Markov decision processes for the calculation of CLV using real data.Chapter 6 considers higher-order Markov chain models, particularly a class of parsimonious higher-order Markov chain models. Efficient estimation methods for model parameters based on linear programming are presented. Contemporary research results on applications to demand predictions, inventory control and financial risk measurement are also presented. In Chapter 7, a class of parsimonious multivariate Markov models is introduced. Again, efficient estimation methods based on linear programming are presented. Applications to demand predictions, inventory control policy and modeling credit ratings data are discussed. Finally, Chapter 8 re-visits hidden Markov models, and the authors present a new class of hidden Markov models with efficient algorithms for estimating the model parameters. Applications to modeling interest rates, credit ratings and default data are discussed. This book is aimed at senior undergraduate students, postgraduate students, professionals, practitioners, and researchers in applied mathematics, computational science, operational research, management science and finance, who are interested in the formulation and computation of queueing networks, Markov chain models and related topics. Readers are expected to have some basic knowledge of probability theory, Markov processes and matrix theory.

Although studies on multiobjective mathematical programming under uncertainty have been accumulated and several books on multiobjective mathematical programming under uncertainty have been published (e.g., Stancu-Minasian (1984); Slowinski and Teghem (1990); Sakawa (1993); Lai and Hwang (1994); Sakawa (2000)), there seems to be no book which concerns both randomness of events related to environments and fuzziness of human judgments simultaneously in multiobjective decision making problems. In this book, the authors are concerned with introducing the latest advances in the field of multiobjective optimization under both fuzziness and randomness on the basis of the authors'continuing research works. Special stress is placed on interactive decision making aspects of fuzzy stochastic multiobjective programming for human-centered systems under uncertainty in most realistic situations when dealing with both fuzziness and randomness. Organization of each chapter is briefly summarized as follows:Chapter 2 is devoted to mathematical preliminaries, which will be used throughout the remainderof the book. Starting with basic notions and methods of multiobjective programming, interactivefuzzy multiobjective programming as well as fuzzy multiobjective programming is outlined.In Chapter 3, by considering the imprecision of decision maker's (DM's) judgment for stochasticobjective functions and/or constraints in multiobjective problems, fuzzy multiobjective stochasticprogramming is developed. In Chapter 4, through the consideration of not only the randomness of parameters involved inobjective functions and/or constraints but also the experts'ambiguous understanding of the realized values of the random parameters, multiobjective programming problems with fuzzy random variables are formulated.In Chapter 5, for resolving conflict of decision making problems in hierarchical managerial orpublic organizations where there exist two DMs who have different priorities in making decisions, two-level programming problems are discussed. Finally, Chapter 6 outlines some future research directions.

This volume covers recent developments in the design, operation, and management of telecommunication and computer network systems in performance engineering and addresses issues of uncertainty, robustness, and risk. Uncertainty regarding loading and system parameters leads to challenging optimization and robustness issues. Stochastic modeling combined with optimization theory ensures the optimum end-to-end performance of telecommunication or computer network systems. In view of the diverse design options possible, supporting models have many adjustable parameters and choosing the best set for a particular performance objective is delicate and time-consuming. An optimization based approach determines the optimal possible allocation for these parameters. Researchers and graduate students working at the interface of telecommunications and operations research will benefit from this book. Due to the practical approach, this book will also serve as a reference tool for scientists and engineers in telecommunication and computer networks who depend upon optimization.

The Subject A little explanation is in order for our choice of the title Linear Opti- 1 mization (and corresponding terminology) for what has traditionally been called Linear Programming.Theword programming in this context can be confusing and/or misleading to students. Linear programming problems are referred to as optimization problems but the general term linear p- gramming remains. This can cause people unfamiliar with the subject to think that it is about programming in the sense of writing computer code. It isn't. This workbook is about the beautiful mathematics underlying the ideas of optimizing linear functions subject to linear constraints and the algorithms to solve such problems. In particular, much of what we d- cuss is the mathematics of Simplex Algorithm for solving such problems, developed by George Dantzig in the late 1940s. The word program in linear programming is a historical artifact. When Dantzig?rstdevelopedthe Simplex Algorithm to solvewhat arenowcalled linear programming problems, his initial model was a class of resource - location problems to be solved for the U.S. Air Force. The decisions about theallocationswerecalled‘Programs'bytheAirForce,andhencetheterm.

When I wrote the book Quantitative Sociodynamics, it was an early attempt to make methods from statistical physics and complex systems theory fruitful for the modeling and understanding of social phenomena. Unfortunately, the?rst edition appeared at a quite prohibitive price. This was one reason to make these chapters available again by a new edition. The other reason is that, in the meantime, many of the methods discussed in this book are more and more used in a variety of different?elds. Among the ideas worked out in this book are: 1 • a statistical theory of binary social interactions, • a mathematical formulation of social?eld theory, which is the basis of social 2 force models, • a microscopic foundation of evolutionary game theory, based on what is known today as ‘proportional imitation rule', a stochastic treatment of interactions in evolutionary game theory, and a model for the self-organization of behavioral 3 conventions in a coordination game. It, therefore, appeared reasonable to make this book available again, but at a more affordable price. To keep its original character, the translation of this book, which 1 D. Helbing, Interrelations between stochastic equations for systems with pair interactions. Ph- icaA 181, 29–52 (1992); D. Helbing, Boltzmann-like and Boltzmann-Fokker-Planck equations as a foundation of behavioral models. PhysicaA 196, 546–573 (1993). 2 D. Helbing, Boltzmann-like and Boltzmann-Fokker-Planck equations as a foundation of beh- ioral models. PhysicaA 196, 546–573 (1993); D.

Managing uncertainty in new product development projects for improved valuation and decision making is one of the most complex and challenging problems in operations management. It is important for any corporation depending on the success of new products and innovations. This work shows how uncertainty can be handled and partly resolved by conducting an information update during the development process. It is one of the first comprehensive models that combine statistical decision theory in form of Bayesian analysis with a real options framework for projects exposed to different sources of uncertainty. The proposed framework makes an important theoretical contribution in addressing this problem, while at the same time being of significant value to managers who face the difficult task of evaluating and managing complex product development projects.

Advances in Queueing Theory and Network Applications presents several useful mathematical analyses in queueing theory and mathematical models of key technologies in wired and wireless communication networks such as channel access controls, Internet applications, topology construction, energy saving schemes, and transmission scheduling. In sixteen high quality chapters, this work provides novel ideas, new analytical models, and simulation and experimental results by experts in the field of queueing theory and network applications. The text serves as a state-of-the-art reference for a wide range of researchers and engineers engaged in the fields of queueing theory and network applications, and can also serve as supplemental material for advanced courses in operations research, queueing theory, performance analysis, traffic theory, as well as theoretical design and management of communication networks.

Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.

Markov decision processes (MDPs), also called stochastic dynamic programming, were first studied in the 1960s. MDPs can be used to model and solve dynamic decision-making problems that are multi-period and occur in stochastic circumstances. There are three basic branches in MDPs: discrete-time MDPs, continuous-time MDPs and semi-Markov decision processes. Starting from these three branches, many generalized MDPs models have been applied to various practical problems. These models include partially observable MDPs, adaptive MDPs, MDPs in stochastic environments, and MDPs with multiple objectives, constraints or imprecise parameters. Markov Decision Processes With Their Applications examines MDPs and their applications in the optimal control of discrete event systems (DESs), optimal replacement, and optimal allocations in sequential online auctions. The book presents four main topics that are used to study optimal control problems: a new methodology for MDPs with discounted total reward criterion; transformation of continuous-time MDPs and semi-Markov decision processes into a discrete-time MDPs model, thereby simplifying the application of MDPs; MDPs in stochastic environments, which greatly extends the area where MDPs can be applied; applications of MDPs in optimal control of discrete event systems, optimal replacement, and optimal allocation in sequential online auctions. This book is intended for researchers, mathematicians, advanced graduate students, and engineers who are interested in optimal control, operation research, communications, manufacturing, economics, and electronic commerce.

Due to the rapidly increasing need for methods of data compression, quantization has become a flourishing field in signal and image processing and information theory. The same techniques are also used in statistics (cluster analysis), pattern recognition, and operations research (optimal location of service centers). The book gives the first mathematically rigorous account of the fundamental theory underlying these applications. The emphasis is on the asymptotics of quantization errors for absolutely continuous and special classes of singular probabilities (surface measures, self-similar measures) presenting some new results for the first time. Written for researchers and graduate students in probability theory the monograph is of potential interest to all people working in the disciplines mentioned above.

A number of methodologies have been employed to provide decision making solutions to a whole assortment of financial problems in today's globalized markets. Hidden Markov Models in Finance by Mamon and Elliott will be the first systematic application of these methods to some special kinds of financial problems; namely, pricing options and variance swaps, valuation of life insurance policies, interest rate theory, credit risk modeling, risk management, analysis of future demand and inventory level, testing foreign exchange rate hypothesis, and early warning systems for currency crises. This book provides researchers and practitioners with analyses that allow them to sort through the random'noise'of financial markets (i.e., turbulence, volatility, emotion, chaotic events, etc.) and analyze the fundamental components of economic markets. Hence, Hidden Markov Models in Finance provides decision makers with a clear, accurate picture of core financial components by filtering out the random noise in financial markets.

The main purpose of the book is to give a rigorous, yet mostly nontechnical, introduction to the most important and useful solution methods of various types of stochastic control problems for jump diffusions and its applications. The types of control problems covered include classical stochastic control, optimal stopping, impulse control and singular control. Both the dynamic programming method and the maximum principle method are discussed, as well as the relation between them. Corresponding verification theorems involving the Hamilton-Jacobi Bellman equation and/or (quasi-)variational inequalities are formulated. There are also chapters on the viscosity solution formulation and numerical methods. The text emphasises applications, mostly to finance. All the main results are illustrated by examples and exercises appear at the end of each chapter with complete solutions. This will help the reader understand the theory and see how to apply it. The book assumes some basic knowledge of stochastic analysis, measure theory and partial differential equations. In the 2nd edition there is a new chapter on optimal control of stochastic partial differential equations driven by Lévy processes. There is also a new section on optimal stopping with delayed information. Moreover, corrections and other improvements have been made.

This book is intended as an introduction to optimal stochastic control for continuous time Markov processes and to the theory of viscosity solutions. Stochastic control problems are treated using the dynamic programming approach. The authors approach stochastic control problems by the method of dynamic programming. The fundamental equation of dynamic programming is a nonlinear evolution equation for the value function. For controlled Markov diffusion processes, this becomes a nonlinear partial differential equation of second order, called a Hamilton-Jacobi-Bellman (HJB) equation. Typically, the value function is not smooth enough to satisfy the HJB equation in a classical sense. Viscosity solutions provide framework in which to study HJB equations, and to prove continuous dependence of solutions on problem data. The theory is illustrated by applications from engineering, management science, and financial economics. In this second edition, new material on applications to mathematical finance has been added. Concise introductions to risk-sensitive control theory, nonlinear H-infinity control and differential games are also included. Review of the earlier edition:'This book is highly recommended to anyone who wishes to learn the dinamic principle applied to optimal stochastic control for diffusion processes. Without any doubt, this is a fine book and most likely it is going to become a classic on the area....'SIAM Review, 1994