Search

A detailed probabilistic treatment is given of a birth-and-death process proposed by Williams (1969) in which the elements of the process bear up to s inanimate marks. Equations for the second-order moments of the process, and approximate marginal univariate solutions, are derived. The exact bivariate solution is given for the case s = 1. For general s the variance of the mark population is also derived. MARKED BIRTH-AND-DEATH PROCESS; MULTIVARIATE LAGRANGE DIFFERENTIAL EQUATIONS; MULTITYPE BRANCHING PROCESS Introduction In a recent paper Williams (1969) has considered the following process. We have a population of elements developing as a linear birth-and-death process and each element bears a number of inanimate marks. When a marked element dies then it and its marks are removed from the population; when a marked element divides, its marks segregate independently and without loss, each with probability I into one of the two daughter elements. It is assumed that marking an element does not affect its birth and death rates (cf. Clifford and Sudbury (1972)). Williams' process was proposed as a model of an experiment conducted on bacteria (the elements) infected by phage (the inanimate marks), by Meynell (1959). (The bacteria are lysogenic with respect to the phage, which means that the phage do not interfere with the behaviour of the bacteria.) His object was to use the marking process in order to separate the birth and death rates of the bacteria; if we denote birth and death rates by A and p respectively, then the mean-value measures on the bacterial population (assumed to develop as a linear birth-and-death process) will estimate the difference (A p). As the marks are inanimate, however, they will only be affected by p and so mean-value measures on the mark population enable us to estimate p (see Williams (1969)). The papers of Meynell and Williams taken together provide a most interesting example of a virological experiment and the probability model of that experiment. Received in revised form 10 August 1973 Research sponsored by a Science Research Council grant and a World Health Organisation research traineeship. 423 This content downloaded from 207.46.13.180 on Thu, 08 Sep 2016 05:01:40 UTC All use subject to http://about.jstor.org/terms 424 BYRON J. T. MORGAN Williams considered the mean-value solution of his model; this was adequate for Meynell's experiment in which population sizes were large. Here we shall give a fuller probabilistic statement of the model and make some contributions to its solution. Two differences from Williams' work lie in the adoption of an upper bound, s, to the number of marks any element can have, and the assumption (except where stated) of birth and death rates independent of time. We 'shall assume general initial conditions which, in Meynell's experiment, correspond to the distribution of phage over bacteria once these have been initially exposed to a phage population. Williams took this distribution to be Poisson; the distribution of phage over bacteria has, however, been the subject of much probabilistic study (e.g., see Gani (1965) and Morgan (1971)). Finally, let us note that the value s = I, on which we shall particularly concentrate, probably gives the most realistic description of Meynell's experiment. 2. The probability model The model is a continuous-time Markov process with discrete states. Let N,(t) be a random variable denoting the number of elements with i marks at time t, i = 0, 1,2, ..., s. Let n = (n0,n1, l , n.., ), Iu = (Uo, u1, *,us), N(t) = (No(t), N(t), --,N,(t)), and suppose N,(O) = ai, i = 0, 1,2, --, s. Let P(n; t) = P(N(t) = n), then corresponding to death we have the transitions, (No, ..., Ni, ---, N,) (No, --, Ni 1, , N,) at probability rate pNi, i = 0, 1,2, ,s, while corresponding to birth we have the following transitions (No, N,) ,(No + 1, .., N,) at probability rate, 2 =1 N 2-' +,ANo, (No, , Ni ,N2i,..., X (No, Ni + 2, , N2iI, at probability rate, This content downloaded from 207.46.13.180 on Thu, 08 Sep 2016 05:01:40 UTC All use subject to http://about.jstor.org/terms On the distribution of inanimate marks over a linear birth-and-death process 425 where we use [is] to denote the integral part of is, and (No, " , Ni, * * *, Nk i Nk, "', Ns) -*(No, ..., Nj + 1, ..., Nk-i + 1, ..., Nj 1, ..., Nl) at probability rate 2N 2-k' i = 1,2,...,C[(k 1)], k = 3,4,...,s (smaller k give transitions already considered above). It is readily verified that the sum of the transition rates is (A + yp) X = oN,, as required in the overall birth-and-death process. Introducing the operator Di, which increases the argument ni of a function of n by 1, we then deduce that the Kolmogorov forward equations (for all possible n) are, d S S (1) +(n; t) P(n; t)(A+ t)1 (n2 + D,P(n;t)(n +1) dt ==0 1=0

In a previous paper a branching Poisson process model was derived to explain deviations from a Poisson process in computer failure patterns. Physically the deviations arise because an attempt to repair a computer is not always successful and the failure recurs a relatively short time later. In this paper we discuss the implications of this model for the use and maintenance of computers.

This paper applies the theory of semi-Markov processes to the construction of a stochastic model for interpreting data obtained from clinical trials. The model characterizes the patient as being in one of a finite number of states at any given time with an arbitrary probability distribution to describe the length of stay in a state. Transitions between states are assumed to be chosen according to a stationary finite Markov chain.Other attempts have been made to develop stochastic models of clinical trials. However, these have all been essentially Markovian with constant transition probabilities which implies that the distribution of time spent during a visit to a state is exponential (or geometric for discrete Markov chains). Markov models need also to assume that the transitions in the state of a patient depend only on absolute time whereas the semi-Markov model assumes that transitions depend on time relative to a patient. Thus the models are applicable to degenerative diseases (cancer, acute leukemia), while Markov models with time dependent transition probabilities are applicable to colds and epidemic diseases. In this paper the Laplace transforms are obtained for (i) probability of being in a state at timet, (ii) probability distribution to reach absorption state and (iii) the probability distribution of the first passage times to go from initial states to transient or absorbing states, transient to transient, and transient to absorbing. The model is applied to a clinical study of acute leukemia in which patients have been treated with methotrexate and 6-mercaptopurine. The agreement between the data and the model is very good.

1. This paper was motivated by some questions of Barnett and Lewis (1967) concerning extreme winter temperatures. The temperature during the winter can be hopefully regarded as generated by a stationary Gaussian process superimposed on a locally U-shaped trend. One is interested in statistical properties of the minimum of sample paths from such a process, and of their excursions below a given level. Equivalently one can consider paths from a stationary process crossing a curved boundary of the same form. Problems of this type are discussed by Cramer and Leadbetter (1967), extensively in the trend-free case and in less detail when a trend is present, following the method initiated by Rice (1945). While results on moments are easy to obtain, explicit results for the actual probability distributions are not usually available. However, in the important case when the level of values of interest is far below the mean, the asymptotic independence of up-crossing times makes it possible to derive simple approximate distributions. (See Cramer and Leadbetter (1967) page 256, Keilson (1966).) There is a dearth of particular examples of processes and trends for which the distributions of interest are known exactly. Such examples could give useful experience of the form of distribution to be expected in typical cases, and could serve as material on which to test out approximate methods. The object of the present paper is to provide an example of this kind. One process for which exact results are available in the trend-free case is the Ornstein-Uhlenbeck process, i.e., the stationary Gaussian Markov process X(t) generated by

The paper represents a generalization of one of the main theoretical results of my Ph.D. thesis. The work is an outgrowth of work first begun by E. J. Hannan and a correct 'conjecture' by P. Whittle. The main theorem of this paper proves the existence of, and gives an explicit formula for, the asymptotic best linear predictor of a certain type of non-stationary multivariate time series from noise distorted observations. The non-stationarity arises from the fact that the signal satisfies a difference equation, which when considered as a polynomial, has only elementary divisors. The proof is accomplished by showing, through Hilbert space and harmonic analysis methods, that the generating function is a limit of the generating functions of the stationary analogue; that is, where the difference function has elementary divisors. Finally, it is shown that this asymptotic generating function exactly predicts null solutions to the difference equation. The proof is direct and due to E. J. Hannan.

A service unit I, with Poisson input and general service times is in series with a unit II, with negative-exponential service times. The intermediate waitingroom can accomodate at most k persons and a customer cannot leave unit I when the waitingroom is full. The paper shows that this system of queues can be studied in terms of an imbedded semi-Markov process. Equations for the time dependent distributions are given, but the main emphasis of the paper is on the equilibrium conditions and on asymptotic results. 1. Description of the model The system of queues, discussed in this paper, consists of two units. Customers arrive at a first unit (I) according to a homogeneous Poisson process of rate 2. Their service times in unit I are independent, identically distributed random variables with common distribution function H(.). We assume that H(.) has a positive, finite mean a and we will denote the Laplace-Stieltjes transform of H( ) by h(s), Re s 0. Upon completion of service in unit I, all customers go on to a second unit (II) via a finite waitingroom. We assume that there can be not more than k customers in unit II and in the waitingroom at any time. If upon completion of service in unit I a customer finds the waitingroom full, then the unit I blocks until a service in unit II is completed. At that time he is allowed to enter the waitingroom. We assume that the service times in unit II are independent, identically distributed random variables with a negative-exponential distribution with mean 1/a. The service times in unit II are also stochastically independent of those in unit I and of the arrival process. The case k = 1, i.e., when no customers can wait between the two units, was studied by B. Avi-Itzhak and M. Yadin[1], T. Suzuki [11] and N. U. Prabhu [7]. These authors allow the service times in the second unit to have a general distribution. In the case of general k, we impose the requirement that these service Received 3 July 1967. Research supported in part by Office of Naval Research Contract NONR 1100(26). 123 This content downloaded from 157.55.39.217 on Mon, 18 Apr 2016 07:16:32 UTC All use subject to http://about.jstor.org/terms

The paper considers an extension to the reversible counters system of Lampard [1]. In Lampard's model the input processes are two independent Poisson processes; this results in a gamma Markov sequence for the time-interval between successive output pulses and a negative binomial Markov sequence for the counts at the times of out-put pulses. We consider the input process to be a bivariate Poisson process and show that the first out-put process given above is not affected, while the second out-put-process becomes of a type studied in the theory of branching processes.

This paper contains a detailed study of the Poisson cluster process on the real line, concentrating on two aspects; first, the asymptotic distribution of the number of points in [0,t) as t→ ∞ for both transient and equilibrium cluster processes and, secondly, a general formula for the probability generating function of the equilibrium process. Asymptotic formulae for cumulants of the process are also derived. The results obtained generalize those of previous writers. The approach is analytical, in contrast to the probabilistic treatment of P. A. W. Lewis.

The paper employs methods of multitype branching processes to evaluate the probability of survival of mutable clones under environmental conditions which are unfavorable to the original parent of the clone. When other factors are taken to be constant, the long-term survival probability of a clone is implicitly demonstrated as a function of the intrinsic rate of mutation carried by this clone. The existence of a mutation rate which maximizes clone survival probability is shown and the effects of environmental deterioration on this optimal rate are studied. Finally, rigorous quantitative results are obtained for the classical situation of a Poisson distribution of offspring numbers. These results are then applied to the biological problem of indirect selection (Eshel (1972)).

This paper considers a finite dam with independently and identically distributed (i.i.d.) inputs occurring in a Poisson process; the special cases where the inputs are (i) deterministic and (ii) negative exponentially distributed are considered in detail. The instantaneous release trate is proportional to the content, i.e., there is an exponential fall in conten except when inputs occur. This model may arise in several other situations such as a geiger counter or integrated shot noise. The distribution of the number of inputs, and of the time, to first overflowing is obtained in terms of generating functions; in Case (i) the solution is obtained through recurrence relations involving iterated integrals which can be evaluated numerically, and in Case (ii) using a series solution of a second order differential equation. Numerical results, in particular for the first two moments, are obtained for various values of the parameters of the model, and compared with a large number of simulations. Some remarks are also made about the infinite dam. FINITE DAMS; POISSON INPUTS; EXPONENTIAL RELEASE; FIRST PASSAGE TIMES; RECURRENCE RELATIONS, NUMERICAL INTEGRATION; SERIES SOLUTION; SIMULATION

Analogues of the central limit theorem and iterated logarithm law have recently been obtained for the Galton-Watson process; similar results are established in this paper for the temporally homogeneous Markov branching process and for the associated increasing process consisting of the number of splits in the original process up to time t. CENTRAL LIMIT THEOREM; ITERATED LOGARITHM LAW; BRANCHING PROCESS; TEMPORALLY HOMOGENEOUS MARKOV PROCESS; SPLITTING PROCESS

In the present paper we deal with two models of the traffic on a divided highway. In both models traffic is flowing in one direction only, in two lanes, of which one (the left hand lane) is used only for overtaking. We suppose that vehicles enter the highway at the same entrance so that the instants at which a vehicle enters the highway form a homogeneous Poisson process, with density λ. Thus λ is the rate at which vehicles enter the highway per unit time. We suppose that there are no junctions, or exits (i.e. the highway extends in one direction to infinity).

This paper considers the autocorrelation function of queue length and the corresponding spectral density (i.e., its Fourier transform). Some general expressions are obtained using generating functions and matrices, and applied to M/M/1 and M [x]/M/∞ queues.

The present paper investigates the limiting distribution of the number of busy channels (queue size) and the remaining lengths of holding times at an epoch of departure for a loss system with general holding times and exponentially distributed interarrival times. Further, it is established that for this loss system in the limit an interdeparture interval length is independent of the queue size at the end of the interval and is distributed according to an exponential distribution with mean X-1. It is also seen that in the limit interdeparture times are mutually independent. LOSS SYSTEM WITH POISSON ARRIVALS, ERLANG'S FORMULA, LIMITING DEPARTURE PROCESS; REMAINING HOLDING TIMES; BUSY CHANNELS

The cars are considered as points on an infinite road with no intersections. They can overtake each other without any delay and they travel at constant speeds. These are independent identically distributed random variables also independent of the initial positions of the cars. The main purpose of the paper is the study of the asymptotic distribution for the number of overtakings (and/or meetings) in increasing rectangles in the time-road plane. Under the assumption of (weighted) Poisson distributed cars along the time-axis we deduce the asymptotic distribution of the standardized number of overtakings (and/or meetings) for large rectangles in the time-road plane. Lastly we shall indicate an application of the results.

Nonzero-sum N-person stochastic games are a generalization of Shapley's two-person zero-sum terminating stochastic game. Rogers and Sobel showed that an equilibrium point exists when the sets of states, actions, and players are finite. The present paper treats discounted stochastic games when the sets of states and actions are given by metric spaces and the set of players is arbitrary. The existence of an equilibrium point is proven under assumptions of continuity and compactness. NONCOOPERATIVE STOCHASTIC GAME; DISCOUNTED MARKOVIAN DECISION PROCESS; EQUILIBRIUM POINT; DYNAMIC PROGRAMMING

The paper considers the dam problem with Markovian inputs, with special reference to the serial correlation coefficient of the input process. An input model is proposed which by giving particular values to the parameters makes the stationary distribution of the inputs one of the standard discrete distributions. The probabilities of first emptiness before overflow are first obtained by using the Markovian analogue of Wald's Identity. From these, the stationary distributions of the dam content are obtained by a duality argument. Both, finite and infinite dams are considered. MARKOVIAN INPUT OF LINEAR REGRESSIVE KIND; MARKOVIAN ANALOGUE OF WALD'S IDENTITY; PROBABILITIES OF EMPTINESS; STATIONARY DISTRIBUTIONS OF DAM CONTENT; DEPENDENCE ON SERIAL CORRELATION COEFFICIENT

Our main concern in this paper is the convergence, as t → ∞, of the quantities i, j ∈ E; where Pij (t) is the transition probability of a semi-Markov process whose state space E is irreducible but not closed (i.e., escape from E is possible), and rj is the probability of eventual escape from E conditional on the initial state being i. The theorems proved here generalize some results of Seneta and Vere-Jones ([8] and [11]) for Markov processes.

Various formulas of Wald relating to randomly stopped sums have well known continuous-time analogs, holding in particular for Wiener processes. However, sufficiently general forms of most of these do not appear explicitly in the literature. Recent papers by Robbins and Samuel (1966) and by Brown (1969) provide general results on Wald's equations in discrete time and these are here extended (Theorems 2 and 3) to Wiener processes and other homogeneous additive processes, that is, continuous-time processes with stationary independent increments. We also give an inequality (Theorem 1) related to Wald's identity in continuous time, and we derive, as corollaries of Wald's equations, bounds on the variance of an arbitrary stopping time. The Wiener process versions of these results find application in a variety of stopping problems. Specifically, all are used in Hall ((1968), (1969)); see also Bechhofer, Kiefer, and Sobel (1968), Root (1969), and Shepp (1967).

SummaryA weir of capacity K is considered in which the water inflow is a process with stationary independent increments. Unless the weir is empty, there is a continuous release of water at unit rate; if K is finite the weir may become full in which case the excess water overflows instantaneously. A weir for which K is infinite will be referred to as infinite dam. For the latter the transient behaviour is well known if the input possesses a second moment (cf. e.g., Prabhu [7]) and serves as the starting point for the present paper. This result is first extended to yield the Laplace transform (L.T.) of the trivariate Laplace-Stieltjes transform (L.S.T.) of the content v(t) at time t, the input X(t) in (0, t) and the total time d(t) in the interval (0, t) during which the dam is dry. (Incidentally, the last two quantities, for relevant time intervals, will be carried throughout.) Then we use a relation between the latter and the L.S.T. of the expected number of downward level y crossings of the v(t) process established in Roes [9]. Since the dam processes considered are Markov processes, we have therewith the L.S.T. of the renewal function of the renewal process imbedded at level y. From this, one finds the L.S.T.'s of first entrance and taboo first entrance times (for their definition see introduction). Next we calculate the first skip times for the infinite dam from the first entrance times and the L.T. of the L.S.T. of v(t). It is then a routine matter to determine the taboo first skip times. From the (taboo) first entrance and skip times we derive the first entrance times for the finite dam, which in turn lead to the renewal functions of the renewal processes imbedded in the finite dam content process v*(t) and hence to the transient behaviour of the finite dam.The advantage of the present approach over the one given in Roes [8] is that it is entirely probabilistic and avoids involved analytic arguments. As a result, the question of uniqueness of the solution does not arise, while more insight is obtained in the structure. The L.S.T. of several first entrance times and first skip times have been derived by Cohen [2] for compound Poisson input.

In mathematical models of phenomena occurring in the general areas of the engineering, biological, and physical sciences, random or stochastic equations appear frequently. In this paper we shall formulate a problem in telephone traffic theory which leads to a stochastic integral equation which is a special case of the Volterra type of the form where: (i) ω∊Ω, where Ω is the supporting set of the probability measure space (Ω,B,P); (ii) x(t; ω) is the unknown random variable for t ∊ R +, where R + = [0, ∞); (iii) y(t; ω) is the stochastic free term or free random variable for t ∊ R +; (iv) k(t, τ; ω) is the stochastic kernel, defined for 0 ≦ τ ≦ t < ∞; and (v) f(t, x) is a scalar function defined for t ∊ R + and x ∊ R, where R is the real line.

The analysis of an organization having various grades of employee has been undertaken in many different contexts. In 1961, for example, Young and Almond considered an institution (of undisclosed type) having six grades of staff. Gani, in 1963, considered ‘Australian Universities’ as an organization and students were graded according to their stage of study. The author used techniques similar to those of this paper in 1964 to analyze deterministically the age structure of the Australian Academy of Science. Indeed, the analyses have usually been deterministic.

A single server queueing system with Poisson input is considered. There are a finite number of types of customers and the service time of the nth customers depends on the types of the nth and the (n – l)th customers. The time dependence of the queue size process will be studied, (it will be clear how the methods of the paper can be applied to other processes of interest,) and limiting as well as transient results will be given.

This paper studies the dependency structure of the intervals between responses in the renewal inhibited Poisson process, and continues the author's earlier work on this type of process ((1970a), (1970b)). A new approach to the intervals between events in a stationary point process, based on the idea of an average event, is introduced. Average event initial conditions (as opposed to equilibrium initial conditions previously determined) for the renewal inhibited Poisson process are obtained and event stationarity of the resulting response process is established. The joint distribution and correlation between pairs of contiguous synchronous intervals is obtained; further, the joint distribution of non-contiguous pairs of synchronous intervals is derived. Finally, the joint distributions of pairs of contiguous synchronous and asynchronous intervals are related, and a similar but more general stationary point result is conjectured.

Let w(t), 0 ≦ t ≦ ∞, be a Brownian motion process, i.e., a zero-mean separable normal process with Pr{w(0) = 0} = 1, E{w(t 1)w(t 2)}= min (t 1, t 2), and let a, b denote the boundaries defined by y = a(t), y = b(t), where b(0) < 0 < a(0) and b(t) < a(t), 0 ≦ t ≦ T ≦ ∞. A basic problem in many fields such as diffusion theory, gambler's ruin, collective risk, Kolmogorov-Smirnov statistics, cumulative-sum methods, sequential analysis and optional stopping is that of calculating the probability that a sample path of w(t) crosses a or b before t = T. This paper shows how this probability may be computed for sufficiently smooth boundaries by numerical solution of integral equations for the first-passage distribution functions. The technique used is to approximate the integral equations by linear recursions whose coefficients are estimated by linearising the boundaries within subintervals. The results are extended to cover the tied-down process subject to the condition w(1) = 0. Some related results for the Poisson process and the sample distribution function are given. The procedures suggested are exemplified numerically, first by computing the probability that the tied-down Brownian motion process crosses a particular curved boundary for which the true probability is known, and secondly by computing the finite-sample and asymptotic powers of the Kolmogorov-Smirnov test against a shift in mean of the exponential distribution.

Some recent work on discrete time dam models has been concerned with special cases in which the input process is a Markov chain whose transition probabilities, p ij , are given by where A(·) and B(·) are probability generating functions (p.g.f.'s). In this paper we obtain some results for the general situation. The convergence norm of the matrix [p ij xj] is found and the results are used to obtain the p.g.f. of the first emptiness time. Distributions of the dam content are obtained and conditions are found for the existence of their limits. The p.g.f. of this distribution is so complicated that its identification in any special case is extremely difficult, or even impossible. Thus useful approximations are needed; we obtain a ‘heavy traffic’ limit theorem which suggests that under certain circumstances the limiting distribution can be approximated by an exponential distribution.

Summary This paper is concerned with a continuous time parameter Markovian sequential decision process, and presents a method which transforms a given continuous parameter problem into a discrete one. It is proved that the optimal stationary policy for the resulting discrete time parameter Markovian decision process is also the optimal stationary policy for the original continuous one, and vice versa. The resulting discrete parameter problem may be more easily solved than the continuous one by applying the linear programming method. A simple numerical example is presented.

In the present paper the solutions of two integral equations are derived. One of the integral equations dominates the mathematical description of the stochastic process { v n , n = 1,2, …}, recursively defined by K is a positive constant, τ 1, τ 2, …; Σ 1, Σ 2, …; are independent, non-negative variables, with τ 1, τ 2,…, identically distributed, similarly, the variables Σ 1, Σ 2, …, are identically distributed.

In this paper we discuss a counter system whose output is a stochastic point process such that the time intervals between pairs of successive events form a first order Markov chain. Such processes may be regarded as next, in order of complexity, in a hierarchy of stochastic point processes, to “renewal” processes, which latter have been studied extensively. The main virtue of the particular system which is studied here is that virtually all its important statistical properties can be obtained in closed form and that it is physically realizable as an electronic device. As such it forms the basis for a laboratory generator whose output may be used for experimental work involving processes of this kind. Such statistical properties as the one and two-dimensional probability densities for the time intervals are considered in both the stationary and nonstationary state and also discussed are corresponding properties of the successive numbers arising in the stores of the counter system. In particular it is shown that the degree of coupling between successive time intervals may be adjusted in practice without altering the one dimensional probability density for the interval lengths. It is pointed out that operation of the counter system may also be regarded as a problem in queueing theory involving one server alternately serving two queues. A generalization of the counter system, whose inputs are normally a pair of statistically independent Poisson processes, to the case where one of the inputs is a renewal process is considered and leads to some interesting functional equations.

The single server queueing system in which service time has probability differential npldt + o(dt) when the system contains n customers will be referred to as linearly state dependent (LSD). Such a system with Poisson arrivals has been extensively analysed both theoretically and numerically by Hadidi and Conolly (1969) employing key results for the waiting time (including service), and equivalent service time, published in Hadidi (1969). It is pointed out in the former paper that the set of differential-difference equations describing the state probabilities is formally identical with the corresponding set for M/M/oo a fact which permits known results to be utilized. It is in fact possible to pursue the analogy to obtain the results of Hadidi (1969) quite simply, in contrast to the rather laborious algebra which his method involves. This is the purpose of the present note.

In this paper we shall investigate point processes generated by random variables of the form 〈gi (Ti ]), i=± 1, ± 2, … 〉, where 〈Ti, i= ± 1, … 〉 is the set of arrival times from a (not necessarily homogeneous) Poisson process or mixture of Poisson processes, and 〈gi, i = ± 1, … 〉 is an independently and identically distributed (i.i.d.) or interchangeable sequence of random functions, independent of 〈Ti 〉.