A Markov decision process for response adaptive designs

The randomized treatment allocation process in a response adaptive clinical trial is formulated as a stochastic sequential decision problem and an algorithm is proposed to approximate the optimal value under the average reward criterion. When the information of previous treatment allocations and associated responses are summarized with sufficient statistics for unknown parameters, the decision process becomes a Markov process, on which a span-contractor operator is defined. It is proven that the average reward under the policy identified from the span-contractor operator converges almost surely to the optimal value. Numerical results reveal that the sequential procedure based on the controlled Markov process shows superior ethical advantage and at the same time produces good statistical power for large sample sizes such as 200 or larger.