Application of Markov Decision Processes for Modeling and Optimization of Decision-Making within a Container Port

In modern container terminals, efficiently managing the transit of the containers becomes more and more of a challenge. Due to the progressive evolution of container transport, traffic management within container ports is still an evolving problem. To provide adequate strategy for the increased traffic, ports must either expand facilities or improve efficiency of operations. In investigating ways in which ports can improve efficiency, this paper proposes a Markov Decision Process (MDP) for loading and unloading operations within a container terminal. The proposed methodology allows an easy modeling for optimizing complex sequences of decisions that are to be undertaken at each time. The goal is to minimize the total waiting time of quay cranes and vehicles, which are allocated to service a containership. In this paper, reinforcement learning, which consists of solving learning problems by studying the system through mathematical analysis or computational experiments, is considered to be the most adequate approach.