Adaptive Container Scheduling in Cloud Data Centers: A Deep Reinforcement Learning Approach

Cloud data centers rely on virtualization to run a diverse set of applications. Container technology allows for a more lightweight execution, in comparison with popular Virtual Machines. Efficient scheduling of containers is still challenging due to varying request arrival patterns, application-specific resource consumption and resource heterogeneity in physical servers. Besides, containers are also more prone to resource contention and performance interference. Cloud providers need to overcome these challenges with a goal in mind: maximize resource utilization to satisfy as many requests as possible. This paper introduces RLSched, a deep reinforcement learning-based (DRL) scheduler that is self-adaptive and automatically captures the resource usage dynamics in the data center. The scheduler is based on a decentralized actor-critic multi-agent architecture that enables for parallel execution and faster convergence. RLSched relies on an enhanced network model with action shaping, which filters invalid actions and prevents the agent to fall into a sub-optimal policy. The proposed scheduler is compared against other state-of-the-art DRL methods on a simulated data center environment based on real traces from Microsoft Azure. The results show faster convergence and higher number of containers placed per session.