1. A Communication- and Memory-Aware Model for Load Balancing Tasks
- Author
-
Lifflander, Jonathan, Pebay, Philippe P., Slattengren, Nicole L., Pebay, Pierre L., Pfeiffer, Robert A., Kotulski, Joseph D., and McGovern, Sean T.
- Subjects
Computer Science - Distributed, Parallel, and Cluster Computing - Abstract
While load balancing in distributed-memory computing has been well-studied, we present an innovative approach to this problem: a unified, reduced-order model that combines three key components to describe "work" in a distributed system: computation, communication, and memory. Our model enables an optimizer to explore complex tradeoffs in task placement, such as increased parallelism at the expense of data replication, which increases memory usage. We propose a fully distributed, heuristic-based load balancing optimization algorithm, and demonstrate that it quickly finds close-to-optimal solutions. We formalize the complex optimization problem as a mixed-integer linear program, and compare it to our strategy. Finally, we show that when applied to an electromagnetics code, our approach obtains up to 2.3x speedups for the imbalanced execution.
- Published
- 2024